Could we solve the problem by voting our current legislators out of office if we don’t like what they’re doing?

First, individual legislators aren’t the problem; the current process is the problem. Second, gerrymandering is a problem exactly because it makes politicians less accountable to voters. 

The maps are drawn with careful attention to voter registration and voting histories in each district, and that tends to secure the districts for the incumbents, making them less responsive to what the voters may think about individual issues. That’s why so many legislators are re-elected without any opposition in the general elections—potential opponents don’t have a chance to win.

More about the problem