That's right!

“You're throwing the baby out with the bath water.”

matches

“Don't get rid of the good with the bad.”