Learn English – encapsulating a positive thing among many negative things

You could try "cut off your nose to spite your face".

This is used where you bring a lot of trouble on yourself pursuing a minor victory, such as getting even with somebody. As it has this 'angry' aspect, it might not be exactly the same as throwing the baby out with the bathwater, but I can't think of a closer one.

You're absolutely right.

A quick google suggests that this is a commonly misquoted phrase.

The phrase should be 'Each day is better than the last'.