How to prevent heat loss through heater/air conditioner wall units

air-conditioningcoolingheaterheatinghvac

HVAC experts out there…

I live in Wisconsin and it gets mighty cold here! I have a condo with 2 wall units that are used for both air conditioning and heat. My heat bill in the cold winter months can get up to near $300 per month (normally my bill is about $100 +/- $10 during the spring/fall and is about $200ish in the summer with the air on). If I keep one of the units on in the living room and turn the one in the bedroom off, my electricity bill is higher. It seems strange but true. If I have both running, they turn on and turn off every 10-15 minutes. They work with my thermostat and whatever it's set to in each room. Whether I have a unit turned off or not, and go next to it, I feel an extreme cold air coming in through it's vent. I think when it turns off (when it heats the room up to the desired temp), that area cools down and it was like not heating it at all. Seems to be a bad vicious cycle.

I guess I'm looking for advice to prevent that heat loss, or to prevent the cold air from coming in from outside. It's about 2 degrees outside. If you want pictures (or make/model numbers) let me know.