Why is the furnace blowing hot air outside

Can you look at the furnace during the time it is blowing cold air and see if the flame is on?

The thermostat has the ability to control the fan, but typically the thermostat sends a "turn heat on" signal to the furnace and the furnace controls are setup to turn the flame on for a X amount of time (to warm up the air) and then turn on the fan once the timer expired (so the air coming out of the vents is not cold). When the heat is turned on is the flame and fan going on at the same time?

Not sure about your furnace, but some have a couple of LEDs that are lit up and can be used to read a status or error codes. If you have the manual you could try to decipher the code while the cold air is running and see if it gives you a clue.

I would start by trying to figure out if the problem is with the thermostat or the furnace by installing a new thermostat and seeing if the problem goes away. If the problem is still there you can just return it. I had a furnace issue where it was going off randomly and would work in heat mode but not cool mode. It turned out that the thermostat was bad and sending intermittent signals.

It's a high efficiency gas furnace, and that's the exhaust vent. DO NOT cover or plug it!

A normal furnace relies on the stack effect, to carry hot exhaust up and out of the building. Since high efficiency furnaces extract most of the heat from the exhaust, the exhaust must be forced out. Since it takes less energy to move the cooled exhaust horizontally, HE furnaces typically exhaust to the side of a building rather than the top. As the gases are cooler, PVC, CPVC, or ABS piping can be used to carry the exhaust to the outside.

In a normal furnace you'll typically only see one motor and fan (blower), which is used to draw in cool air and blow warm air throughout the house. In an HE furnace you'll also find a draft inducer or purge motor and fan, which is used to draw in combustion air and push out exhaust.