Rooms above garages are often done improperly. It's amazing, because it really isn't all that difficult, but a lot of people seem to mess it up.
Unfortunately, there's no "easy" way: You're going to have to remove some drywall to inspect and see how the insulation in the ceiling is done. There may or may not be a gap in the space, depending on how the ceiling was done. Even just taking the temperature above the insulation in that space will tell you a lot (above the insulation, it should be close to room temperature).
Drafts in that space are absolutely killer, it means there is a direct source of outside air. Black on the insulation is a good sign of airflow, indicating drafts.
![enter image description here](https://i.stack.imgur.com/06sr3.jpg)
Basically, if the ceiling space isn't totally sealed, the only thing you can do is turn the garage into a conditioned space (very expensive, in terms of both construction costs and ongoing energy costs), and even then, if the problem is between the insulation and sub-floor, it won't help.
What should be there at a minimum is a continuous vapour barrier, with a layer of insulation on top. The vapour barrier should be sealed to the subfloor or the vapour barrier from the walls above. The insulation should go all the way around the edge (in the headers), so the insulation is continuous from the floors up to the subfloor. There should absolutely be no drafts, exposed concrete, or anything directly connected to the outside that is uninsulated.
![enter image description here](https://i.stack.imgur.com/YDiMK.jpg)
The best way (in my opinion) to seal this space is to use closed-cell spray foam, and form a continuous seal across the bottom of the entire subfloor. This gives you great insulation, no drafts, and acts as a vapour barrier as well (vapour barrier is not needed in this case -- though some building inspectors still don't understand this, so check your local codes). It also is better at ensuring fumes from vehicles in the garage can't get into your living space (in theory, vapour barrier prevents this as well, but 6mil vapour barrier is easier to puncture than a couple inches of hardened foam).
![enter image description here](https://i.stack.imgur.com/pgLYK.jpg)
Though you may be able to find something obvious and fix it, be prepared that there's a possibility that the only "fix" is going to be to completely tear down the ceiling of the garage and re-do it properly.
If you're going to spend money and time on this, do it right.
Best Answer
If you are considering separating the alcove in the first picture into a separate room, you may have a problem with building codes, and more importantly, safety. Even if the alcove has access to the hallway, it does not appear to have a window, which may be considered a hazard, both as to air and alternate egress in the case of a fire or other emergency.
You need to check with the housing or building authority before separating space.
You also may have a problem with your landlord, depending on what your lease says. Usually renters are not permitted to erect anything structural without permission.
If you are merely asking about a curtain for the archway, there are sound insulating curtains you can find online, but they are likely to reduce noise, not eliminate it.