The classic, olde-fashioned aproach used on our summer camp porch:
Short solid wall, screen above.
Large interior shutters, hinged at the top, ropes at the bottom, run over pulleys up in the roof, attached to cleats. Given purpose and era, those had small windows in them - you are looking for larger windows, principle would be the same, but you might want them a bit narrower if using glass (due to weight). Plastic glazing (polycarbonate twin-wall) would be much lighter - good plant light, not so good for views, not fragile.
The shutters were pulled up into the roof when summer came, and left there unless there were bad storms. The two at the corner overlapped and were rigged so that one went up first and the other had to follow it, or they wouldn't go. When they were down, a toggle at the bottom kept them from blowing and flapping about.
It's more fiddly to build than removable glass, but does simplify storage quite a bit. Height of shutter/window that opens can't be more than width of porch (but the solid wall portion could be glazed.)
On the third hand, if there's a good portion of non-window-area house wall on the back, you could store removable windows there against the house wall.
Responding to comment: I cannot find a picture of the old, simple system. The picture I can find that's closest is this more complicated bi-fold system, but if you imagine it with a single, unhinged window panel, and ropes/pulleys it's pretty close - it would just cover more ceiling, and be easier to build.
It's an intriguing question. On one hand, trapped air is heated through radiation. On the other, it acts as an insulator. Keep in mind that we aren't talking about airflow, but heat flow (by conduction, convection, and radiation).
The answer probably depends on how many of your windows are exposed to the sun, where extreme temperatures are generated between panes. In those cases, the insulation benefit of the second pane is probably outweighed by the efficiency lost due to greenhouse effect.
For those windows which are usually shaded, the second pane creates a trapped-air buffer, which dramatically reduces conduction loss.
I don't think a conclusion can be drawn with the information given. You might update your question to describe the building and property features in more detail. (Don't put it in a comment.)
Best Answer
I've used the plastic window stuff before (literally just finished putting up some a few minutes ago) and it works quite well. Another option to look into is storm windows, they go a long way to helping keep a house warmer. You can purchase storm windows in standard sizes at the usual hardware stores, or order custom fit ones if they don't have the ones you need.
There are also weatherstripping products you can get for your windows. Wired currently has a decent overview of what your options are.