Drafty windows on sides – how to seal

An outside mount with no sill would at least give you a drip edge there. It's just installation step one: rest on sill. So be careful trying to mount an exterior one. For an inside mount, remove and measure the screen. Have them replicated as storms and hold them in with clips to promote annual use. Caulk it shut if you must. And the rest of your trim while you're at it (the other half of your problem). You may need to install backstops for the new window as your screen is held in place by butterfly latches.

I highly suggest dropping that cash on completely new windows which would be operable and more efficient; inherently greener. (saves you money)

If you caulked it and there's still a leak, you didn't caulk the whole gap. Air isn't going to come through silicone caulk. For such a small gap, expanding foam is overkill and may do more harm than good if it expands too much and widens the gap, which could cause the window to start sticking.

Another possibility is that there are additional gaps and holes in the area. Windows--especially old ones--are notorious sources of air leaks. Caulk the heck out of every gap you can find. If the windows have wood trim, caulk that to the drywall and the windows using color-matched or clear caulk. If you really want to do a thorough job, you can temporarily remove the trim and seal the exposed area between the rough opening and the window with expanding foam--be sure to get the kind that Tester101 recommended for this--the low-expansion window & door foam--and then replace the trim. And if your windows are of the very old variety that have poor seals, add stick-on compressible foam gaskets to the sliding sashes that contact the frame. You'd be surprised how much that can help.