Primer dries fast, so 50's will probally be OK, especially if it gets some direct sunshine. Give it some time however, before painting. I have used exterior paints as low as the 40's as long as night time temps are not going much lower, or paint/primer at least 4 hours before temps drop a bit. Some paints do give a temperature range. There are low temperature paints that will cure down to upper 30's with no problem. The main problem with priming or painting in cooler temps, is that you must let it cure enough before applying the next coat. That may take a couple of moderate temp days as it will not cure in the colder nights, dew or excessive moisture, as that will delay the curing process or discolor the finish.
In my experience in painting my own house over 50 years I only sand when necessary, that is when the paint is loose, or, the surface is glossy. I just repainted some trim on my house that has Semi-Gloss paint. And, I sanded it first. I did this to improve the adhesion of the new paint. But, most of my house has flat paint, and I never sand it unless I'm blending an area where I had to remove some loose paint.
Update, 11/14/13: I re-read this today, including all the posts. It's interesting to see the different opinions about this subject. No wonder you had a question after searching the net.
My take on this is that if you are a perfectionist, you'll decide to remove it all. If you're just painting to resell, you'll scrape the loose off and paint. If you're going to keep it and use common sense, you'll scrape, sand where you need to feather edges for aesthetics, and paint. That's assuming its flat paint. For glossy finishes, you really better sand it to ensure the new paint sticks permanently. Incidentally, I always clean before scraping with Tide and bleach solution and brush hooked to expandable pole, and sometimes with a power washer to quickly knock off loose paint. Also, after sanding, I blow off the sanded surfaces with a blower, or brush it with a big paintbrush, or wipe it with rubbing alcohol or water.
Recently I used the tool below to "sand" the loose paint off my deck before repainting and was very impressed with it's efficiency. I like how it gets down into the cracks between the paint and wood and smooths out the ridges, as well as how quickly its use is. You might want to pick one of these up and give it a try.
Finally, one last tip. You can always do a test by painting a small area, waiting several days, then trying to scrape it off. That's how I learned how important sanding is for glossy paint.
Best Answer
Personally, I feel safer on an extension ladder leaned up against the house, compared to a free standing step ladder. They topple too easily. But you do what makes you feel safest. What kind of wood siding do you have? If it's horizontal like ship lap, I would use a brush. I tried spraying mine once and in the process of trying to get paint up into the cracks under each board, the paint built up and ran. So I ended up going over it immediately with a brush anyway. Eventually I stopped bothering with the sprayer. But if it is vertical siding, spraying is fine.