If you've replaced most of your lightbulbs with CFL or LED bulbs, they present a more inductive load to your power line than incandescent bulbs. This inductance, usually caused by a power converter inside the bulb, doesn't like it when the switch opens.
This inductance will actually force current through the switch as it opens, causing a much larger arc than would normally be expected from incandescent bulbs.
You'll also see this on circuits using 12V lighting (some track lighting and other specialized lighting).
The arcing is safely contained inside the switch, but it will wear the switch down over time, presenting a greater resistance in the power path and heating up. While it will never pose a fire hazard (as long as it's UL rated or similar) it may eventually fail.
If you want to avoid the noise associated with the arcing, get a dimmer that is compatible with all the lamps on the circuit and make sure they are dimmable. The dimmer switches the load electronically, and even if you never use the dimming feature the arcing will not occur. However, the same issues are still present and dimmers under these circumstances will wear out more quickly and fail just as the switches would.
You could also replace the lamps and lamp styles to reduce the arcing. Remove all the lamps, check to make sure there's no arcing (if there is, there are other loads you need to remove, or you have low voltage lighting) and then put each lamp in one by one, testing between each lamp. You may find that it takes a certain number of lamps (in which case a 20 amp switch might resolve the problem, even though the circuit may only be 15 amps), or certain types or styles of lightbulbs or lamps are the problem.
Best Answer
It's probably just a broken switch. My guess would be that the switch contact is worn out to the point that it does not make contact, but jiggling it a bit (by turning off and on again) may solve the problem, at least for a little while.
Replacing a light switch is a pretty simple matter for a DIYer. I recommend swapping it out and seeing if that solves the problem.