The flexible armored wire you are seeing is often referred to as AC or BX cable. (BX was a brand name like Romex is for NM) Bx was used extensively prior to 1960ish and is still used today. There are no current code requirements that require this kind of cable in residential construction and it has almost all but been replaced with NM. As you have probably already figured out, BX uses different box connectors, and many of the device boxes you have in place were probably made to terminate BX cable.
Although it is perfectly legal to use, and actually quite safe, it is rarely used because of it's higher cost and relative difficulty to install and terminate. The only caution I would emphasize is the ground conductor in older BX. it was very common not to use a separate ground conductor when using BX, and often when there was a separate ground wire, it was a smaller AWG that the black/white/red current carrying conductors. this was never a problem if the BX armor was properly attached to the metal junction boxes and again properly connected at the panel, thus making a substantial grounding path.
As far as the bonus question. BX is heavy and probably brings a good price at the metal recycling brokers. A couple of years ago when junk metal was really hot, a good load would have been worth hundreds of dollars. It was a popular find for junk dealers and the target of theft.
For a wired phone (or the base of a cordless), it is most likely caused by having the phone wiring running directly next to electrical wiring. Since it only happens when this light is on, most likely it's the wire going from the switch to the light fixture - though it could also be caused by a run somewhere else. In general you don't want any communication (telephone, network, audio, speakers) running in parallel near by to any AC wiring, as you'll get induced hum. Always keep them at least 6" apart, or when they do have to cross, make them cross perpendicular (90 degree angle) to each other.
For an older cordless phone, it could be that your phone is operating on a frequency harmonic of 60 Hz (which is what the AC line and ballast are working on -- 50 Hz in europe). Try changing channels, and see if that makes the problem go away. Note that even with cordless phones, where the base station is connected can be subject to the same problems as any wired phone.
If it's a wiring problem, then it's best to disconnect and re-run that wire from somewhere else that isn't right next to AC wiring.
If it's not wiring, or you can't re-do wiring, one of the easiest fixes may just be to get a new cordless phone. Make sure it's a DECT phone (most are, these days) as those are frequency hopping, use digital communications, and are compatible with other wireless devices like WiFi (802.11a/b/g). You can get multi-handset cordless phones for <$100, and the nice thing is you can locate the base station anywhere near a phone line, and have a couple remote handsets with just a charger base that plugs into the wall, no phone wiring necessary.
Note that even if you go the cordless route, you may still get hum induced on the line by this particular wire, so you may need to disconnect the phone line that runs into the kitchen from the rest of your phone wiring.
Best Answer
From your back side picture it is apparent that no small gauge low voltage wiring goes to the box with the four prong plug. So unlikely that the plug was used for wired phone service at any time in the near history.
It is almost sure that someone at one time wanted to power some ill conceived device with power through a four prong plug. If you open up that box you will undoubtedly find that some pins on that plug are wired into the power circuit that passes through that electrical box.
Sometimes it is just hard to understand some of the schemes that folks will dream up and cobble together, whether it is safe or not.