Those tone generators are fantastic tools for tracing data cabling and phone cabling - but they also have several modes on them.
One of the modes is for tracing the cable through the wall, so the device will beep if it's in the general vicinity, so that you can know where abouts it is without having to have contact with it. However, this is not so great for verifying which cable is the one you've got it hooked onto when you've got a bundle of cables.
For that, there's usually a 2nd mode which will only beep if you come into direct contact with the wire.
The first point of call would be making sure that you've got the tracer set to the correct mode so that it will only beep if you're in direct contact with the wire.
If it's still beeping and in that mode, then you've got a short somewhere, which will be why the cabling isn't working. It might be worth pulling it out and replacing it if you're in a situation where that is possible (your ADSL Sync Speeds might go up drastically if you do that too).
For a wired phone (or the base of a cordless), it is most likely caused by having the phone wiring running directly next to electrical wiring. Since it only happens when this light is on, most likely it's the wire going from the switch to the light fixture - though it could also be caused by a run somewhere else. In general you don't want any communication (telephone, network, audio, speakers) running in parallel near by to any AC wiring, as you'll get induced hum. Always keep them at least 6" apart, or when they do have to cross, make them cross perpendicular (90 degree angle) to each other.
For an older cordless phone, it could be that your phone is operating on a frequency harmonic of 60 Hz (which is what the AC line and ballast are working on -- 50 Hz in europe). Try changing channels, and see if that makes the problem go away. Note that even with cordless phones, where the base station is connected can be subject to the same problems as any wired phone.
If it's a wiring problem, then it's best to disconnect and re-run that wire from somewhere else that isn't right next to AC wiring.
If it's not wiring, or you can't re-do wiring, one of the easiest fixes may just be to get a new cordless phone. Make sure it's a DECT phone (most are, these days) as those are frequency hopping, use digital communications, and are compatible with other wireless devices like WiFi (802.11a/b/g). You can get multi-handset cordless phones for <$100, and the nice thing is you can locate the base station anywhere near a phone line, and have a couple remote handsets with just a charger base that plugs into the wall, no phone wiring necessary.
Note that even if you go the cordless route, you may still get hum induced on the line by this particular wire, so you may need to disconnect the phone line that runs into the kitchen from the rest of your phone wiring.
Best Answer
Yeah, that's a bad joke, I wonder who your phone company is...
If you insist on DIY... it does not particularly need to be waterproof - the splice connectors are filled with grease, so they are waterproof. Buy a plastic electrical box (no holes) and cut slots in the box part so you can get the cable in place without cutting it.
Don't make the slots too large. If you want to try for waterproof, use duct seal (gray electrical putty, non-hardening) around the cables in the slot. Screw the cover on, pack some more duct seal on from the outside.
If you choose to use self-fusing tape, choose from the electrical aisle (well, it may not be in your average hardware store/home center, so you might need an actual electrical supply store to find it), not late night TV or plumbing repair. The box is going to be more robust and easier to access.