There probably is no neutral at the switch box. A switch with "one set of wires" (I am guessing you mean one cable - pictures or a clear description would help me be accurate) is almost certainly a "switch loop." Under newer codes that should be a 3-wire (red-black-white-ground - ground is not counted in naming the cable, so "3-wire") cable to supply a neutral to the switch box, but under older codes a 2-wire black-white-ground cable was often used.
One wire (should be black) will be the hot to the switch. The other (should be red on the 3-wire, or on 2-wire the white wire which SHOULD have red tape or paint applied to it) will be the switched hot, from the switch, back to the first fixture. The neutral for the circuit will be at the fixture in this case.
(Pardon - North American color codes - other parts of the world do have other conventions, involving blue, brown, etc...)
Likely what you need to do is move the new light's wire to one of the other lights, not the switch.
Edit, Add - what you probably are doing is putting the one new light in series with all the other lights in parallel between it and neutral. If you remove the bulbs in all the other lights, odds are good that the new one will go out. If you remove most of them, you'll probably find that the new fixture gets dimmer, and the remaining lights on the circuit start to glow a bit...all the lights should be in parallel (from hot to neutral) but what you have now is hot, the one light, an intermediate connection from the one light to all the other lights, and through those other lights to neutral.
Best Answer
It is not possible to power a recessed light from the existing switch box. There is no neutral in this box, but only a line hot (always hot from the box for the existing receptacle) and a switched hot going to the existing receptacle.
The white wire in the switch box is not a neutral, but instead is being used to carry either the line hot or the switched hot from the switch to the receptacle.
Our tract house is wired this way and we have lamps plugged into the switched receptacle in each bedroom. I have considered getting a plug-in wall mounted lamp in one room (to replace a floor lamp) but haven't done it yet.