For this to happen there must be some voltage to the light socket. It may not be noticeable with a standard incandescent bulb because it apparently is not the full voltage and current. If you are comfortable in what you are doing, you could measure the voltage across the two wires and ground to see what you get. Incorrect wiring could be the problem, such as an improper shared neutral leg, or the neutral being switch instead of the hot leg. This can be found using a voltmeter and referencing to a known ground. If your not sure, leave it to an electrician. Wiring is not a hobby, and can be lethal!
The answer is not a cut and dry one in your case. The troubleshooting is a process. You will need an understanding of a multiple light parallel circuit. You will also need a proximity type voltage tester and possibly a VOM.
Assuming the voltage feed starts at the switch (not always the case, but normal) you will need to verify input voltage at the line side of the switch with the switch in the off position. If you have voltage there, turn the switch on and verify voltage on the load side of the switch. At this point, a VOM is handy to test voltage across the hot and neutral and/or ground. If this looks good, proceed to the closest light fixture, and with the switch on, test the center hot tab in each fixture with the no-touch tester. In your case I would think you may not see any voltage based on your question. This could mean that you have an open neutral, an open hot, or at worse, a shorted hot to neutral/ground.
Since there are several possible reasons for your condition, it would take a lot of tutorial to explain every possible scenario. Assuming you do not see voltage at the fixtures, the basic technique for troubleshooting will be to start at the last known verified voltage point, then follow the wiring and check all connections in the junction boxes. Visually check the bulb sockets for broken or shorted metal tabs. This should be done with the power off at first, looking for obvious loose or disconnected wires, then with the power on using your voltage tester. Since this condition occurred after you adjusted the height of the sockets in the fixtures, I suspect tension on some wire may have pulled a wire out of a socket base or out of a wirenut in the j-box. Unfortunately, most can/pot lights have a built in j-box attached to the top of the fixture. This necessitates dropping the fixture down below the ceiling to access the j-box, or getting access from above. (attic).
Basically, you are following the circuit looking for an open or short, just like following a hose, looking for a water leak or stoppage.
This is not a hard job, but extreme caution must be taken when testing energized circuits. If you do not have good electrical skills, the proper test equipment, or a logical understanding of switched paralleled circuits, then this job is better left to a pro.
Maybe one of my buddies here on SE can add a good graphic showing this type of circuit and the test points. A simple line drawing showing the switch and junction points would be a great edit.
Best Answer
If the last light goes out when the second to last is removed, the wiring error may be:
Where electricity travels through the second to last bulb, on it's way to the last bulb. If so the last two bulbs may also have been dimmer than normal (if they are incandescent).
The bulb type matters here: using CFL or LED lights during testing can add to the confusion.