If the joints were made up properly in the junction box then there is nothing much to do by removing the fixture. If the fixtures have been working for years is not very likely a wire joint has suddenly failed.
More likely you have a bad connection on the switch or need to recheck your wiring coming to the switch.
Check for power coming to and leaving the switch and power at the light.
Is this normal?
No, but it is not especially unusual.
Do I have a problem with my wiring setup?
Insufficient information to tell.
Do LEDs behave this way?
It is a common problem with some types of LED "bulb" in certain situations.
LED replacements for incandescent lights are relatively new so a lot of house wiring and accessories that people have already installed are designed to work with incandescent bulbs, not with LED replacement bulbs.
Some older accessories that need to draw a small amount of power at, for example, a switch location that has no neutral (only live and switched live) do so by drawing a tiny current through the light bulbs controlled by the switch, the current is too small to make an incandescent light bulb glow but is enough to make most kinds of LED bulb glow.
The working part of Incandescent bulbs, (the coiled tungsten filament) operate at 120 V AC or 230V AC and draw around 500 mA current when working.
Each of several working parts of a LED bulb (each individual light emitting diode itself) operates at about 2V DC and draws maybe 8 mA from the 120 V side of it's driver circuit.
So it can only take a very small current to make a LED begin to light up.
Since LEDs are low-voltage constant-current DC devices, there has to be some driver circuitry somewhere to make them work with 120V AC. This circuitry is either built into the "bulb" or is a separate box. There are many different designs for the driver circuitry and some of these are more susceptible to producing the sort of effects you report.
I don't know if induced voltages from long runs of wiring next to each other are enough to start to drive a LED, but there are probably many circumstances where this happens unexpectedly - it isn't a certain indication of a wiring fault. Far from it.
I tested with a voltage sensor, and it does indeed beep for a split second when hold it up to the bedroom light wire while turning on the bathroom light.
This does suggest some sort of inductive transient voltage being generated. Are there any fluorescent lights in the bathroom?
To properly diagnose this issue you will probably need more equipment - or hire an electrician.
Best Answer
As the comments above indicated, the problem was a dimmer switch that I didn't know existed.