GFCI outlet with led light not on but have power

This sounds highly unusual some plug mounted gfci s need to be reset whenever they receive power but as you said you can unplug it and plug it in without having to reset it so I would check the wiring at the switch to make sure that it is breaking the hot wire and not the neutral also make sure the bare ground wire is not contacting the neutral at the outlet or in the switch box.

My suspicion: an open ground

This is based on two factors:

1. Your three-lamp tester is unable to trip the GFCI, but the GFCI tests OK. This is because the internal TEST button doesn't rely on the ground (it goes load-hot to line-neutral or vice versa), but the three-lamp tester's TEST button is connected hot-to-ground, and thus requires a solid ground connection to work.

2. Pushing the TEST button on your tester yields a "hot/neutral reverse" indication despite it testing correctly initially -- the "correct" indication is due to capacitive coupling between neutral and ground providing a current path "home" for the current through the right-hand lamp.

I'd check to make sure the green wire is securely connected to the box in your case; if it is, then the problem's upstream somewhere as it appears this receptacle was wired using conduit (note the locknut and fitting at the bottom of the box). If you can't find it still -- call in an electrician, as they should have better tools (such as an earth fault analyzer/loop impedance tester) that can be applied to the problem.