Regular Outlet to GFCI Outlet with four wire connections and no ground

The red lead is wired to hot on BOTH the load and lead terminals of the GFCI. Likewise, the white lead is wired to neutral of both the load and lead terminals.

This sounds insanely wrong, and if it is in fact an Edison circuit -- two hots, shared neutral -- then a GFCI cannot possibly work right because the current on the neutral will not be the same as the current on the hot being measured by the GFCI. It will trip frequently.

I don't like shared neutral circuits at the best of times because remember the current on the shared neutral can be as high as the sum of the currents on the hots. Just because the neutral has no voltage does not mean it has no current, but the overcurrent protection on the breakers is on the hots. You don't want to get into an overcurrent situation on the neutral because nothing stops those wires from overheating.

Is this wired correctly, and if not, how do I rewire it so that it is?

There was a similar circuit in my house when I bought it and I ended up just pulling new correct wiring through the walls and doing the whole thing over. I concur that when you are rewiring an old house, everyone who lived there before you was dangerously ignorant; I've found terrifyingly bad wiring in GFCIs.

So, solution one: rewire everything so that there is one circuit for the GFCI and everything downstream that you want protected by it, and another circuit for the non-protected stuff.

However, if you don't want to actually pull new wire through the walls and do it all again, the next best thing to do is solution two: get a two-pole GFCI breaker and replace the breaker in the panel. Now you can throw away the GFCI outlet and replace it with a regular outlet. The downside is (1) expensive, and (2) when it trips you have to go to the panel to reset it.

I just replaced the outlet by splicing the outlet inline i.e. both the incoming and outgoing are connected to the hot lead terminal, same with neutral.

That's solution three, and it will work, but whoever wired it up originally might have wanted things downstream of the GFCI to be protected by it. This solution breaks that property.

GFCI devices work by comparing the current incoming on the hot wire to the current leaving on the neutral wire. If there's a difference, it means electricity is leaving the circuit somehow (a "ground fault") and the GFCI cuts power*. The GFCI doesn't do anything with the ground conductor—in fact, you don't even need to have one at all. One way of upgrading a 2-prong outlet (without ground) to a 3-prong outlet is by inserting a GFCI device upstream or using a GFCI outlet.

So I'm guessing there is a small amount of current in your scenario, and when you touch the white and ground together, some of that current goes into the ground and doesn't return through the GFCI neutral. Do any of you lights have timers, nightlight, etc.?

(BTW I'm not sure it matters for this question but your description of how the wires up is a little confusing to me. LINE and LOAD are opposite sides of the GFCI device, but you use them interchangeably. If you want a downstream device to be protected by the GFCI it must be connected to the LOAD side. Anything on the LINE side is considered upstream and is unaffected by the GFCI.)