Electrical – Both prongs of an outlet carries live electricity of a switched plug

I live in an old house which was build in 1920s.

On the hallway, there is an outlet and a ceiling light which are both control by one switch.

Recently I use contact pen style tester to check the outlet.

I found out both prongs of the outlet carries live electricity at the moment the switch is at it's OFF position. When the switch at the ON position, one side of the outlet is hot, the other side is neutral, or it carries no electricity.

I decide use a multimeter to measure it's voltage reading of the switched outlet.

when the switch is at its ON position:

I put red probe of the multimeter into Hot, the black probe into Neutral,
the voltage reading: 120 volts

when the switch is at its OFF position:

I put the red probe into Hot, the black probe into Neutral,
the voltage reads: 2.8 volt

I put red probe into Hot, black probe into D-shape prong(Ground)
the voltage reads: 0 volt

I put the red probe into Neutral, black probe into Ground
voltage reads: 0 volt

I believe the Ground is not grounded, and there is no neutral wire when the switch is at OFF position

I use a long extension cord, plug it into an outlet of another room, to bring the neutral wire next to the outlet for testing.

I put red probe into the Neutral prong of the extension power cord that pluged into an outlet of another room

when the switch at its ON position:
Hot prong of the outlet: reads 120 Volts (hot to Neutral of a different outlet)
Neutral prong of the outlet: reads 0 Volt (Neutral to Neutral of a different outlet)

when switch at OFF position:
Hot prong of the outlet: reads 120 Volts (hot to Neutral of a different outlet)
Neutral prong of the outlet: reads 100 Volt (Neutral to Neutral of a different outlet)

The conclusion:
the switch, the outlet, and the ceiling light, work normally. The switch controls ON and OFF of the light and the plug.

BUT, when the switch is at OFF position,
Both prongs of the outlet carries live electricity, one prong with 120 Volts, an other prong with 100 Volts'

Is it normal? or is it dangerous?

• edited as Harper suggest, thank you Harper.

Thank you, Retired Master Electrician
Thank you, Kris
Thank you, Ken

My problem is solved now.

Kris is right.
In old connection, the hot wire was not switch, instead, it switched the neutral.
I rewire the circuit, connect hot wire to the switch, twisted the neutral wires together and cap it.

After rewired the Outcome:
When switch OFF: Hot to Neutral reads: 0. with pen tester, both prongs unlit
When switch ON: Hot to Neutral reads: 120 volts, with pen tester, Hot prongs lit, Neutral prongs unlit

Thank you. All.