Yes, you can
Jeremy Crawford confirms this on Twitter.
Clint McReynolds
@CJMcReynolds
@JeremyECrawford After Haste ends, PHB says target can't move or take actions. Does this include reactions? Advantage to hit target?
Jeremy Crawford
@JeremyECrawford
Actions and reactions are different. If an effect, like the haste spell, shuts off one, it doesn't necessarily shut off the other. #DnD
Other supporting evidence
In the PHB, the Incapacitated condition differentiates between actions and reactions:
Incapacitated, PHB 290
An incapacitated creature can't take actions or reactions.
Ramifications
Actions and reactions are different.
If you had haste and fly on (concentration can be dealt with via potion, buff from ally, magic item, etc), and someone cast a dispel magic on you, you can still catch yourself with a feather fall.
If you were frozen with lethargy, you can still muster the "energy" to cast shield, hellish rebuke, absorb elements, or even counterspell, and perhaps you can narrate it as one desperate wave of the hand and a breathless incantation.
Ending haste does not incapacitate you. It just makes you lethargic and unable to take actions.
Your option 3 is correct.
When the spell ends, the target can’t move or take
actions until after its next turn, as a wave of lethargy
sweeps over it.
Not "when the target is no longer under this effect" or "when the target no longer has Haste"; the condition for losing a turn is when the spell ends. So it doesn't matter that there are 2 spells affecting them simultaneously - if a Haste spell ends, you lose a turn.
However, they are still under the effects of a Haste spell.
Until the spell ends, the target’s speed is doubled, it gains
a +2 bonus to AC, it has advantage on Dexterity saving
throws, and it gains an additional action on each of its
turns.
Their speed is still doubled, they still have +2 AC, they still have advantage on Dexterity saving throws, and they still gain an additional action. They can't move, and they can't take actions, but this doesn't prevent them from getting any of those benefits. It just prevents them from using some of them.
Best Answer
This effect is not magical at this point, just, as the dictionary definition puts it,
So, since this is merely an exhaustion equivalent, dispel magic and antimagic field can not get rid of this effect, and if used during the spell's duration would simply end the spell and cause the exhaustion to happen sooner.
However, there is still a way to get rid the exhaustion, although you would have to talk to your DM about it. The text on the Greater Restoration spell says:
Since this is a form of exhaustion the argument could be made that this lethargy counts equivalent to a level of exhaustion, but due to the wording of the Haste spell, this solution can't be considered RAW, so if your DM doesn't allow it, then there is no other way to get rid of these effects using spells, although you can talk to your DM about finding a magic item to help you as well.