The hand does persist

A successful save only changes whether/how the target is affected by the spell

The target must make a Strength save. On a failed save, the target takes 2d6 bludgeoning damage and is restrained for the spell's duration.

In fact, every spell persists for the duration even after a successful save unless there is specific mention of the spell ending.

A spell's duration is the length of time the spell persists... Some spells specify that their effects last until the spells are dispelled or destroyed.

[emphasis mine, note it only says some spells]

This is because the saving throws only allow targets to avoid effects of the spell rather than cancel the spell as a whole.

Many spells specify that a target can make a saving throw to avoid some or all of a spell's effects

How this works with other spells

For example, charm person will last the whole hour unless it is dispelled, even if the target succeeds on the save (although the spell then has no effect) since there is no mention of the spell ending otherwise.

You attempt to charm a humanoid you can see within range. It must make a Wisdom saving throw, and does so with advantage if you or your companions are fighting it. If it fails the saving throw, it is charmed by you until the spell ends or until you or your companions do anything harmful to it. The charmed creature regards you as a friendly acquaintance. When the spell ends, the creature knows it was charmed by you.

No. Movement forced by a spell is not willing movement.

"Willing movement" (and being "willing" in general) is never defined in the rules, though it's referenced in spells like booming blade (SCAG, p. 142). Thus, since it's not a defined game term, we default to the natural English definition of the term.

According to Dictionary.com, "willing" means "disposed or consenting; inclined". However, the movement from the infestation spell (XGtE, p. 158) is forced by the spell on a failed Con save; the target has no choice about whether to do it, so it's certainly not doing so of its own volition. As such, it seems clear that movement forced by such a spell is not willing.

Jeremy Crawford supports this interpretation in an unofficial April 2016 tweet in which he addresses a similar question about the dissonant whispers spell and its interaction with booming blade:

Would you take damage from the second part of Booming Blade if you fail a save against Dissonant Whispers and move away?

Booming blade hurts you if you move away willingly. Dissonant whispers forces you to move—doesn't trigger BL.

but if they move after whispers in their turn (to get back into melee, for example) that counts, right?

Yes.

Crawford provides the same explanation in an earlier unofficial tweet in February 2016:

Would the movement caused by a failed save on Dissonant Whispers be willing or unwilling movement?

If a spell forces you to move, as dissonant whispers does, you're not moving of your own volition.

Does that mean the movement from Dissonant Whispers doesn't provoke opportunity attacks?

The movement in dissonant whispers can provoke opportunity attacks, since it uses your reaction (PH, 195).

The same logic would follow here regarding infestation and booming blade; the spell is forcing the targeted creature to move, so the creature is not doing so willingly. Thus, booming blade's extra damage would not trigger.

(Sidenote: infestation does not trigger opportunity attacks either, because it moves the creature without using its action, reaction, or movement.)