The Grappled condition and the Grappling subsystem don't restrict the use of arms

It doesn't matter how many people you have grappling a target, the target will always be able to use their somatic components. If you want a system to model grappling as physically restraining an opponent's limbs, you'll need to develop one. 5e doesn't provide anything there. Since you can't grapple a limb, there's no situation where you'd need to figure out if it interferes with hand-holding, and no rules are provided.

Spells target whoever the caster wants them to target

The spell does require a caster to target a circle of willing creatures, but the caster, as usual, chooses what valid target to select for spellcasting. 5e does not have any system in place to make the players of spellcasters unable to choose the targets they'd like to select from the information at hand. A caster could choose a circle that included a grappling opponent, if they wanted to and the grappler was willing and the GM ruled they had linked hands properly, but they could also choose a circle without said grappler as long as it's a valid circle. The caster, not the spell, chooses the target.

Basically, grappling is not at all an effective way of shutting down spellcasting of any kind, least of all teleportation effects (including Plane Shift). Even if you take the Grappler feat so you can retrain an opponent and even if you also have Mage Slayer and have taken the three rounds worth of actions it takes to grapple a mage, shove them prone, and restrain them, they can still totally Plane Shift away no problem. You can't even hit them with your attack of opportunity, unless they decided to take you along for the ride (you get the attack after the spell happens).

Worst of all, with Plane Shift in particular, the spellcaster can, if they like, cast the spell on you instead, which will kill you if they succeed at a melee spell attack (that they make without advantage or disadvantage if they are restrained, since you are also restrained), since you will then find yourself in a random location on the Deadly Plane of Death and Very Slow Time with no way to return unless you also know Plane Shift.

You must release the target to attack, probably ending the grapple

D&D 5e has a broad intent that any word which is not explicitly given special rules meaning, instead has its plain English meaning, though I don't have a citation prepared for it. In plain English we see this:

Using at least one free hand, you try to seize the target ... you can release the target whenever you like

While "release" technically has multiple meanings, I find this context unambiguous in having Release mean "stop holding with your hand". From this plain English reading, I think it's clear that between the time that you "seize the target" and the time that you "release the target" your hand is occupied by that target; in other words, in order to use both your hands for a 2-handed weapon attack or other purpose you must release the target.

There does remain some ambiguity as to whether that actually ends the condition though. The sentence in question states:

The condition specifies the things that end it, and you can release the target whenever you like

But there isn't a strict connection indicating that "release the target" is an additional thing, not part of the set specified by the condition, which ends the grapple. It does seem likely that it was intended to be so based on the juxtaposition of the concepts, and supported by the subsequent section which unambiguously states that the means of ending a grapple are not entirely restricted to those specified by the condition.

Regarding "Common sense"

In a comment you mentioned that you don't bring real-world common sense into D&D, which is fine. Sometimes the rules explicitly contradict reality; after all you can't complain about someone casting a fireball on account of "magic isn't real". However, D&D also isn't a computer Rules Enforcement System where ambiguous edge cases have a single answer that's always correct and non-negotiable. Instead, D&D is adjudicated by Some Guy who, in my experience, is much more likely to be swayed by "It doesn't make sense that my Half-Orc Barbarian is physically incapable of throwing a gnome" than by "the rules don't say that letting go stops the grapple"; your mileage may vary depending on the DM. In general, I would say that ambiguous situations not clearly covered by rules are more likely to take whichever interpretation most closely matches common sense even if other times common sense is ignored.