There is nothing to prevent a character from casting Witch Bolt on a target that the character has grappled.
Grappling places the target in the grappled condition which reduces their movement to zero. It only takes one hand free per Page 74 of the 5e Basic Player Rules.
Using at least one free hand, you try to seize the target by making a
grapple check,...
Allowing you to cast Witch Bolt with the other free hand due to the rule on Somatic spell components on page 205 of the PHB
Spellcasting gestures might include a forceful gesticulation or an
intricate set of gestures. If a spell requires a somatic component,
the caster must have free use of at least one hand to perform these
gestures.
However the target of the grapple is free to attack the person who is grappling him. Which will cause concentration checks if damage is done to the caster.
Also as a house rule, a referee may decide that because Witch Bolt is a lightning effect and since the caster is grappling the target, that the caster is subject to the effects of the spell due to how electricity and lightning works in the real world. Again this is not in the RAW rules, it would be a house rule. I am mention because in my experience it is common for referee apply what they know of real life to various rulings even in a fantasy game like D&D.
While it is useful in that it prevent the target from moving beyond the 30 foot range of the spell, the fact the target can still attack means that it is likely that concentration checks will ensue. If failed it negates the point of casting the Witch Bolt in the first place. Combined with the disadvantaged ranged spell attack roll makes this a marginal tactic in my opinion.
If it misses initially, it fizzles out.
The text for Witch Bolt is clear that it only takes effect on a hit:
Make a ranged spell attack against that creature. On a hit, the target takes 1d12 lightning damage, and on each of your turns for the duration, you can use your action to deal 1d12 lightning damage to the target automatically.
On a miss, the spell slot is wasted and the warlock would need to try again next turn.
Yes, the spell is situational. It has its uses, though there are many drawbacks and shortfalls, but I wouldn't tell a guy not to pick it if he really wanted to.
Best Answer
At Low ACs, Witch Bolt is better. At high ACs, Firebolt is better
The trick with Witch Bolt is that the initial casting of the spell isn't spammable at level 1. You only get to use it twice, and if either casting fails, the damage fails completely. Conversely, with Firebolt, you can keep using the spell even if prior uses failed. So if Witch Bolt successfully lands, you have a source of guaranteed damage, but if it fails, you can't just endlessly recast it. You can try again the next turn, and... well.
The average damage from these spells, over five rounds, looks like this:
The To-hit 11 is your case: you'll hit on an 11 or better, meaning you hit 50% of the time. AC values assume +3 to primary spellcasting modifier.
The other consideration is that the damage-over-time effect of Witch Bolt cannot crit, but the repeated uses of Firebolt do each have a chance to crit. Outside of very high AC creatures though (To-Hit of 18 or 20, which correspond to ACs of 23-25; very unlikely for a Level 1 character) Witch Bolt will end up being better.