First off, these things sometimes happen. If this is a one-off occurrence, I wouldn't be too concerned: people sometimes get it into their heads that they need to do something "weird" for no good reason. I was playing in an In Nomine game as an angel who could possess people/animals; I somehow decided that I needed a monkey, and we spent about as long planning a zoo heist (that never happened).
So, one-off: don't worry too much.
If this is a more common event, or if it looks like it might be (Tom sounds like he might encourage this kind of "weird strategy" thinking), there are a couple of things that have been helpful in my games:
talk to the "problem" player(s): remind them nicely (and privately) that there are other players who need to be included in the action and the planning. yes, this is the default answer to anything with the problem-players or problem-gm tags; it's the default answer for a reason.
remind the players that time is passing in-game. In this particular example, while Tom and Frank were discussing what to do with the interlopers, the captain could be struggling against his bonds, the captors could be rummaging around the cargo hold, a sailor could be whimpering behind a crate. As the planning session goes on, the captors might get more violent, roughing up a sailor who gets in their way. D&D does encourage some "free action" tactics talk among the players, but time does flow.
if they still don't get the hint, a brigand could actually accost the PCs: demanding money, ordering them around, etc.. Roll for initiative! And, remember that the brigands are totally getting a surprise round (are those still a thing in 5E? Sorry, my experience is almost all 3.5/PF). I'd encourage relaxing some of the rules about how long talking-type resolutions take (or, at least, suspend initiative if someone rolls well enough to warrant a brief cease-fire), but there comes a point where conflict is inevitable (this is a good thing: the whole point of RPGs is resolving conflict) and the PCs can't just talk amongst themselves any longer.
alternatively, poll the players to see if they want to continue with strategy or move into action. Tom and Frank can talk about tactics all they want, but if the cleric steps in to stop the pirates from harming their ship's captain, Tom and Frank will have to respond. This can backfire! Tom and Frank can be angered by the cleric acting "before they're ready"; therefore, I encourage mixing this in with #2: "Tom and Frank are whispering about what to do; Cleric: you see the brigands roughing up the sailor; Druid: they're dumping something unpleasant from the hold into the bay. What do you do?"
I'd personally be hesitant to actually use one, but I've seen suggestions to use a small hourglass (a 30-60 second timer) to let the players discuss how to respond to a new situation before in-game time resumes. The in-universe logic being that the PCs have been traveling together for long enough that they can quickly communicate how to respond to violence with a subtle gesture, where the players need to actually talk.
You'll have to be a little careful to not step on Tom's fun in planning intricate strategy, but that's relatively easy: make sure that there are plenty of encounters where the PCs know what's coming far enough ahead that they can talk for a few minutes in-game before the enemies know they're there.
The Dominate Beast spell will end
There was a (somewhat confusing) developer tweet about this topic. Jeremy Crawford stated (emphasis mine):
In #DnD, the exceptional trumps the general. (No longer being a valid target trumps condition carryover.)
When a wildshaped Druid reverts to their normal form, we have those two conditions: the target changing type, and condition (charmed) carrying over. The target changing type makes it an invalid target which, as confirmed in the sage advice tweet, trumps the charmed condition carrying over.
Since the spell would no longer has a valid target once the druid changes back, the spell will end.
What about the other two Dominate X spells?
For completeness, let's run through the two other dominate spells.
Dominate Person requires a humanoid target. All official PC races are humanoid, so while the druid is in its normal form it can be targeted and charmed by a Dominate Person spell. If the druid wildshapes into a beast, its type changes to match that beast. So, the druid is no longer a humanoid, the target of Dominate Person is invalid, and the spell ends. In this way, it works in reverse of the Dominate Beast spell.
Dominate Monster works on any creature. Whether in beast form, humanoid form, or even in a Circle of the Moon druid's elemental form, the druid is still a creature. Thus, even going in and out of wild shape won't make the druid an invalid target. Since the target is still valid, the Dominate Monster spell will still have full effect, even if the druid wild shapes.
Best Answer
The D&D 5e Player's Handbook, chapter 9, refers to fighting as what happens during combat, which according to the "Combat Step by Step" sidebar is that familiar situation in which initiative is rolled and everyone takes turns:
However, the rules don't strictly define "fighting", meaning that it's also plausible that the DM adjudicates the term under the standardly understood English definition of the word, engaging in a fight. Since a fight is precisely what happens in combat when initiative is rolled, this has the same meaning.