Establish A Rule Disagreement Resolution Tool for your Table

In my reply to this question, I recommended establishing, before the game session, or before the game while session zero is underway, what the "rules disagreement resolution tool" will be at your table.

The objective of this tool -- or something similar that you need to tailor to your specific table -- is that any "huh, it works like what?" question needs timely resolution and a ruling, so that play may continue.

From your description, you have not established that tool. Before your next session, make clear before play begins that GMs need to have fun too, and that their reaction to your ruling damaged the game and damaged your fun.
* Note: it is desirable to get player buy-in to the resolution tool. Once presented, I strongly recommend that you not run another session until the resolution tool is both decided and accepted by everyone at the table. You should actively solicit the players' suggestions and inputs when finalizing the tool.

With the above in mind, here is an example tool to resolve rules disagreements and keep play moving. This isn't the only way to do it, but I've seen it work very well:

1. Player disagrees with how something works. Says it works another way.

2. DM to player: Make your case. (Define time limit. 1 minute, 2 minutes, whatever you are comfortable with)

3. Player: case made briefly

4. DM listen ...

5. DM makes ruling.

6. Play now continues.

Getting the group to buy in to this (or your similar tool) is the key to avoiding the situation you ran into. It is also a matter of table courtesy (bring this point up as well before play begins the next time): we are here to have fun, not get into emotional arguments nor personal attacks.

Your interpersonal relationship with the persons making the challenge may end up with taking on the character of a test of wills. Unless the whole table agrees with the tool, you'll have this happen again. Get the players' involved in crafting the final form of the resolution tool for your table so that you reduce that element of the problem.

If you can't get that buy in, your instinct to give up on that group of players may be the sad future for this group. But you may be able to heal the wound from that session and proceed with more game and more fun.

A last suggestion: if a given player just won't give up on an argument, get up from the table and declare that it's break time. Immersion is gone for good at that point. Once you've all had a chance to calm down, to reduce the intensity of feeling that arose in that disagreement, it becomes easier to resolve.

Other suggested ways to resolve rules disagreements:

• For a disagreement that comes up during play:
  Flip a coin (or roll a die) to pick one of the possible rulings, then move on. After the game, look it up and settle on a most correct answer. (@Thunderforge has seen this work in his games).

• @anaximander recommends this rule before the game starts.

  "If we disagree, you have three minutes to either find the rule in the PHB, or convince me it should work how you want. After that I'll make a ruling and tell you why I think it should be that way. There will be no further discussion, and the rule will work that way for the remainder of the session. After the session I will have a more thorough read of the rules, and if you're still unhappy we can discuss further. At the start of next session, I will announce a better-thought-out rule, which we will use from then on, unless we find flaws in it."

Suggest a resolution for the combat but let the players decide if they accept it.

If not, make a new suggestion or fight it out.

For example, based on your best judgment and gut feeling, you guess that the enemy would get three swings at the primary fighter characters and one at someone else. So you suggest to players: rather than play this out in detail, how about you kill them, each spellcaster uses one 1st level spell, and I get to make three attack rolls at the frontline fighter and one at a random party member. Is this okay?

Then let the players narrate how it goes, if they are so inclined.

Obviously, do not use this when there are significant hidden factors at play, non-standard enemies, a complicated combat with three or more sides, etc., as those are usually interesting to play out and hard to estimate with any reliability.

Note that this is an inherently fair solution, in that the players can always decline. This is also a modular solution: "How about you kill all the melee opponents while the goblins are shooting at you; I get two shots at everyone and the grunts get four swings total divided as you want, and all the grunts are down afterwards?"