Well, as you stated, a smart approach when this happens is to just ask the player, but with that out of the question here are some options I might use in this situation:
Give objectives.
Try to add in something extra to the fight, maybe a lever to pulled, or even better, a door to be unlocked (either locky-picky, or with hacking). A cool way to use this is to have a battle that the group (including him) isn't ready to face, but giving them an "out". Just make sure that they know that they can't win this fight — nothing is worst than hubris getting in the way of an awesome escape scene. With this the rest of the group still has stuff to do (the hacker can try and open the door while someone strong might try to barricade them a bit, etc.) while he is getting challenged to "real" fight.
Boss with minions. This is less fun in the long run, but here and there it could do you wonders — have a fight that centers around a main baddy. This leads to one of two options:
a) That he has to focus all of his efforts on the baddy, giving them time to fight the goons; or the somewhat better b) of letting them take on the baddy (merely being strong isn't enough, it needs some skill or combination of skills to kill him) letting them show off what they are good at, and giving him the feeling of power that comes with mowing down 7 baddies in 2 rounds.
Split the party either before the fight, or through the sheer size of it (he can't be everywhere at once, can he?), so that he might take out everyone in one room, but that doesn't quite cover the next room where the other part of the group is fighting.
And yes, having less combat, even in Savage Worlds, is a good thing — it helps you evolve as a GM, and it's often more fun than "another random fight". Fights are cool, but they aren't the only cool thing, not in role-playing games: the coolest thing, is playing a role.
As you say, this house rule works well for your group. A new player objecting to a house rule they don't understand is no reason for you to change it.
And they don't understand the point of the house rule. They've observed their fellow players and seen the rule's results, and are trying to adjust their actions to fit what the rule is meant to encourage — and they've misunderstood. Instead of engaging in new, creative roleplaying, they're just copying what they see other players do in a kind of cargo cult roleplaying.
This is also the source of their frustration: they think they've solved the puzzle, and they perceive you denying them the reward for solving it. They haven't understood it though, and rewarding them for missing the point will not help them integrate into your group.
Having put in effort and missed the "target" of understanding the point of the bonus experience, though, you should meet them halfway:
"It's clever how you noticed what the other players did to earn the bonus experience. That's great, but it's only halfway to getting the point of the bonus. The bonus is a reward for individual creativity in roleplaying, so simple copying doesn't earn the bonus.
"Does that make sense? This is an important tool that has worked for us, and I want you to understand what it's for so that it works for you too."
You need to emphasise that you're willing to help the new player get up to speed with the group. Emphasise that, because that's what the conversation should be about — not about whether the rule is useful or not (you already know it is for your group), and not whether the player gets to challenge the award (that's not their job in the group dynamic). Just take it as given that this is how it works, that's not up for discussion and don't bring it up for discussion. Move past those givens without even mentioning them, and instead move right to discussing with the player the point of it and how they can reach that point.
P.S. — Contrary to recent online RPG community fashions that resist XP rewards, this is a common and widespread way for RPGs and groups to work, and entirely functional when done fairly and impartially. Current fashion is simply that — passing fashion. Fashion is not legit grounds to tell you it's a bad way to run your game.
Best Answer
It's up to you to decide.
First, the DM has final authority over how XP is divided. The DMG has advice on giving XP, but it gives a lot of variants and ultimately leaves it up to the DM. Therefore, since you're the DM, the real answer to your question is "you decide".
An XP race might be the kind of game your players actually want to play, but that should be discussed with your players and agreed upon ahead of time.
Sharing XP is better for the game.
That being said, DMG 260 states,
The key word here is "participate". In your encounter, everyone in the party is participating--they rolled initiative, they took turns, and so they get experience for it.
First, consider the ingame perspective. Sure, the other party members did not actually reduce the HP of the monster, but they could have affected its outcome in other ways. Were they fighting other monsters instead? If so, they could have been keeping those other monsters from ganging up on the lone PC. Even if they were just standing and watching, their presence likely affected the circumstances of the fight, especially if they all took turns. Given that XP is not mere fighting experience (wizards don't learn spells directly from killing), it isn't hard to imagine how they could have learned something from being in the encounter.
Second, consider the out-of-game incentives you're creating for your players. Are you going to divvy up XP according to the share of damage each PC did to each creature? Are you going to encourage the PCs to compete to isolate monsters and go one-on-one? How will you handle some PCs being much better at killing than others?
Because levels are so important in 5e, any XP system other than "everyone gets XP divided evenly" turns the game into a competition for XP. Some of my favorite encounters are ones in which combat isn't the goal--some PCs are trying to save some NPCs, while the others hold off an attacking force, for example. Is it fair if only the PCs that did actual fighting get any XP for that encounter?