Can the spell "Command" be used to cause unwitting damage?
RAW: Yes, you can, IF that damage is unbeknownst to the target (and in most cases the caster as well). You could not tell the target to walk off a cliff, jump into the spike pit, or smash its face into your Paladin's Warhammer. You could however tell the target to walk down a hallway you suspect but are not certain may contain traps. Why? Because it wouldn't be "directly harmful to it", it would be potentially indirectly harmful to it. What if it doesn't trigger any of the traps you aren't even sure are (and may not even be) down there? So no, it could not be used as a foolproof trap detector because unless the GM tells you and you (your character) becomes aware there are traps in said hallway, the spell would not fail. Let's say you poisoned one of their 5 rations and then told them to eat one? You can't be sure which one the target will eat, so it's still indirect.
As for the in-universe thought process? You are exactly correct. Self preservation would cause the magical command to fail, to cease functioning, to not take effect. If the target is unaware that it needs preserve its own life from something that might occur as a result of following the given command, then there would be no need for self preservation, and thus it would follow your command.
While it's hard to judge balance, especially when it's not clear what you're balancing against, these two house-rules both have a pretty effective point of comparison... in an older edition.
In v3.5 of Dungeons & Dragons, Wizards had the option to something like what you suggest. They could opt to prepare only part of their spells and keep the rest open, so they could be filled in later. Wizards were considered one of the most ridiculously powerful and flexible classes in the game.
What you are proposing is even more powerful, because the Wizard at least required at least 15 minutes of rest to prepare a new spell in an open slot.
In my games, I've seen this feature used by a few players. (Most opted to just prep everything at the start of the day.) Here's what happened:
Gameplay slowed down. Whenever my players would run into a problem that they could conceivably solve half an hour later, the Wizard would go down his spellbook and look for possible solutions that they could prep. Wizards tend to have a lot of spells in there, so this would take a while. Lots of banter would ensue over which spell the Wizard might prep. Half the time, in the end no spell was prepped and a different solution was chosen altogether.
Players would look to the Wizard even more than normal. A Wizard who preps everything at the start of the day is incredibly flexible. A Wizard who can prep anything once you know what you're up against is just unreasonably flexible.
The Wizard would ultimately stop using the feature because it's a huge mental load. As the spellbook grows, the number of things a Wizard could do in a given situations rises very rapidly. It was doable for a few levels, but it quickly became too much.
In the end, most players chose not to use the ability because it was both too good and too cumbersome. It would take too much table-time away from having fun and made a lot of encounters too easy.
You are likely to experience the same things as level goes up; the caster's options will go up rapidly, which increases their cognitive load and makes them much more powerful. If you think picking what to do in combat is bad when you have 5 spells to pick from, imagine how bad it gets when you have your entire spell list to chose from.
One of the reasons 5e cut down on how many spells you can cast at a given moment is because combat flows a bit more smoothly that way. And, like I said in my comment, there is a reward in actually thinking ahead now that you will lose. It's not your fault (or problem) that your players aren't gathering information about what they will be facing.
(As for hounding players to select spells; I always rule that after a long rest you prep the same spells as the day before unless you tell me you want to make changes. Most players have a default loadout that they use most of the time anyway).
Best Answer
I don't see any problem with your house rule, but I'm not sure it's really applicable. Part of the problem is terminology--there's a difference between a round and an action or "turn."
In any round, people take their action according to initiative. If my mage character is fighting an orc, and I win initiative, I get to act first in the round: my action is my turn. Suppose on Round 1 of the combat, I try to hit him with my dagger and miss. He then takes his turn and tries to hit me with his weapon--that's his turn.
For Round 2, I decide combat just isn't my style and cast Command, telling the orc "Freeze!" He hasn't had his turn yet--so if he fails his Wisdom save, when his initiative comes up, that is "on his next turn." He loses his action that round. So the rule has the effect that you want, at least as far as that goes.
I'm not finding anything in RAW one way or another, but it seems to me that any action that the orc could take, including fleeing combat, would be covered by that Command as long as he failed the Wisdom save.
In addition, to my eyes the Opportunity aspect is a gap in RAW.
The Command spell has other examples can effectively put the target "out of combat": for instance, "Drop" will make him drop his weapon, "Grovel" will make him fall on the floor. But as written, the target would be able to take a reaction that round, and then freeze, drop, or grovel the next round.
To my mind, that makes no sense, so I agree with you that the rule change is a good one. If I were GM, I would write that rule change so that if the target fails their Wisdom save, they cannot take a reaction until their next turn after they have performed the Commanded action.
This does not unbalance the game, so long as you remember that enemy casters can also use this spell, and use it to the party's disadvantage.