I'd like to encourage my players to engage in more dynamic melee combat, especially at low levels. We have tried to use the Flanking optional rule on DMG page 251, but weren't happy with the results.
Next session I'm going to introduce two house rules:
Drop Prone: When a creature you can see attacks you with a ranged attack, you can use your reaction to fall to the ground, gaining the prone condition. Attack rolls (including the first one) against a prone target have advantage if the attacker is within 5 feet, or disadvantage otherwise.
Give Ground: When a creature attacks you with a melee attack, you can use your reaction to impose disadvantage on the attack roll and move 5 feet away from the creature without provoking an opportunity attack from it. Your movement will be halved next turn. You cannot Give Ground if the attacker has advantage.
Both reactions are supposed to be available for any creature, not only PCs. Being relatively new to 5e I have concerns regarding possible balance issues. What parts of my house rules can apparently benefit (or hurt) only specific classes?
Let's assume that the combat system is balanced as-is: that is, that the existing tools the system provides for defensive options in combat have the appropriate trade-offs between cost and benefit.
The basic tool to give an attacker disadvantage is the Dodge action. In addition to the obvious impact of having to spend an action taking Dodge (mostly: not being able to attack or cast a spell), there's a slightly subtler consequence. A creature has to choose to Dodge before the attacks start. This means the Dodging creature is gambling that the cost of forgoing some other action now will be worth the overall defensive benefit in the future.
Drop Prone and Give Ground instead give a defensive benefit now in exchange for a reduction in movement in the future. So the cost is paid after the benefit is received. Even setting aside the actual costs, this is a much better deal for the character! Suppose the house rule was simplified to just:
Desperate Dodge: Any character may spend their reaction to take the Dodge action until the start of their next turn. If they do, they cannot take an action on that turn.
This is much better than the regular Dodge, because a character is guaranteed never to use it unless they really need it. And they can plan around not being able to act on their turn. If this option existed, nobody would use the regular Dodge action.
Cautious Dodge: Any character may spend their reaction to take the Dodge action until the start of their next turn. If they do, they forego their move on that turn.
Cautious Dodge is much better than Desperate Dodge, which was already better than Dodge: it trades the major disadvantage of losing an action for the much more manageable disadvantage of losing movement.
Drop Prone and Give Ground are both generally significantly better than Cautious Dodge. They are slightly more limited (not affecting all attacks, having other minor drawbacks, etc), but their costs are generally lower as well. And much of the time characters only need to protect themselves against a single attack, and can mitigate or ignore the other downsides.
Given that these options are good enough to make one of the basic combat actions much worse by comparison, I would say they are unbalanced relative to the other actions characters have in combat. They remove one of the basic balancing mechanisms by allowing characters to trade immediate benefits for relatively small future costs.