As Miniman's answer points out, you cannot grapple as an opportunity attack because an opportunity attack does not give you an Attack action. However, your situation doesn't actually call for an opportunity attack. Instead, it sounds like you had readied an action. From Basic Rules page 72:
Sometimes you want to get the jump on a foe or wait for a particular circumstance before you act. To do so, you can take the Ready action on your turn so that you can act later in the round using your reaction.
First, you decide what perceivable circumstance will trigger your reaction. Then, you choose the action you will take in response to that trigger, or you choose to move up to your speed in response to it...
When the trigger occurs, you can either take your reaction right after the trigger finishes or ignore the trigger. Remember that you can take only one reaction per round.
When you ready an action, you declare that you intend to do something out of your turn in response to something specific happening. For example, you can ready an action by saying "If an enemy comes through that door, I will fire an arrow at them." If an enemy does come through the door, you can then immediately use the readied action and shoot them.
Therefore, if you state on your turn, "I want to grapple the gnome if he tries to get past me," and you have not taken another action on your turn, you would be able to grapple the gnome as he moves past you. If the gnome doesn't try to get by you, the action is wasted. You can also choose to ignore the trigger and take an attack of opportunity as your reaction instead.
Keep in mind that you must declare a Ready action on your turn. Ready actions are actions, like attacks or casting spells. If your turn had already passed when you declared you'd grapple the gnome, your DM would be right in ruling that you can only make an attack. However, if you declared it on your turn, and hadn't already attacked or acted, your DM should have allowed you to grab the gnome.
No. An opportunity attack grants you a single melee attack. Shoving requires taking the Attack action.
See PHB pg 195.
Opportunity attack:
You can make an opportunity attack when a hostile creature that you
can see moves out of your reach. To make the opportunity attack, you
use your reaction to make one melee attack against the provoking
creature.
Shoving:
Using the Attack action, you can make a special melee attack to shove a creature prone. If you're able to make multiple melee attacks with the Attack action, this attack replaces one of them.
For much the same reason, an opportunity attack also only grants you a single melee attack and not the multiple attacks granted by the Attack action.
Jeremy Crawford states that one way to shove outside of your turn is to take the ready action to prepare a grapple/shove when your target escapes your reach.
Grappling/shoving are part of the Attack action (PH, 195). Take the Ready action to grapple/shove outside your turn.
[Sage Advice]
Note that this means you sacrifice your Action on your turn.
Also see related for grappling: Can you make a grapple attack as an Attack of Opportunity
Best Answer
The key here is whether the "special attack" requires the attack action or just a melee or weapon attack. An opportunity attack is a melee attack. It is not an attack action. Therefore, special attacks which require an attack action cannot replace it, whereas special attacks which require only a melee attack can.
No for Grapple and Shove
From Sage Advice:
Yes for Disarming Attack (the fighter maneuver)
Disarming Attack:
Opportunity Attack:
Melee Attacks
I think that it's fairly clear that an opportunity attack is a weapon attack, and can therefore become a disarming attack if the fighter expends a superiority die.
Yes for the Optional Disarm Rule (note that there is no disarm rule in the PHB)
Disarm Contest:
As above, it only requires a weapon attack, not an attack action. However, you will need to ask your DM (if you are not the DM) as to whether they are using this optional rule.