could this work?
No. All weapons with the "heavy" property have the two-handed property as well. Here's exactly what each property does (PHB 147):
Heavy. Small creatures have disadvantage on attack rolls with heavy weapons. A heavy weapon's size and bulk make it too large for a Small creature to use effectively.
Two-Handed. This weapon requires two hands when you attack with it.
The heavy property itself isn't a problem here, but the two-handed is. Since all heavy weapons are also two-handed, you must use two hands to wield them. The enlarge spell doesn't give you the ability to wield a two-handed weapon in one hand, even if that weapon wasn't enlarged with you. Your DM might allow it, but it's certainly not part of the Rules As Written.
So what can you dual wield, then? Here's the rules on two-weapon fighting (PHB 195):
When you take the Attack action and attack with a light melee weapon that you're holding in one hand, you can use a bonus action to attack with a different light melee weapon that you're holding in one hand. You don't add your ability modifier to the damage of the bonus attack, unless that modifier is negative.
Based on this, you can only dual wield light melee weapons. You don't even need the two-weapon fighting style, since all the fighting style does is allow you to add your ability modifier to the bonus action attack. There is a way to get around the "light" limitation, namely the Dual Wielder feat (PHB 165, emphasis mine):
You master fighting with two weapons, gaining the following benefits:
- You gain a +1 bonus to AC while you are wielding a separate melee weapon in each hand
- You can use two-weapon fighting even when the one-handed melee weapons aren't light
- You can draw or stow two one-handed weapons when you would normally only be able to draw or stow only one.
This feat allows you (among other things) to use any one-handed melee weapons for two-weapon fighting.
In conclusion, you're not going to be two-weapon fighting using greatswords, even if you're enlarged. The closest you're going to get would be two longswords with the Dual Wielder feat
Side note: If you want to do two-weapon fighting purely from an optimisation perspective, you may want to reconsider. It is generally accepted as one of the weaker playstyles, as it uses your bonus action to be effective (needed to cast smite spells on a paladin, for example), and still can't quite compete with great weapon fighting, especially if you factor in the Great Weapon Master feat. It doesn't fall completely behind other playstyles, so if you want to do it because of a character concept, go ahead. But if it's exclusively for optimisation, it might be better to just use a single greatsword.
A potion of growth or an enlargement spell
By changing to medium size, temporarily, that disadvantage will go away. Both the potion and the spell make that possible.
Comment:
this point got some discussion early in this edition's release. (This question is related but not identical, as is this one). This little piece of verisimilitude -- which parallels a similar rule about weapons for Large and Huge creatures not getting a damage increase if a medium sized creature uses it (DMG p. 278)-- avoids the absurdity of a halfling wielding a greatsword, while still allowing for a variety of other absurdities.
FWIW, this small sized character can ride a medium sized creature. Gnome and halfling Rangers (for example) can take good advantage of that if they are Beast Masters.
In the interest of Rules as Fun (RAF1)
In making a ruling that is not strictly RAW, consider what is behind the character build, and the use it is making use of Small size. As the DM, consider the impact of ruling that the character counts as Medium for all rules purposes, such that the bulky paladin armor makes up the difference. In that case, a maul-wielding halfling might provide a bit of light comedy, but it would have very little impact on game balance. It's only if the player expected to get all of the benefits of Small and yet avoid any negative consequences of that size that the character concept begins to border on "cheesy" as well as funny. (Thanks to @NeilSlater for this point)
1RAF
Regardless of what’s on the page or what the designers intended,
D&D is meant to be fun, and the DM is the ringmaster at each game
table. The best DMs shape the game on the fly to bring the most
delight to his or her players. Such DMs aim for RAF, “rules as fun.”
We expect DMs to depart from the rules when running a particular
campaign or when seeking the greatest happiness for a certain group of
players. Sometimes my rules answers will include advice on achieving
the RAF interpretation of a rule for your group. I recommend a
healthy mix of RAW, RAI, and RAF! (Jeremy Crawford, Sage Advice Compendium, page 1-2)
Best Answer
“Underwater Combat” rules provide a framework for unfriendly environments
There are basic rules on PH p.192, under Squeezing Into a Smaller Space for dealing with creatures in spaces too small to accommodate them. These don’t go into the level of detail you are suggesting, involving restrictions on various weapons.
It sounds like you wish to create a feeling of the characters being “out of their element” in the cramped goblin warrens. The game designers accomplished this with their Underwater Combat rules. These rules can serve as a framework for any unfriendly environment.
Here is what you would do for melee weapons:
For your goblins, the ability to use any weapon may be irrelevant, if they are using “standard issue” short swords and short bows. But you may wish to rule, for example, that any small or smaller creature has room to use any of their weapons effectively.
This is the quick and fun way to handle such environments. The party may want to adjust their tactics a bit, but (used in moderation) no player should feel frustrated about their character being made ineffective.
Useless weapons are like monster immunities
If you go beyond sorts of things in the “Underwater Combat” rules, then you may have some work to do for play balance. If you make certain weapon completely unusable in the caverns, you effectively give the denizens immunity to damage from those weapons.
The Creating a Monster: Step 9 section of the DMG (p. 277) states:
In your case, monsters would only be immune/resistant to some weapons, but you are probably still limiting some of your PC’s damage expressions. Take your specific case into consideration (that is, how the damage expressions of the PC’s in your party would be effected) and determine if you should apply the challenge rating modifiers in the Effective Hit Points Based on Resistances and Immunities table on DMG p. 277.
Don’t spring the rules on your party
Especially if you go beyond the “Underwater Combat” style rules, make sure you give your players fair warning about whatever strictures you would impose. If a passageway is too cramped to use a PC’s favored weapon effectively, the character should realize that.
Be aware, if you make the environment too unfriendly to the party, they may simply decide not to enter.