No
But the argument is not so straightforward.
The problem with this question is that the RAW are murky enough to allow for some leeway in interpretation.
Going by RAI, the intention was clearly to use Two-Weapon Fighting with one-handed weapons, and not game the system by attacking with a versatile weapon while holding it in two hands, arguing that it still counts as a one-handed weapon for the Dual Wielder feat.
The Rules
Dual Wielder
You can use two-weapon fighting even when the one-handed melee weapons you are wielding aren’t light.
While this qualifies the longsword for two-weapon fighting, the Versatile property states that:
Versatile
This weapon can be used with one or two hands.
A damage value in parentheses appears with the property—the damage when the weapon is used with two hands to make a melee attack.
This explicitly states the need to use both hands to make a two-handed attack and use the bigger weapon dice, thus disqualifying this use case from the Dual Wielder requirement (one-handed melee weapons).
The Conclusion
While you can use a thrown handaxe and then attack with the longsword, you can only do so using the one-handed dice the Versatile weapon tag provides.
I'm not sure if I got your question right but
You already apply the ability modifier to the 1d4 attack with Polearm Master.
This is from the sage advice compendium:
Polearm Master Can I add my Strength modifier to the damage of the
bonus attack that Polearm Master gives me?
Yep! If you have the feat
and use the Attack action to attack with a glaive, halberd, or
quarterstaff, you can also strike with the weapon’s opposite end as a
bonus action. For that bonus attack, you add your ability modifier to
the attack roll, as you do whenever you attack with that weapon, and
if you hit, you add the same ability modifier to the damage roll,
which is normal for weapon damage rolls (PH, 196). A specific rule,
such as the rule for two-weapon fighting (PH, 195), might break the
general rule by telling you not to add your ability modifier to the
damage. Polearm Master doesn’t do that.
You don't need the Two-Weapon fighting style to do that, it's already incorporated in the feat.
Best Answer
Ability Score Improvements are almost always better for DPR
Calculation
On 4th level you could raise your attack stat from 16 to 18, or take Dual Wielder. I will assume you have the Two-Weapon Fighting style, while fighting against AC 10, 14 and 18. Criticals are also included in the final DPR.
Base DPR:
Damage 2 x (1d6+3) = 13
AC10: Hit chance 80%, 10.75 DPR
AC14: Hit chance 60%, 8.15 DPR
AC18: Hit chance 40%, 5.55 DPR
ASI:
Damage 2 x (1d6+4) = 15
AC10: Hit chance 85%, 13.10 DPR
AC14: Hit chance 65%, 10.10 DPR
AC18: Hit chance 45%, 7.10 DPR
Dual Wielder:
Damage 2 x (1d8+3) = 15
AC10: Hit chance 80%, 12.45 DPR
AC14: Hit chance 60%, 9.45 DPR
AC18: Hit chance 40%, 6.45 DPR
You can look at it this way, the damage is the same, as on average 1d6+4 = 1d8+3, but after the ASI you have higher hit chance.
Other considerations
Dual Wielder gives you +1 AC too, but if you are Dex primary, an ASI gives you the same, and +1 save, +1 skills, +1 initiative.
If you are Str primary the ASI is much less helpful in this regard. (But in this case you could just take Great Weapon Fighting, for significantly higher DPR)
Dual Wielder lets you draw two weapons without sacrificing an action. This can be important in cities, but in the wilderness and dungeons you can just run around with one of your weapons already drawn.
Exception
Take Dual Wielder if the only magic weapon you have is unusable without Dual Wielder (because it's not light).
+1 to attack and damage from a magic weapon plus the increased die size is better in itself, but if you consider all the monsters resistant or immune to mundane weapons, it is the highest priority.
Conclusion
You should only pick Dual Wielder if: