The Dueling fighting style doesn't benefit natural weapons
The description of the Dueling fighting style says:
When you are wielding a melee weapon in one hand and no other weapons, you gain a +2 bonus to damage rolls with that weapon.
The key here is that you must be wielding a melee weapon in one hand, and not wielding any other weapons.
As indicated by the question, there are many ways for PCs to gain natural weapons (though unless the player picks a race with a natural weapon, multiclassing is necessary in order for the player to also have the Dueling fighting style).
For instance, the description of the tabaxi race's Cat's Claws trait (VGtM, p. 115) says:
Your claws are natural weapons, which you can use to make unarmed strikes. If you hit with them, you deal slashing damage equal to 1d4 + your Strength modifier, instead of the bludgeoning damage normal for an unarmed strike.
Now, let's consider whether the tabaxi's claws count as "wielding a melee weapon in one hand and no other weapon". (Any natural weapon that is unrelated to your hands, such as a Bite or Stinger attack, is excluded by default for obvious reasons.)
It would be paradoxical for it to work
First of all, let's approach it just from a logical perspective.
User blackaurox asked a similar question on /r/DNDNext on Reddit about benefiting from Dueling while cursed with werebear lycanthropy, and this line from hphammacher's comment on the post quite simply sums up the self-contradiction of allowing claws to let you benefit from Dueling:
If claws are weapons it sounds like there's one in each hand. If claws are not weapons then dueling does not apply.
Essentially, in basically every case in which you have claws (or similar) as natural weapons, you will have them on both hands. Even if we accept that claws can be considered "wielded" in your hand, if you have them on both hands, then you can't benefit from Dueling. And if you don't consider them to be wielded in your hand, then Dueling doesn't apply anyway. You can't logically treat them in one way for one hand, but treat them a different way for the other hand.
Natural weapons aren't "wielded" - they're body parts
I would strongly disagree with the idea that natural weapons such as claws can be considered to be "wielded" at all. They're a part of your body, just one that can be used to attack in a way that does more damage than a punch or headbutt. You can wield a sword, but you don't "wield" claws (or a bite, or a stinger).
(The same logic applies to unarmed strikes. They're also a part of your body, they're explicitly not weapons, and you don't "wield" your fists any more than you wield your head or your feet.)
However, natural weapons are weapons, according to the SAC
As of October 2020, the Sage Advice Compendium now explicitly addresses whether natural weapons are considered weapons:
Are natural weapons considered weapons?
Things designated as weapons by the rules, including natural weapons,
are indeed weapons. In contrast, unarmed strikes are not weapons. They
are something you do with an unarmed part of your body.
This somewhat contradicts an unofficial tweet from May 2018 by rules designer Jeremy Crawford:
I'll boil it down for you. When we wrote the "Player's Handbook," we meant the weapons on the weapon table when we wrote "weapon," unless we said otherwise.
However, these statements can be reconciled fairly simply. (For context, the Twitter conversation centers around UA: Centaurs and Minotaurs; in their playtest versions, both races had "natural melee weapons", though they were not noted as being usable for unarmed strikes.)
Natural weapons are considered weapons (though a separate category from simple and martial weapons). Crawford's tweet is merely an admission that the designers simply did not always consider natural weapons when designing every game feature that references a "weapon", so some elements of the game were designed with the intent of working with natural weapons while others were not.
Crawford essentially confirmed this inconsistency in another tweet elsewhere in the same thread (responding to questions by game designer Dan Dillon, who had not yet joined WotC's D&D team at that time):
That [2016 tweet saying "Their attacks count as weapon attacks,
but the system doesn't consider their body parts weapons."] was part
of a discussion about the magic weapon spell. I stated that it was
meant to be cast on weapons that are objects, but that casting it on
natural weapons would break nothing.
Gotcha. So paladins can smite with whatever natural weapons by intent,
and magic weapon at DM's discretion?
That's right!
Of course, what this means is that it's impossible to divine the intent behind whether a game feature that refers to a "weapon" is intended to apply to natural weapons or not. Even when it's not, it doesn't break anything in the game to let the game feature apply to natural weapons as well... But in this case, even a logical reading of the rules doesn't allow natural weapons to benefit from the Dueling fighting style.
Given that they're not wielded or treated as weapons for most game purposes, natural weapons don't allow you to benefit from the Dueling fighting style. (Correspondingly, they also don't prevent you from benefiting from Dueling; they're body parts, not "wielded" weapons.)
Best Answer
The damage from Unarmed Fighting is not an attack, and only attacks are made magical.
The Insignia of Claws says:
“Such attacks” here refers to your unarmed strikes and attacks made with natural weapons. It only makes your attacks magical. Features that are not attacks are not made magical. Since the damage from Unarmed Fighting is not an attack, it is not made magical by the Insignia.
That said, there’s nothing wrong with a DM allowing the damage to be magical. Nothing about the game is going to break.
Monster resistances are inconsistently worded.
For some monsters with resistance, it won’t matter if the Unarmed Fighting damage is magical or not, since it is not an attack, and many monsters have resistance to bludgeoning only from nonmagical attacks. For example, an Air Elemental;
However, some monster resistances do not discriminate the source of the damage. For example, the Archmage’s resistances are:
So if a monster’s resistance specifies attacks, then the Unarmed Fighting damage bypasses it whether or not you rule it is magical. If the resistance does not specify attacks, then it only bypasses resistance to nonmagical bludgeoning if you rule that the damage is magical.