[RPG] way to increase the damage done by the claw attack for a Tabaxi

Two basic ways: increase stats, increase number of attacks

One of the features of D&D 5e is that weapons have fixed damage dice unless they have the versatile property: their utility with one hand or two. The claw does not fit into that category the way that a long sword(1d8/1d10) or a quarterstaff (1d6/1d8) does.

Unarmed strikes also have fixed damage output: a tabaxi claw attack is an unarmed strike.

... your claws are natural weapons, which you can use to make unarmed strikes. If you hit with them, you deal slashing damage equal to 1d4 + your Strength modifier, instead of the bludgeoning damage normal for an unarmed strike. (Volo's Guide to Monsters, p. 115)

With no attribute bonus, the claw does 1d4. (Average 2.5).
With a strength of 18, the claw does 1d4 + 4. (Average 6.5)
That's a 160% increase.
Compare that to a normal unarmed strike: 1 + Str mod.

On a hit, an unarmed strike deals bludgeoning damage equal to 1 + your Strength modifier. You are proficient with your unarmed strikes (Basic Rules, p.76)

In short, the tabaxi already has "more damage" with the claw attacks.

With more attacks, the claw does 2d4(5), 3d4(7.5), 4d4(10), depending on how you set up your PC for additional attacks: as Fighter, Ranger, Monk, etcetara. Depending on the strength or dexterity mod applied to the attack, add the bonus to the damage on a hit.

One additional way: score a critical hit

That will add an additional d4 to the damage.

If the tabaxi wants to do more damage (in terms of a larger damage die) per strike, they should use a weapon the way that all of the other humanoid races do (or become a Monk and earn levels in that class).

Be a monk

You can use Dexterity instead of Strength for the attack and damage rolls of your unarmed strikes and monk weapons.

Choosing the character class of Monk will ensure that unarmed strikes are 1d4, per the Tabaxi slashing attack noted above, and then the damage is boosted for monk unarmed strike damage to 1d6 at level 5, 1d8 at level 11, and 1d10 at level 17.

Magic Item: Insignia of Claws

This item cropped up in the Hoard of the Dragon Queen published adventure. It adds +1 to hit and +1 to damage for both unarmed attacks and natural weapons (Tabaxi claws meet both criteria). Whether or not it shows up in your campaign is up to your DM. But it should be OK.

The jewels in this insignia of the Cult of the Dragon flare with purple light when you enter combat, empowering your natural fists or natural weapons. While wearing the insignia, you gain a +1 bonus to the attack rolls and the damage rolls you make with unarmed strikes and natural weapons. Such attacks are considered to be magical.