[RPG] The Relentless monster trait can prevent a certain amount of damage from downing the creature; is the amount arbitrary

How is this difference reconciled? Is there something I missed about Nimble Escape?

It isn't. No.

It has been said before on this site (but damned if I can find it) that the CR of the monsters in the Monster Manual were not set using the methodology set forth in the Dungeon Master's Guide - there were established by eyeballing and playtest.

Notwithstanding, the DMG does say (p. 237):

Creating a monster isn't just a number-crunching exercise. The guidelines in this chapter can help you create monsters, but the only way to know whether a monster is fun is to playtest it. After seeing your monster in action, you might want to adjust the challenge rating up or down based on your experiences.

See Ogre CR calculation: is it wrong or am I missing something? for a calculation that goes the other way.

Think of the CR calculation as trying to work out how to fly a spacecraft to the moon with calculations on the back of a napkin.

In Unearthed Arcana: That Old Black Magic (2015), they playtested a Tiefling Variant with an Abyssal Tiefling subrace. The Abyssal Tiefling gets this Abyssal Arcana trait instead of the original Infernal Legacy trait:

Abyssal Arcana. Each time you finish a long rest, you gain the ability to cast cantrips and spells randomly determined from a short list. At 1st level, you can cast a cantrip. When you reach 3rd level, you can also cast a 1st-level spell. At 5th level, you can cast a 2nd-level spell.

You can cast a spell gained from this trait only once until you complete your next long rest. You can cast a cantrip gained from this trait at will, as normal. For 1st-level spells whose effect changes if cast using a spell slot of 2nd level or higher, you cast the spell as if using a 2nd-level slot. Spells of 2nd level are cast as if using a 2nd-level slot.

At the end of each long rest, you lose the cantrips and spells previously granted by this feature, even if you did not cast them. You replace those cantrips and spells by rolling for new ones on the Abyssal Arcana Spells table. Roll separately for each cantrip and spell. If you roll the same spell or cantrip you gained at the end of your previous long rest, roll again until you get a different result.

Abyssal Arcana Spells

d6	1st Level	3rd Level	5th Level
1	Dancing lights	Burning hands	Alter self
2	True strike	Charm person	Darkness
3	Light	Magic missile	Invisibility
4	Message	Cure wounds	Levitate
5	Spare the dying	Tasha’s hideous laughter	Mirror image
6	Prestidigitation	Thunderwave	Spider climb

I would assume that homebrew could trust this list to be at least somewhat balanced as possible spell replacements, since it made it to UA. However, keep in mind that UA is not strictly canon, but rather playtest material from WotC game designers.

In that regard, the Infernal Legacy trait looks a lot like a Dragonmark (from Unearthed Arcana: Eberron (v1.1)) or the Magic Initiate feat (PHB, p. 168). You could also skin one of those feats into a racial ability, as a homebrew that wouldn't deviate too much from the rules.

And completely RAW, you could always do a Feral Tiefling with the Devil's Tongue trait (SCAG, p. 118), too.