No, Rage is only replaced when you enter the Supernatural Fury

The way this is worded, the Rage feature is not removed upon reaching level 3. Instead the feature is disabled while in the Supernatural Fury. This means that you could enter a "regular rage" but these features are disabled while in the alternate state.

This shows that the wording is probably flawed

Because of the earlier bit in the feature: "You gain the ability to cast and concentrate on spells, even while raging", you could also cast spells while in a "regular rage." This is almost certainly not the intent.

You have it almost right

The only other thing to consider is the Barbarian's reaction, movement, and free object interaction. If they can attack a hostile creature with any of those or take damage as a result of any of those they can also ensure their rage doesn't end on the first turn.

Generally, the most common way that would happen is for the Barbarian to provoke an attack of opportunity and that attack to be made successfully and result in some damage, but many other options exist; for example, the barbarian could deliberately step on a caltrop or other hazard.

All of those ways, except perhaps getting to use your reaction to attack turning the same turn, are almost always bad and unnecessary-- the Barbarian should just wait until they take damage to super rage instead. Reaction based attacks currently at-best require you to be attacked and most require you to be hit, so those are kind of a bad idea, too, unless you have built around it. Commander's Strike is an exception, if an allied battle master fighter goes before you in the first round of combat and you are not surprised and you are in position to make an attack (it doesn't have to be a melee attack so that last part isn't unreasonable) and the GM rules you can take reactions before your first turn in combat, but it is bad for a battlemaster to use, generally, because it takes both a bonus action and an attack and your reaction and that is a lot of stuff so it's 1) unlikely your allied battlemaster fighter has this maneuver and 2) unlikely it's better for them to use it even if they do have it than to do two additional attack-equivalent thingies and leave you with your reaction available.

Pretty much the only scenarios I can come up with where you'd want to do this are where you are involved in a decent-sized melee and there's an AOE trap you can trigger to hit you and the enemies but not your party and you want to benefit from your resistance to damage and where you have an allied battlemaster fighter as a result of having used a Deck of Many Things and he or she stands in the back with a longbow and uses Commander's Strike instead of attacking.

So, in essence, yes, your reading is correct, but technically no, and the answer might change as better ways to use your reaction to attack during your turn are published.