[RPG] In Pathfinder’s Advanced Race Guide’s Race creator, what does the prerequisite for Advanced ability scores mean

No.

Character Advancement
When adding new levels of an existing class or adding levels of a new class (see Multiclassing, below), make sure to take the following steps in order. First, select your new class level. You must be able to qualify for this level before any of the following adjustments are made. Second, apply any ability score increases due to gaining a level. Third, integrate all of the level's class abilities and then roll for additional hit points. Finally, add new skills and feats.

The order of how you level matters. You're allowed take a feat the same level you gain the prerequisites only because gaining feats is the last thing you do. If the prerequisite is a skill, spell, or ability, you already have it by the time you take the feat.

Of course, if the prerequisite is another feat, you must already have the prerequisite feat before you can take the desired feat. However, since character advancement is an ordered process, if you gain multiple feats in a level¹, you can choose to take the prerequisite feat first. You can now take the desired feat.

In your example, you can take neither feat first since neither feat has its prerequisite met.

¹ Class abilities that give bonus feats give them at the time you normally add new feats (after skill points). It's unclear if you choose your standard feat before your bonus feat or vice versa, but I would generally allow players to select feats in any order if it matters.

The quoted rule about -2 dexterity is for a creature that changes sizes. A creature that is naturally a given size has all its racial ability modifiers already calculated into its race description.

Note: the Trox's RP-built monstrous race and the Bestiary character race option are consistent with each other; the one showing the RP expenditures has the last two points of Strength on a separate line.