The Terms Extradimensional Space and Nondimensional Space Are Interchangeable
This becomes clear when looking at how the terms are used in context. The most useful example is the portable hole (DMG 264) (20,000 gp; 0 lbs.), which says that it
is a circle of cloth spun from the webs of a phase spider interwoven with strands of ether and beams of starlight. When opened fully, a portable hole is 6 feet in diameter, but it can be folded up to be as small as a pocket handkerchief. When spread upon any surface, it causes an extradimensional space 10 feet deep to come into being. This hole can be picked up from inside or out by simply taking hold of the edges of the cloth and folding it up. Either way, the entrance disappears, but anything inside the hole remains.
The only air in the hole is that which enters when the hole is opened. It contains enough air to supply one Medium creature or two Small creatures for 10 minutes. (See Suffocation, page 304.) The cloth does not accumulate weight even if its hole is filled (with gold, for example). Each portable hole opens on its own particular nondimensional space. If a bag of holding (see page 248) is placed within a portable hole, a rift to the Astral Plane is torn in that place. Both the bag and the cloth are sucked into the void and forever lost. If a portable hole is placed within a bag of holding, it opens
a gate to the Astral Plane. The hole, the bag, and any creatures within a 10-foot radius are drawn there, the portable hole and bag of holding being destroyed in the process. (DMG 264)
Emphasis mine. Thus, even within the portable hole's description, the terms extradimensional space and nondimensional space are used interchangeably. That's because such spaces are--no matter which term is used--technically demiplanes:
This catch-all category covers all extradimensional spaces that function like planes but have measurable size and limited access. Other kinds of planes are theoretically infinite
in size, but a demiplane might be only a few hundred feet across. Access to demiplanes may be limited to particular locations (such as a fixed gateway) or particular situations (such as a time of year or a weather condition). Some demiplanes are created by powerful
magic, some naturally evolve, and some appear according to the will of the deities.
In the D&D cosmology, also known as the Great Wheel, the planes are connected in a specific fashion, as depicted in the diagram on page 153. (The diagram does not show demiplanes, because the location and even the existence of these extradimensional
spaces is constantly changing.) (DMG 147)
Therefore the extradimensional spaces of the bag of devouring (DMG 274) (n/a gp; 0 lbs.), the mirror of life-trapping (DMG 262) (2000,000 gp; 50 lbs.), the 2nd-level Sor/Wiz spell rope trick [trans] (PH 273), and the 7th-level Sor/Wiz spell Mordenkainen's magnificent mansion [conj] (PH 256) are the same kinds of spaces as the nondimensional spaces of the bag of holding (DMG 248) (2,500+ gp; 15+ lbs.), the pocket paradise created by the rod of security (DMG 236) (61,000 gp; 5 lbs.), and the quiver of Ehlonna (DMG 265) (1,800 gp; 2 lbs.).
Speculation
The Player's Handbook uses exclusively the term extradimensional space, while the Dungeon Master's Guide mixes the terms extradimensional space and nondimensional space freely. I think it was one of those things the authors never expected anyone to care about, hence not worth fixing... not expecting that over a decade after its release such term mixing would bother careful readers of their game.
The spell dimensional anchor may or may not prevent a martial adept from using a maneuver like shadow jaunt
While the spell 4th-level Sor/Wiz spell dimensional anchor [abjur] (PH 221) lists some of the spells, spell-like abilities, and psionic abilities that it prevents, the spell also "completely blocks extradimensional travel," which all spells of the conjuration subschool teleportation employ to commit instantaneous transportation, the subschool's description saying, "Teleportation is instantaneous travel through the Astral Plane" (173).
The 2nd-level Shadow Hand maneuver shadow jaunt [special] (Tome of Battle 79) et al. are presented with the descriptor [teleportation], and the [descriptor] description, says, "Most of these descriptors have no game effect by themselves, but they govern how a maneuver interacts with other maneuvers, powers, spells, or abilities" (44).
So one reading is that because the spell dimensional anchor blocks extradimensional movement, and the [teleportation] descriptor indicates that the maneuvers are teleportation effects, the maneuvers shadow jaunt et al. are blocked by the spell dimensional anchor. Another reading says that because the spell dimensional anchor lists spells, spell-like abilities, and psionic abilities and shadow jaunt et al. aren't any of those and that because the teleportation descriptor is otherwise undefined in Tome of Battle and used, instead, throughout much of the game's remainder solely as an indicator that a spell or effect is part of that conjuration subschool, the shadow jaunt et al.'s descriptor in this context carries very little weight, the spell dimensional anchor being unable to prevent the maneuvers' use and likewise allowing teleporation into and within, for example, an area warded by the 6th-level Clr spell forbiddance (PH 232-3).
Because the official Tome of Battle errata document is a mess, this is likely to go unresolved except by the DM.
However—and whether this is unintentional or by design, we'll, again, likely never know—, Tome of Battle also says, "Unless the description of the specific maneuver or stance says otherwise, treat it as an extraordinary ability" (40), and shadow jaunt et al. do not say otherwise, making the maneuvers some of the few instantaneous transportation abilities that can be used in, for example, an area affected by the 6th-level Sor/Wiz spell antimagic field [abjur] (PH 200).
Best Answer
Incapacitate them and use Dimensional Shackles
The most straightforward analog is probably to incapacitate the creature and then put some Dimensional Shackles on them before they recover. The shackles only require an action to place on a creature, but that creature must be incapacitated (remember that other conditions, including stunned, unconscious, and paralyzed, also incapacitate). Compared to the dimensional anchor spell you've cited from a previous edition, the problem shifts from hitting them with dimensional anchor's attack roll to making them fail their save against an incapacitating condition. Once they are incapacitated, they do not get a save against the shackles. So this puts the shackles on the same order of difficulty as dimensional anchor: one good hit (or failed save) will do it.
However, if an enemy is incapacitated and your goal is simply to kill them before they escape, then using your action to do as much damage as possible might be the better option, since there's a good chance your party can finish them off before they recover from the incapacitating condition and take their next non-incapacitated turn to escape. On the other hand, if you're trying to capture them alive, this is a good option. Of course, be careful of very strong enemies (or if your group plays with automatic skill check success on a natural 20), since they have the potential to break out of the shackles with a DC 30 Strength (Athletics) check. (But note that they are only allowed one escape attempt every 30 days.)
There are a wide variety of ways to inflict the incapacitated condition, including lowly 1st-level spells like Tasha's hideous laughter, so the main obstacle to setting this plan up is the acquisition of the Dimensional Shackles.
High-level option: Wish a dimensional/teleporation ward into existence
There are several spells, such as Mordenkainen’s Private Sanctum, that can ward a large area against both teleportation and planar travel. These would be perfect if not for their long casting times, which make them unusable during combat. However, if you have access to a Wish spell, you can use it to bypass the casting time. Wish has a casting time of 1 action, and can "duplicate any other spell of 8th level or lower. You don't need to meet any requirements in that spell, including costly components. The spell simply takes effect." If you use Wish to duplicate an 8th level Private Sanctum, you can ward a 600-foot cube as an action.
Keep in mind that since you are warding the entire battlefield rather than a single creature, you are also cutting off your own escape via the same means, unless you first use an action to dismiss the sanctum.