Like every boom/bust cycle, the "d20 bust" was what happened when the "d20 boom" ended.
What's the d20 Boom?
I don't know if that's an official term, but it's one I use because it works, and it fits the idea of a bust pretty well. If you look back to when 3.0 came out, it did an interesting thing that no game with it's reach had done before: it made it easy to use it in third party material.
This was due primarily to the OGL. 3.0 got very popular in the market, and now there was a way to write stuff for it. Lots of third party publishers jumped on that. Some (like Paizo) did official Wizards licensed and even D&D branded material. Others were able to use the d20 system trademark, which was pretty liberal to use. The most notable exception is probably the Book of Erotic Fantasy, which Wizards blocked from using the d20 logo, so it used the OGL instead.
Here's a list, which may or may not be complete (I honestly don't know). It's really big.
As you can see there, a lot of publishers jumped on board with the d20 system. Far more than we'd seen for any one system in the past. Everybody thought that with so many people playing a d20 based game in 3.0, that there would be a market for all kinds of books. Quite a few of them did well, but...
At The End of Every Boom...
So what's the bust? Take 3.5 changing enough rules that 3.0 books were nontrivial to bring in to a 3.5 game (thus hurting the value of the third party 3.0 content), combined with the sheer number of competitors. While most of the books would sell somewhat, there was simply too many people making too many books for the amount of buyers. At the time, digital distribution for RPG material wasn't a significant thing, and so you had to have the money to do a print run and get a book into stores to sell them (both things require significant upfront investment and can lose you a lot of money if it sells below expectations).
Stack that on top with the sheer number of books that Wizards put out, and the money dried up. What happened is that we saw fewer people making fewer books. That's the "bust" at the end of the boom. It's worth noting that some people still did alright in this market, but the era of people jumping in to make d20 branded books in large numbers was over.
Post-d20
This sounds like a bit of a silly term, since Pathfinder is d20 based (but not actually a d20 system as 'd20' is a trademark owned by Wizards). Pathfinder maintains a state closer to the idea of the d20 system era by having its own OGL and being friendly to third parties (see this related question for info) than D&D itself did, as 4e moved away from that.
In general though, the only way I've heard post-d20 be used is simply for systems that had their genesis in the d20 system, but continued with it after d20 itself stopped being supported. In that sense Pathfinder definitely qualifies: it came into existence in large part because Wizards stopped supporting d20 and the OGL with 4e and left Paizo in the cold with no game to write content for. They had to do something, so they took the 3.5 SRD, made it their own, and here we are. Mutants & Mastermands is also heavily influenced by d20 but isn't a d20 system per-se, hence 'post-d20'.
Another thing that changed after the d20 is digital distribution went more mainstream. Games like Fate can be successful today without needing the financial backing to do a print run and distribution to stores. While they likely won't sell as many copies, the margins per copy sold for a digital product are much higher. That's opened the market up to new and smaller games to thrive, where they likely would have been strangled in the cradle at the height of the d20 boom.
While a game like Fate had a much better chance of success in the market in the conditions created by the end of d20, it's not really a 'post-d20 system' as it's not really a derivative of d20 in any way.
Best Answer
It began with going into a "Dungeon" (the areas underneath a fortress) during the development of the game Dungeons and Dragons.
Origin in Publication: the dungeon is where Underground adventures take place
The dungeon as a place to conduct an underworld adventure is spelled out in the opening paragraphs of Volume III of the Original Publication of Dungeons and Dragons Wilderness and Underworld Adventures on page 3. (© COPYRIGHT 1974 • TACTICAL STUDIES RULES, by Gary Gygax and Dave Arneson).
Why Dungeons and Dragons?
The "Dungeons and Dragons" game was going to be called "The Fantasy Game" as far back as 1971. The founders of TSR had difficulty in selling the game initially. Gary Gygax had been working in game rules and fan magazine publishing since about 1965. He and his early partners (Arneson, Kaye and Blume) doubtless knew how important a title/name/brand was in getting a product to market, even the small market of miniatures and wargames enthusiasts. It was renamed in 1973. The game rules (cf. above) indicate that "dungeon" had become standard jargon during the game's development.
Two Meanings of "Dungeon" in the game's context
By the time of the game's first publishing run, dungeon had two connotations in this new sense:
the generic setting for an underworld adventure and is below ground, dark, and dangerous
the D&D campaign/setting run by a given person.
The second usage is probably as old as the playtests Arneson's playing group, and later Gygax undertook (early 1970's) in the adventures under Arneson's Blackmoor castle.
This jargon (unique to a small community of gamers) is seen in the TSR newsletter The Strategic Review #6 (Volume II No. 1, Feb 1976) on page 7 in one of E. Gary Gygax' articles:
The jargon had apparently become so ingrained -- among the people playing in the settings Arneson (Blackmoor) and Gygax (Greyhawk) used to playtest original adventure milieus -- that the authors saw no need to explain why they used "dungeon" as opposed to something else.
In an interview with theonering.net Gygax said that his initial vision was that this game would be for gamers. It took a little while for the TSR team to see, and capitalize on, its broader appeal. When the initial set of rules was published, it was a case of "gamers publishing for gamers (and we all speak in the same jargon so we don't need to explain it)."
Personal Experience: the second usage had spread, and was present as far back as 1975, when I first began to play Dungeons and Dragons. We would ask one another "whose dungeon are we playing in?" to decide who would be the referee/dungeon master for the next session. It obviously preceded our experience -- we lived in Virginia and the game spread from the Wisconsin-Chicago area (in our case, through a game store where a friend bought the first boxed set any of us had seen). There was no internet to spread memes or jargon at the speed that they now spread.
Per @Lexible's comment on this not always translating well into other languages, like the French edition called Donjons et Dragons ... from the Wiktionary etymology article.
Another example is Brazilian Portuguese, Caverna do Dragão (i.e. "Cave of the Dragon," or "the Dragon's Cave"), which doesn't include a cognate of "dungeon." Crossing languages barriers often leads to idiomatic rather than literal translations.
1Per @RobertF a dungeon crawl has developed into a standard gaming model where a band of diverse adventurers (wizard, fighter, elf, dwarf, ranger, etc.) battle monsters in an underground setting, including a final boss monster -- a literary template being the Mines of Moria from J. R. R. Tolkien's The Fellowship of the Ring.