Growing up in the early to mid 1990s, visiting various arcades was truly fascinating. At home, the SNES already felt amazingly futuristic, and its games endlessly fascinated and impressed me. Yet, stepping into an arcade was like walking into a time machine ten years into the future. (From my perception and understanding of the world at the time.)
Daytona USA, Sega Rally and Cruis'n USA made my jaws drop to the floor and my eyes to become as big as saucers. As far as my mind could tell, these games were almost or basically at photo realism. The thought of ever playing such games at home felt like a joke; it was basically never going to happen.
Even the older arcade machines seemed ancient in comparison to the "big three" new ones I mentioned above. I mean the ones which used "sprite tricks" to build up a fake 3D world rather than actual polygons. Yet even those older ones were far beyond what consumer consoles up until the latter half of the 1990s could achieve at home.
I remember how jealous I was of this kid who had the lousy Virtua Racing port on his Mega Drive.
By the time I got my Sega Saturn and got to play Daytona USA and a demo of Sega Rally at home, in far inferior but still impressive versions (considering the cheap hardware it was ported to), words could not possibly describe how excited I was for the future.
But then, as the 1990s went on, the arcade games stopped impressing. Sega Rally and Daytona USA did get new games in their series, but they were nothing like the originals. By the time the Dreamcast came out, the gap was basically nonexistent. Instead of continuing to release amazingly beefy arcade machines, they started using the lame consumer console hardware! That took away most of the point of the arcade machines to me, the other of course being that you got to sit in a car-like seat with a wheel and pedals.
But why did this happen? Simply stating that "hardware got powerful enough to make affordable consumer devices" doesn't explain anything, because the whole point was that arcade machines could (and always did) cost a lot more than the consumer hardware, for obvious reasons. And that was their appeal; that they could contain expensive and specialized hardware configurations for each game, and deliver mind-blowing experiences.
In the mid-1990s, I was convinced that by year 2000, it would be possible to pay a buck in an arcade to play true photo-realistic simulations. That never happened. Now it's year 2020 and arcades don't at all exist in the form I think of them. The ones that still exist have nothing to do with either the classic 1980s "golden era" games/atmosphere, nor the 1990s "cutting edge hardware" era. It's just a bunch of lame, cheap PCs racked up with ugly chassis around them.
And before you say so, no, I don't consider anything that can be done at home to be "good enough" by any means. Games/simulations have gone backwards since the early 2000s as far as I'm concerned, less and less trying to look and behave like reality. However, this really is besides the point, because my question is why they didn't just keep using the same (obvious) philosophy for the arcades: beef them up with the most powerful hardware that money currently can buy, clustering it up if necessary, to deliver absolutely mind-blowingly realistic and complex 3D worlds which just cannot be done at home.
I don't understand why they seem to have said at some point in the late 1990s or early 2000: "I think we're done now. Let's stop caring." and everyone seemingly went along with it, even though there were endless opportunities to keep making fantastic and ever more realistic simulations.
Best Answer
Why didn't arcade games keep scaling? Because the problems of making a game stand out scaled in a different direction than the benefits arcade cabinets provide.
Decades ago, as mentioned in the question, a game could stand out by simply having 3D graphics, or doing other things that required hardware that could not practically be included in consumer home gaming devices. As technology improved, that became less and less the case, and now there is essentially nothing you can make a video game do that commercial hardware can't handle. Nicer hardware might get you more polygons or better lighting effects or even better physics simulations, but you can't get the same qualitative difference as 3D graphics in a world of otherwise 2D games.
Once we reached that point, the main challenges of developing impressive new games shifted to things like innovating on mechanics, making game worlds bigger and deeper, improving 3D model fidelity, and others. These things do not necessarily demand more of the hardware, but they require much more development resources.
Considering those shifts, it is often not worthwhile to make an arcade machine, in terms of both user experience and profit.
Let's say you wanted to try the "expensive hardware" strategy of building an arcade cabinet with today's technology. You get the best graphics money can buy: a 4K monitor and 4 RTX 2080 TI connected with SLI. You get the most powerful processor on the market: an Intel Core i9-10980XE. You get all the memory you can cram into a PC motherboard: about 64GB. And you fill it out with more storage than you could possibly fill with one game. This adds up to several thousand dollars worth of hardware, and you can do real-time ray tracing in 4K, which is pretty cool.
Now you need to make a game that actually uses that hardware. You're going to need designers, modelers, artists, animators, programmers, and others to spend a lot of time making a game with the fidelity to take advantage of that hardware. And in the end, this is all still commercial gaming hardware, so you can provide a lower resolution graphics option and the game will run on people's computers at home. And once you sell the game to people directly, most people probably would not consider it worthwhile to travel somewhere to pay more money to play the same game with marginally better graphics than what they can get in the comfort of their own home. Plus, if people buy your game to play at home, you don't have to handle selling hardware.
I recently went to a Dave & Busters, which is probably one of the best places to see modern arcade games. Almost every game there distinguished itself not by being more visually impressive, but by having an input device that you would rarely see on consumer gaming setups, like a steering wheel and pedals, or a gun, or a dance pad, or a single big button. For the most part, they were designed to be easy to understand, and quick enough to finish that other people could get a turn.