There's no intrinsic limit; it depends on the hardware and the software implementation.
What limits the EVM is the design choice of the specific community that uses the EVM, because there is always a trade-off between decentralization and performance.
On one side, there's the full centralization. This lets you control all the data and operations and apply all the scaling mechanisms you want to increase the performance. Usually, you split the workloads in parallel to use multiple CPUs and DBs. The amount of gas/sec is only limited by your willingness to spend money on more hardware.
On the other side, there's the full decentralization. This is a game you play with all the other peers, so you need the lightest workload possible to achieve that, so literally, anyone can run it and stay in sync with all the other nodes. Not considering PoW for consensus but only computing power to validate blocks, Bitcoin is a masterpiece on this side: at maximum, your machine needs to elaborate ~1.5Mb of data every ~10 minutes.
In between, there are all the other choices between performance and decentralization. Ethereum poses that choice to a ~15 sec per block, with a max of 30M gas per block (15M gas target, dynamic up to 30M max).
This limit is what now (mid-may 2022) the community considers a good balance between speed, decentralization, TPS, and security.
But I also remember when, years ago, the max gas per block was 3.1M, then 4.7M ;) And a lot of people were against more raises to 8M, 10M, etc. Even though a raise alleviates the need to have more space for more applications, you also leave behind more and more nodes that cannot sync without proper hardware - an NVMe SSD is currently mandatory if you want to stay in sync with the network - and you move the decentralization bar towards the centralization.
Note: Vitalik set this as a trilemma, not a dilemma, adding security as one of the elements to consider. For the sake of simplicity, I think this can be left out of the discussion.
Best Answer
The Ethereum Virtual Machine (EVM) is the computer that all full nodes in the Ethereum network agree to run. When there is code / data on the blockchain, consensus is needed to agree on what that code does. Everyone agrees on how the EVM should behave, and everyone has the same data on the blockchain, so everyone will compute the same answers. From this perspective, it looks like a single World Computer leading to that description.
Not all nodes perform the code execution. There are different types of nodes and light clients, such as those on mobile or devices with less resources, do not perform all the code execution. However, this means that light clients need to place trust on some service and slightly more vulnerable to getting deceived.
All full nodes perform the code execution for their own security. By doing this, full nodes can verify the computation result on their own, and they do not have reliance on some service. So instead of why is the miner the only node that gets the block reward, it's more about why do other nodes decide to perform the computation (or not).