The chances of breaking the pipe with equipment are the same whether capped or not. You can minimize the risk further by digging out the riser and plugging the fitting below.
If you live in an area subject to hard freezing, you will need to make provision for draining any lateral line serving the former head. Lines are usually cleared by blowing them out in the fall. A capped riser will have mostly air, but any horizontal pipe serving only that riser will fill with water and need to be cleared. This may just be a matter of removing the cap before clearing the lines.
First we need a Manufacturer and possibly if you can find it, a model number. Irrigation control valves like this have two parts, a diaphragm operated water control valve and a vacuum breaker. Typical Rainbird system shown here.
The control valve uses an electrically controlled needle valve that operates the main diaphram valve through servo action (small force controls large force). Cross section to give an idea how this works. When the needle is seated, full water pressure operates on the upper side of the diaphragm, shutting water flow off. When the needle opens, the pinhole in the diaphragm is smaller than the needle opening, pressure dumps and the diaphragm opens.
Where your leak is occurring is in the vacuum breaker (right hand side below). This device operates to prevent water siphoning back into the water line through the control valve if water pressure drops. It is essentially a check valve with one side open to the atmosphere that closes under pressure and if the parts are damaged through wear, foreign object stuck in the seat, corroded in place, etc. becomes as you've found, a large volume leak under pressure.
This is where having all that manufacturing information is important as most of these can be repaired if you can find the repair kits. Otherwise, it's complete replacement time.
Exploded diagram of the above example Rainbird unit less the solenoid control.
Champion Classic Brass 3/4" Automatic Actuator and Anti-Siphon with Union
Manufacturer is Arrowhead-Champion http://www.championirrigation.com
I've used Irrigation Direct for other product, their replacement parts page. There are other businesses listed. If you have a large irrigation supply anywhere near, they probably carry the product. Arrowhead is a well known manufacturer of brass plumbing fittings.
The Champion product has been around in manual systems since the 1930's. To get the Vacuum Breaker loose, you might need gentle heat on the body. Keep it on the breaker side.
Breakdown of the original Manual Valve and Anti-Siphon with Union
And the bits in the Anti-Siphon Valve that are giving you grief
Best Answer
Sounds like a broken line under ground between the point you are checking the pressure and the Heads, with the pressure at 20 there is probably not enough to push the head up but it should be flowing even though not extended.
When I have this problem I will usually leave the water on for a while and go find the wet spot. Dig down and fix the break. If the pipes were full and you had a cold winter there may be several broken lines.
Also running the water to find the wet spot always gets some dirt and possibly gravel in the line so I will remove the heads and flow all the dirt out , or cleaning the heads then flowing the stuff out then putting the heads back on. Best to pull them first.