It could have to do with the trip curves of the breakers, and/or the ambient temperature differences between panel locations.
Trip Curve
For example, let's say all the circuits on Phase A are drawing 3 times the rated current. The trip curve for these breakers, say they will trip between 10 and 30 seconds at 3 times current. So the total current on Phase A at the main panel is 270 amperes, or 2.7 times the rated current (100 A). The trip curve for the breaker in the main panel, says that it should trip between 12 and 35 seconds at 2.7 times current. If the breaker in the main panel is a bit more sensitive, you can see that it might actually trip before any of the secondary breakers.
Ambient Temperature
As well as a trip curve, circuit breakers will also have an ambient temperature curve. Breakers are designed and calibrated to operate at 40°C (104°F) ambient temperature. If the breaker is operated at an ambient temperature of -10°C to 24°C (14°F to 75°F), the breaker will be able to carry more current. However, if the breaker is operated at an ambient temperature between 41°C to 60°C (106°F to 140°F), it will carry less current and could lead to nuisance tripping.
For example, if the 100 ampere breaker is operating at 140°F ambient, you might find that it actually trips at only 85 amperes.
High Altitude
When breakers are installed at higher altitudes, the current has to be adjusted due to the reduced cooling effects of the thinner air. According to the documentation I could find, the adjustment multipliers are as follows:
- 0 - 6600 ft. (0 - 2011 m) -> 1
- 6600 - 8500 ft. (2011 - 2591 m) -> 0.99
- 8500 - 13000 ft. (2591 - 3962 m) -> 0.96
If you were installing a 100 ampere breaker at 10,000 ft., the breaker should only be expected to carry 96 amperes (100A * 0.96 = 96A).
Though in your situation, the breakers are likely all installed at a similar altitude. So this it likely not the problem, unless the air near the main panel is significantly thinner for some other reason.
After calling my city's building inspectors, I was told the buried EMT can serve as the grounding conductor between a main breaker panel and a subpanel if the conduit also uses grounding strap fittings to bond the conduit directly to a busbar. The panel knockout to conduit/locknut physical contact is not sufficient.
Best Answer
Do not connect the sub panel to grounding electrodes. A grounding conductor must be run back to the main service panel. The main service panel is the only connection to a grounding electrode system in a single building without a separate means such as a genset. The underground metal pipe could be used as an electrode, but it sounds distant. So you should install two ground rods near the service entrance and connect those to the service panel.