You haven't stated much about the subpanel in the pool building, and it would be nice to have 100A in the garage given the plans you have, and the plans automakers have for future vehicles.
So I would simply extend the pool circuit as 100A. You can do that with lugging only - you won't need to add a bulky circuit breaker in the potentially small pool panel, since the 100A main panel breaker feeding the pool will also protect the run to the garage, being all 100A.
100A feeder is typically #1 Aluminum, unless you own a copper mine, in which case #3 copper will suffice. I'm not a fan of copper because most splice lugs and subpanel lugs are zinc plated aluminum, and why introduce dissimilar metals and pay more for the privilege?
On the garage panel, I see a need for at least 14 spaces (assuming you graduate to 240V compressor, table saw and welder, plus 4 120V circuits plus two 240V EV chargers. The worst mistake you can make is short yourself on breaker spaces, so think big - like 30 spaces. You do need a shutoff switch in the garage subpanel, and usually people just select main-breaker panels but the amp rating doesn't matter. 30 space panels typically have 150-200A main breakers, which is fine.
Never rely on 'double-stuff' breakers to solve full-panel problems. Most breakers these days need to be AFCI and/or GFCI, and those are simply not available in double-stuff.
It would sure be nice if the pool panel had "thru lugs" for onward connection to the garage. Think about changing the pool panel for one with thru lugs.
You start with an 8AWG copper wire...
You are correct that for a 100A feeder, the associated ground wire needs to be an 8AWG copper or 6AWG aluminum wire. If you are using a four-wire direct burial cable, then just use the ground wire in the cable as it will be quite amply sized for the job it is doing; if you are using PVC conduit, then bare 8AWG copper would be my choice to keep conduit fill down.
...and then upsize from there based on how much upsizing of the other wires you do
When you start upsizing wires for voltage drop control, the size of the ground wire is governed by NEC 250.122(B):
(B) Increased in Size. Where ungrounded conductors are
increased in size from the minimum size that has sufficient
ampacity for the intended installation, wire-type equipment
grounding conductors, where installed, shall be increased in
size proportionately, according to the circular mil area of the
ungrounded conductors.
For instance, using 1/0 aluminum wire for the hots and neutral in your example vs. the 1AWG that is required to meet Code ampacity rules gives us a size ratio of 53.49mm2/42.41mm2 based on the NEC Chapter 9, Table 8 conductor sizes (we can do this math in mm2 since the conversion factors cancel out here) or about 1.26. Multiplying that by the 8.367mm2 area of an 8AWG copper conductor gives us about 10.55mm2 for the resulting wire. This isn't a standard conductor size though, so we should use a 6AWG copper conductor (which has 13.30mm2 of area) or a 4AWG aluminum conductor for this ground wire.
Best Answer
Put it this way. I weigh 210 pounds. Should I buy a latter rated for 300 pounds? Or is 225 pounds the correct ladder to use?
See, that’s one of those deals where more is always better. The goal isn’t to match safety limit to actual use, you only need to exceed safety margins but it works in your favor to exceed them by a wide margin.
The #1 thing you need to care about is spaces
Spaces are the places you can put a circuit breaker. You need one for every circuit in 120V-land, and you can’t rely on using twin/duplex/double-stuff breakers anymore due to Code changes. So your first priority in panel selection is plenty of spaces for now and future use — including potentially converting to a 30A/240V or 50A/240V setup. Since you’re saying travel trailer, that’s a TT30, 30A/120V. Running 240V appliances is out, for now.