One trench the whole way will be easier to pull cable through. You could make it direct, one straight line, or closer to it than otherwise.
Fences can get blown over, run into, replaced for service, or removed for aesthetic reasons, which will become more of a pain to deal with if you run conduit along the fence. Also its less pleasurable to look at conduit than to not.
Your municipality or county should be able to tell you your local minimum depth for buried cable. It depends on your jurisdiction if you're asking about code as I assume.
I would recommend oversizing the conduit significantly to make pulling easier and allow for easy upgrades in the future. 2" can't be that much more than 1". It will be more durable and it could save a lot of work later on.
You can save costs on grounding by using bare wire, or using metal conduit as your grounding conductor. But I would not bury metal conduit to avoid dealing with corrosion. Even a bare ground conductor inside a plastic conduit is going to fare worse than a sheathed over time.
Paint a white line where you plan to excavate and call your utility locating service out to mark obstacles. The white line or shape will keep them from needing to mark unnecessary things in far off parts of your yard.
![excavation marking](https://i.stack.imgur.com/rCtxB.jpg)
Rent a trencher and make the work easy on yourself.
![trencher](https://i.stack.imgur.com/OeI0J.png)
Bury conduit larger than you need, because excavation sucks.
Dig depth is often 2foot minimum for nonmetallic, 6 inch minimum for metallic, but ask your codes department what they will approve.
For wire gaguge, use a calculator like http://wiresizecalculator.net/.
![calc](https://i.stack.imgur.com/EqXbP.png)
By my estimate you would want 6awg, but maybe the direct patch can cut down distance and let you use smaller.
With rigid or intermediate metal conduit ($$) you can follow code without going very deep - 6" under dirt, 4" under a concrete cover of at least 4" thickness extending at least 6" to either side. Unless you want a concrete path through your garden along the route of the conduit, that's not likely what you want.
Rigid and intermediate metal are tough enough that you won't likely damage them while digging with hand tools.
PVC needs to be 18" under unless concrete protected.
Table 300.5 of the NEC should clarify all that for you.
Best Answer
The trench actually needs to be deep enough that the TOP of the conduit is at least 18" deep.
Frankly, you'd be better off paying the slight upcharge for schedule 80 the whole way regardless, because it IS a slight upcharge and you would need schedule 80 for the parts coming out of the ground anyway, and it's a lot harder to break.
The difference is exactly what @isherwood's comment calls out - once it's buried, it's hidden and non-obvious, and "the rules" are that conduit hidden underground needs to be protected. Some landscaper planting rosebushes does not expect to find a power conduit 6" underground, and you won't always be around to tell them it's there, so there are rules.
There are ways to go shallower, but they cost serious money (Using rigid metallic conduit, pouring concrete cover that extends beyond the conduit) and are usually not practical for most people as a result. The conduit on the exposed face of the building can be seen and avoided, and is required to be at least schedule 80 (below 8 feet) as insurance against accidental impacts.
If the issue is that you don't want to see it running on the outside of the house, the simplest (and shortest) solution is usually to run it inside the house if you are heading to "the opposite side of the house" - drywall repair is trivial compared to ditch-digging, and you can use EMT or (I'd personally not) NM cable for the "inside the house" part of the run (still need wet-rated wire outside.)