Have you considered a remote for the house end of the garage hookup? I'm thinking a z-wave controllable light switch or something in the garage, and a remote control in the house. That way you can ignore the 12/3 romex and weird wiring (of that part of the circuit) and get on with your day.
I'm unclear whether it's allowed by code (especially with the colors you have), but with 3 wires, you've got enough for a single 30 amp circuit - ground, neutral and hot. Is that enough for the garage? That would at least be safe.
tl;dr - if you are going to all the work, and a subpanel, you presumably want a bit more than 20 amps (think it needs to be 30 amps minimum for code these days, and 60 amps is probably better.)
You'll have to dig a ditch. At that point, my opinionated opinion is that you should go ahead and put in conduit, and an additional conduit for any current or future possibility that you might want cable, phone, network, etc out there. Ditches are expensive and a lot of work - conduit is cheap, once you have the ditch...Often cheaper (with wire) than "direct burial" cable, and far more resistant to damage in the future - plus it does offer you the potential of pulling out the wire and pulling in new wire if there ever was a problem - but that's low odds of you making any use of it on he electrical side. Network, quite possible.
Before digging a ditch in a city (especially) backyard, call Dig-Safe and have all (Offically known about) services (gas, phone, electric, water sewer & things you may not know about) located. Turn off the circuit to the garage - I would not worry too much about where it is (unless it runs in a conduit that you might be able to re-use - which is not too likely), but odds are that you'll find it when digging, and it's less exciting if it's turned off when you do.
Any wire used must be rated for wet locations - not difficult, just be sure it is. Any exterior conduit is assumed by code to be wet (and that's generally true.)
If the portion of the backyard you are crossing is not travelled by cars and trucks (not crossing the driveway) depth is sufficient if the TOP of the conduit is 18" below finished grade. Be sure to lay "buried electric line below" tape in the top 6" of the trench fill. If you are not digging below frost line (4 feet or more where you are, probably) you definitely need to bring the ends of the conduit up vertically at buildings, and provide a slip (expansion) joint, as the conduit will move with frost. That's generally needed even if the conduit is buried below frost-line as well, unless it's going straight into a basement below frost-line, but its especially critical when the conduit is above frost line. If you don't find the cost phohibitive, a layer of XPS foam over the top of the conduit provides one more indication that there is something there (when someone else is digging, later) and can reduce frost movement a little bit (or a lot if it's wide.) Alternatively, 2" of concrete over the conduit provides some serious protection on top of the conduit, and reduces the required depth to 6" in Rigid or IMC metallic conduit (which may be well worth it in your situation to save on digging) or 12" in PVC conduit.
If you happen to want a walkway that would happen to run where the electric service would, a 4" thick concrete slab extending 6" beyond the conduit reduces the required burial depth to 4" (ie, right under the slab.)
Look for NEC table 300.5 for more detail.
Best Answer
You can reuse the wires one of several ways.
Run a separate ground wire
Since it is a retrofit, you can add ground wire. (NEC 240.130). The 3 circuits can share one ground wire if they all originate in the same panel (as of NEC 2014‘s new 240.130D). The ground wire must be installed properly, and an appropriate size for each circuit it's grounding. If your fattest conductor is 10 AWG, a 10 AWG ground wire can protect all 3 circuits under NEC 2014. If I recall, 8 AWG bare copper wire is readily available.
Hots and neutrals from the same circuit must be routed together. A 240.130 retrofit ground may follow a different route.
If an existing wire is already bare and appropriate size, you already have your ground wire!
Wire larger than 6 AWG can be redesignated as ground by wrapping it in green tape at each accessible point.
Ask your inspector to re-designate
Generally you cannot re-designate a conductor to be ground, although an entirely bare copper wire can only be a ground.
Ask your local inspector if there's a way to allow this anyway. Local jurisdictions can override Code.
Forget ground and use GFCI
GFCI's may be required anyway for your new circuits.
You can use the GFCI rules, which allow ungrounded 3-prong outlets when fed by a GFCI breaker or outlet. All the outlets must be labeled "No Equipment Ground".
They must also be labeled "GFCI protected" if they are a plain outlet fed by a GFCI upstream.
In this case, take care not to connect ground to anything. Nothing is safe except a ground wire run all the way back to the main service panel.
Make the barn a main panel, isolated by a transformer
This is a complicated one. Wire the barn as a main panel. It must have its own ground rods and neutral must be bonded to ground in the panel. However this neutral must not bond to the neutral back at the house.
It is isolated from the house by a transformer. This is a "mini" version of the service the power company provides to you.
The key is a transformer of appropriate size. It must be single-phase and with a VA rating exceeding your planned load. I often see them on Craigslist at sane prices, e.g. 5K VA for $100. Typically, you can jumper the secondary to either supply 240/120 split phase like your house, or 120-only at twice the amperage.
For amp capacity of a transformer that you've jumped for 240V, easy peasy - divide VA by 240. For instance, A 5K VA transformer is 5000/240 or 20.83 amps. It can be supplied by a 20A breaker and 12 AWG wire, and can output 20A to each side of the panel. If jumpered for 120V-only, it can provide twice that or 40A.
Main breakers that size will be tough to come by, don't bother, just backfeed a regular breaker.
Multiple circuits? Somewhat.
Generally NEC allows only one circuit per voltage. So you can have 240V on one pair of wires, and 120V on a second pair.
You can have more circuits for specialized uses which require a separate circuit, for instance the third pair of wires can be used for lighting controlled from the house. I would use that loophole to power the barn interior lighting on a separate circuit from the outlets - that way if the table saw kicks and trips the breaker, you aren't plunged into darkness with your hands 3 inches from a spinning blade.
Re-designate wire? Mostly no.
Generally, wire function is decided by color, and they cannot be re-designated unless they are larger than 6 AWG (at the request of distributors, who do not want to stock multiple wire colors in large sizes). However, neutral (white or gray) wire can be re-designated hot (e.g. to allow 10/2 to feed 240V loads). Also, an entirely bare wire can only be used as ground.