Move the run over a few feet, which would it stick it beside the air duct (I think there's about 1/2" between the duct and the adjacent stud. Is it safe for them to be so close?)
There is nothing I see in the NEC that puts restrictions on running electrical line near HVAC as long as it meets other code requirements. It must be properly anchored to the structure, meaning that when it is run it should be anchored or stapled to the wall studs. HVAC does not count. In absolutely NO circumstances should you ever run electric line INSIDE of HVAC!
Leave the run as is, move Lord Duckington from his throne, cut away the plaster between the two hole, drill holes in the studs and feed said wire through it, fix mess (I think this wall is load bearing, so this might be a bad idea).
A small hole for electrical line in the middle of the stud will not affect the load bearing capacity of the wall by any meaningful amount. This is safe and probably the best option.
Re-wire so that this run is getting its power from the circuit on the wall behind it. They don't have much on them, so I think combining them is probably safe. In this case, would it be safe to have a few inches of romex going from one box to the next and just connect everything with wire nuts?
This is another option and would probably involve less plaster patching when you are done. The only thing you should make sure of is if the lead line is coming to the switch box. If the power is coming to the receptacle or luminaire rather than the switch then this will not work. Make sure to wire everything properly in parallel if you do this.
Pull off the baseboards, drill a new hole in the floor, and move the box a 6 inches to the left (not sure if the lath is behind the baseboards or not).
If you feel that there is enough room there not taken up by the HVAC system then sure, you could rerun it this way from the basement. In fact, with your tall baseboard you can hide the plaster damage and fishtape it up to the new outlet location. Again, please make sure that there isn't a vent running up that wall stud going to upstairs, which seems likely from your photo.
Move
Lord Duckington I am sure doesn't approve.
The breaker was not tripped but I did flip all the breakers off and on
a couple times to be sure.
Based on this and other comments, it sounds more like an open somewhere in the circuit and not a short. The other possibility is maybe (big emphasis on maybe) a bad breaker that didn't trip for a short. Here's some good reading about this subject
Half of the original circuit is powered by one breaker and the other
half powered by a different breaker
(1) Consider physically disconnecting the half supplied by the new breaker from any cable(s) fed by the old breaker. (2) Or, as @Craig stated, simply turn off the old breaker. This is the fail-safe option.
On the same breaker there is the bathroom plug with GFI, light fixture
and fan all working as expected including the GFI.
Based on what you have described, he rewired the portion(s) of the circuit that were acting odd, which implies that your open/loose connection/gremlins are somewhere between the OK portion of the old circuit and what he rewired. If you can't live without power in the bathroom you mentioned, then consider option (1) above.
If you disconnect the old circuit where the new begins, you should be minimizing the chance of aggravating the situation. By turning the old breaker off, you completely remove the chance of aggravating the situation.
Best Answer
To answer your question about handling the wire, I'd suggest you turn off power to the whole house, pull the wire to where you can work with it, spread the black/white/ground wires so they aren't touching, and strip a tiny bit of sheath off the black and white wires. Secure the wire so it's not likely to move around and inadvertently short out. Now you can turn the power back on and poke at it with your multimeter.
As far as handling it in the longer term, the following is based on the assumption that the wire is live, or can be switched live, so should be dealt with properly. If you do a bunch of exploration and convince yourself that it's always going to be dead, then ignore the following.
The textbook thing to do is follow it back to the source and nip it there. However, that might result in a lot of drywall repair.
Failing that, the best code compliant approach I can think of is to put an old work box in the next stud bay and pull the wire back and into it. Strip a bit off the wires and nut them individually. Apply a blank cover plate and you're good. (Turn off the breaker while cutting nearby and working with the wire.)
It does look like you could probably pull the errant wire into the existing junction box, which would be good, but you'd probably not get enough wire in (6") to be code compliant.