I removed the bathroom vanity and found some pipes. What might they be

The new stack in the top right should connect downstream of the toilet. It can pass behind the toilet and then run parallel to the side wall to the corner. This leaves you with a wet vent via the vanity that servers the bathtub and toilet:

                     Vanity    New Stack
                       |          |
                       |          |
   Shower ------------\|          |
                       |          |
                       |   Toilet |
                       |/----     |
                       |          |
                       |          |
                       |/---------/
                       |
                       |
                       |

As a minimum, the wet vent would need to be 2” as shown on your drawing.

Cast iron. Which is conveniently already in the configuration you need at your local scrap yard: old radiators. That is, assuming this doesn't have to be potable.

Other than that I'd guess we're talking about some expensive exotic steel you probably can't wield very easily anyway, like lab equipment.

I'd like to think that regular old black pipe would stand up to this, especially if you could keep the fittings out of the flames and you mount it allowing for expansion.

However and whatever you build it out of, I'd test a mock-up before you start making endless loops only to find it pops your wields or the fittings. It's not the pipe walls, it's the joints. As long as they aren't subject to physical damage, your no where near hot enough to damage the metal.

Also... if you ever do get the pipes hot enough to melt with water inside them, something has blown up. What are we building here and where's the safety valve(s)? ;)