Are expansion gaps needed when gluing down linoleum

Either way will work. My first impression would be to try to remove the old lino, especially since there is so little of it. If it turns out it too difficult, then simply remove all the loose lino, lay down your foam underlayment and go right over the whole mess. Good luck.

I don't see any nail-heads or screw-heads in the carpet edge trim, so you probably have one like this:

You might be able to save it, but they're cheap so I'd just replace it with a new one. Pry it up taking care not to damage the door trim. The carpet in the door opening should be loose enough that you can get access to the linoleum underneath (there may be a tack strip across the opening but you should still have a little room to play with). Finish removing the old linoleum flooring, then install the new flooring, gluing it down as you would normally. Remove the old edge trim from the carpet -- I'm familiar with ones that are folded onto the carpet edge to hold it, so just bend it back. Install the new one at the edge of the linoleum. Push the carpet into the new edge trim, making sure it's taut, then bend the trim down to hold the carpet; use a piece of 2x4 cut to the width of the opening so that it bends evenly.