Yes, movement may continue [at a penalty] when special movement speed has run out

You've definitely called out the correct section on movement speeds found in the book, but it's worth noting that in the time since the original printing of the PHB, there was errata added to make this kind of movement more explicitly clear:

Climbing, Swimming, and Crawling

Each foot of movement costs 1 extra foot (2 extra feet in difficult terrain) when you're climbing, swimming, or crawling. You ignore this extra cost if you have a climbing speed and use it to climb, or a swimming speed and use it to swim. At the DM's option, climbing a slippery vertical surface or one with few handholds requires a successful Strength (Athletics) check. Similarly, gaining any distance in rough water might require a successful Strength (Athletics) check.

—Movement, Player's Handbook, pg. 182 (with 2018 Errata)

So what we see here is that the movement cost only applies so long as the creature does not use their movement speed for that particular type of movement. It doesn't say that you're unable to use your walking speed simply because you have a climbing speed.

So if you're using your climbing speed, you'd use 1 foot of movement for each foot you move. Then, if you run out of climbing speed, you can keep going at a rate of 2 feet of movement for each foot you move. As a result, your third option, moving 15 feet with climbing speed, then moving 25 / 2 feet with their remaining walking speed is correct; though it should be noted that Rules-as-Written, there's no requirement to round distances, since 5th Edition does not "snap to a grid" by default. So it's perfectly legal for a character to move exactly 12.5 feet (or in this case, 27.5 feet in total) during their turn.

With certain items, the distinction is meaningful

By default, there is no meaningful distinction between having a climbing speed and being able to climb without spending extra movement, assuming the resulting speeds are equal. However, certain items affect your walking speed without changing your climbing speed:

• Boots of Speed double your walking speed specifically. A Tabaxi PC's climbing speed is unaffected by them, but they could keep climbing if they have walking movement left.
• Horseshoes of Speed increase a horse-like creature's walking speed by 30 feet. (As an aside, the UA Centaur has the Equine Build feature, which makes certain climbs cost 4 extra feet instead of 1 extra foot. RAW, Second Story Work and Athlete would (humorously) negate this.)
• Boots of Striding and Springing set your walking speed to 30 feet.

With these items, it is generally more beneficial to be able to climb using your walking speed without penalty, than it is to actually have a climbing speed. For example, using their movement:

• A Tabaxi PC (Speed 30', Climb 20') climbs 25' (20' + 5', with the additional 5' coming from half their remaining walking speed).
• A Tabaxi PC with Athlete (Speed 30', Climb 20') climbs 30'.
• A Tabaxi PC with Boots of Speed (Speed 60', Climb 20') climbs 40' (20' + 20').
• A Tabaxi PC with Athlete and Boots of Speed (Speed 60', Climb 20') climbs 60'.

The original PHB seemed to imply that simply having a climbing speed was enough to avoid spending extra movement while climbing, even if you weren't using your climbing speed. A stealth change was made to the Climbing, Swimming, and Crawling section around the time that the November 2018 errata came out that clarifies things (D&D Basic Rules 2018, p. 67; emphasis mine):

Each foot of movement costs 1 extra foot (2 extra feet in difficult terrain) when you’re climbing, swimming, or crawling. You ignore this extra cost if you have a climbing speed and use it to climb or a swimming speed and use it to swim.