tile
My basement bathroom has a concrete floor substrate, on which I will be installing and also have a baseboard of stone tile. Some of the wall meeting the floor will be tile (currently cementboard)
and some regular drywall but even the drywall will have tile baseboard.
Should I tape and thinset the corner where the wall currently meets the concrete substrate and should I do it for both the drywall as well as the cementboard, because of the tile baseboard?
You could use an aluminum transition strip from the vinyl to the tile. Some types of transition strips are really intended to be installed under the higher tile layer so this may limit your ability to hide fasteners that secure the strip in place. Here are some examples:
...
You can normally find an assortment of different styles of aluminum transition strips at hardware stores and big box home centers.
Joe, sorry to use the "answer" function again instead of adding another comment, I wanted to attach more pictures.
The tile I used was not available with a bullnose so I finished it flat on the sidewall. The edge cant be seen normally so I did not spend time making it perfect, but for your situation I retract my note about never finishing an edge with caulking. If you end flat to the wall around that corner it would be perfectly fine to use caulking for aesthetic purposes, use paintable caulk so you can paint a straight line down the whole side when you paint the wall.
Sorry I misunderstood your question, my situation was similar to yours and wrapping the tile around to the outside seemed like the best solution for me, and it looks pretty good that way (at least my wife did not complain, that's all that really matters). 
Best Answer
Nope. Don't tape the floor to the wall anywhere.
Install the floor first and then install the wall tile.