To answer the question rules-as-written:

Wherever the DM thinks appropriate. There appears to be no answer beyond that (especially nothing as specific as a table of default destinations) using official 3.5e materials - 3PP d20 material or earlier editions may be another matter.

For a more useful answer, we need to go beyond rules-as-written (which after all is the whole point of having a DM in the first place.) The best answer therefore depends on the style of game being played, of which there's infinite variation but here's a few ideas:

The narrative answer:

Wherever adds a good story twist, which probably means a populated area with NPCs to interact with. If it's the PCs first visit to that plane, a large city would probably be a good way to introduce them to the plane, as well as providing a useful home base. On the other hand, a small settlement would provide for more intimate interaction with local NPCs and lead to easier story hooks.

The conjectural cosmological physics answer:

The 3.5e Planar Handbook (p151) mentions planar breaches between planes, where the dividing line between two planes has worn thin. It follows that this would be a likely entry point to another plane, as the barrier between the two would provide the least resistance at these points. Planar material could be flowing in or out of this breach depending on the relative density of the planes concerned.

The Hollywood movie answer:

As mentioned in the opening question, there may well be a specific location on the destination plane that is related somehow to the point of origin of the travelers. Travelers from the plane of water may arrive in the middle of a lake or ocean, for example, while those coming from the plane of fire may arrive in a volcanic area.

To the best of my knowledge, there's no definitive list of these, and in many cases it's necessary to tweak the idea to something that would be appropriate on the destination plane - arrival from Mechanus may seem difficult to do if the destination doesn't have clockwork, but a ring of standing stones shaped like cogs feels like a good fit.

The random chance answer:

The DM grabs whatever maps or charts they have of the destination plane (sketching something out if necessary), makes up a grid reference scheme if needed, and rolls dice. Having a few easily adaptable generic maps at hand (with some locations that can easily be tailored to any plane) can be useful here. Tables would work equally well if preferred.

The murderhobo answer:

The PCs arrive at the entrance to the lair of planar monsters that need to be killed, because... they're not currently dead enough, and they have valuable stuff to take.

The "Teach them a lesson"/"My players are masochists" answer:

(Thanks to Zachiel for - unintentionally - reminding me to include this)

They arrive in the worst possible location. Covered in flames on the Plane of Fire, drowning on the Plane of Water, inside solid rock on the Plane of Earth, between two rotating cogs on Mechanus, five thousand feet above one of Arcadia's layers, etc. You'll probably never run into the problem of unspecified destinations ever again...

It's pretty much just like you said

Except more familiarity in this case means being very familiar, an actual game term defined in the description of the spell teleport, which says that

“Very familiar” is a place where you have been very often and where you feel at home.

Emphasis mine. So go ahead and pick anywhere that's very familiar. But it's perfectly reasonable for the GM to bar you from preparing the spell word of recall to transport you back to, for example, that prison cell in the evil overlord's dungeon or that brothel where you spent the night a couple of times. (You know, unless you actually felt at home in either place.)

Also note that word of recall is a higher-level spell than the spell teleport, so the spell's allowed to be pretty good.