To answer the question rules-as-written:
Wherever the DM thinks appropriate. There appears to be no answer beyond that (especially nothing as specific as a table of default destinations) using official 3.5e materials - 3PP d20 material or earlier editions may be another matter.
For a more useful answer, we need to go beyond rules-as-written (which after all is the whole point of having a DM in the first place.) The best answer therefore depends on the style of game being played, of which there's infinite variation but here's a few ideas:
The narrative answer:
Wherever adds a good story twist, which probably means a populated area with NPCs to interact with. If it's the PCs first visit to that plane, a large city would probably be a good way to introduce them to the plane, as well as providing a useful home base. On the other hand, a small settlement would provide for more intimate interaction with local NPCs and lead to easier story hooks.
The conjectural cosmological physics answer:
The 3.5e Planar Handbook (p151) mentions planar breaches between planes, where the dividing line between two planes has worn thin. It follows that this would be a likely entry point to another plane, as the barrier between the two would provide the least resistance at these points. Planar material could be flowing in or out of this breach depending on the relative density of the planes concerned.
The Hollywood movie answer:
As mentioned in the opening question, there may well be a specific location on the destination plane that is related somehow to the point of origin of the travelers. Travelers from the plane of water may arrive in the middle of a lake or ocean, for example, while those coming from the plane of fire may arrive in a volcanic area.
To the best of my knowledge, there's no definitive list of these, and in many cases it's necessary to tweak the idea to something that would be appropriate on the destination plane - arrival from Mechanus may seem difficult to do if the destination doesn't have clockwork, but a ring of standing stones shaped like cogs feels like a good fit.
The random chance answer:
The DM grabs whatever maps or charts they have of the destination plane (sketching something out if necessary), makes up a grid reference scheme if needed, and rolls dice. Having a few easily adaptable generic maps at hand (with some locations that can easily be tailored to any plane) can be useful here. Tables would work equally well if preferred.
The murderhobo answer:
The PCs arrive at the entrance to the lair of planar monsters that need to be killed, because... they're not currently dead enough, and they have valuable stuff to take.
The "Teach them a lesson"/"My players are masochists" answer:
(Thanks to Zachiel for - unintentionally - reminding me to include this)
They arrive in the worst possible location. Covered in flames on the Plane of Fire, drowning on the Plane of Water, inside solid rock on the Plane of Earth, between two rotating cogs on Mechanus, five thousand feet above one of Arcadia's layers, etc. You'll probably never run into the problem of unspecified destinations ever again...
Yes, Plane Shift can bring you to any layer of the 9 hells
Evidence from the Plane Shift spell
The spell Plane Shift specifically says that it can transport the party directly to the palace of Dispater, which is located in Dis, the second layer of Hell. Thus it can clearly bring you directly to the lower layers of Hell.
Getting the material components to enable this transition, however, will be difficult. To utilise the spell you require a tuning fork specifically attuned to the layer of Hell you want to plane shift to.
Finally, Plane Shift doesn't create a portal, it transports the people affected by the spell to a different plane of existence.
Contrast this with the 9th level Gate spell which specifically states:
You conjure a portal [...]
Even if Plane Shift did create a portal, the fact the spell text specifically says it can transport creatures to Layers other than the 1st Layer of Hell would be a perfect example of the specific beats general rule.
Evidence from the DMG
The DMG section on the Outer Planes has two big caveats that we can drive our caravans through down to the lower levels of the Nine Hells:
It states (emphasis mine):
Planar portals don't connect to the lower levels of the Nine Hells by Asmodeus' orders
- If planar portals couldn't connect to the lower layers, there would be no reason for Asmodeus to make this order. As a result we infer two things:
- Asmodeus has told the other lords that they are not to allow the creation of planar portals on their layers (aside from Avernus).
- Planar portals can be connected to the lower layers...they just aren't created by the native denziens, who would likely work very hard to permanently close/guard any that did exist due to the aforementioned orders.
The Infinite Staircase section states
[...] on any given plane, there can be multiple doors to the Infinite Staircase, though entrances aren't common knowledge and are occasionally guarded by devas, sphinxes, yugoloths, and other powerful monsters
- This is certainly one type of portal (using the literal meaning of the word portal to mean door) that connects different planes, which is explicitly designed to connect to every plane of existence.
The DMG also has a specific section on the Plane Shift spell (under Planar Travel), discussing it's limitations.
Best Answer
Teleportation Circle (Sor/Wiz 9) from the PHB does what you need, lasts 10 min/level and can be made permanent. The transport is one-way.
Edit: It has 10 ft. Diameter. So assuming 30ft movement speed, you can send 12 people in as a double move action and they can move clear on the other side. Consider that the teleport is instantaneous. The circle gets ready immediately again after someone went through. That means 120 soldiers per minute. I think it is possible to storm enemy fortifications quite nicely with that. And with some planning you can transfer even faster. Another edit: Read the spell text. You just need to stand on it to trigger it.