The primary goal for planning out a 3.x character is to ensure that you receive level-appropriate powers at each level. This is complicated by the fact that it’s extremely unclear what is appropriate at each level; the different classes get wildly different levels of power at various levels (especially high levels).
Worse, the system uses stringent requirements and option “chains” that rely on one another constantly. A character that evolves naturally is extremely unlikely to meet the requirements of the feats, powers, and so on that are now appropriate to the new direction he’s moving in.
If you change directions, and are taking “starter” feats in this new direction (feats a lower-level character taken), you are, by definition, not getting level-appropriate feats.
Thus, yes, it is generally best to think of your mechanical options in terms of what you hope to have at the highest level you expect to hit. If you don’t, when you get past the level you have planned out to, you risk simply not having level-appropriate options to take.
What it takes to plan a character
Spellcasters simplify planning
Spellcasters, particularly the ones that prepare different spells every day, are massively better off in this regard. Spells are just-about-guaranteed to be level-appropriate (since they are leveled), do not rely on knowing certain prior spells (allowing you to change direction easily), and, if you prepare spells, you can totally switch everything you’re doing each day.
Plus, more importantly, spells are by far the most powerful force available to Pathfinder characters. When you play a fullcaster (gets up to 9th-level spells), you following the highest available standard for what “level-appropriate” might mean. A 6th-level spell can be, and very often is, much better than anything an 11th-level gunslinger or monk gets. Mundane and martial characters in Pathfinder are often limited by Paizo’s sense of verisimilitude; spellcasters get a free pass on that, and are only limited by spells per day (which are plentiful by mid levels).
All that power frees you up a lot on your other choices. You can pick weak feats, and it won’t matter very much because your spells are so good. You can even pick weak spells, and if you’re a prepared spellcaster it still won’t matter that much, because you can always just prepare different spells tomorrow.
Mundane and martial characters are much more complicated to plan out
Counter-intuitively, the classes that seem “simplest” at first glance are actually the most complicated. They easily get “locked in” to certain paths, with extremely harsh punishments for trying to find a new path, and there are abundant “traps” you can get locked into that prevent you from getting level-appropriate abilities.
Thus, you have to carefully figure out exactly what you’re getting each level, figure out how that compares to the spells available at that level, and try to make sure you don’t fall too far behind. Inevitably, you will fall behind spellcaster’s potential, but with care and the right group, you may be able to keep up with the actual spellcasters that you play with. You will have to work harder than they do to accomplish that, however.
So you have to look at all the options available to you, pare away the ones that just don’t keep up (and a lot don’t), and then you have to make sure you are hitting all of the prerequisites you need to in order to get those things on time. In many cases, you will have to commit yourself quite strongly in order to accomplish this; you will not be able to have your character naturally evolve in a new direction. In some cases, you will have to deny yourself options you like, because they require too much, pull you too far away from getting the requirements you need in the future, and so on. For example, only a couple of monk archetypes progress well into mid-to-high levels; the overwhelming majority start falling behind very early. Those archetypes are therefore not available to you if you want to keep up later in the game, because they lock you in and deny you the options you need later.
Your DM can help massively
By allowing extensive retraining, merging and modifying archetypes to mix ‘n’ match to get the features you want, and relaxing or eliminating requirements liberally, a DM can dramatically improve your life. It can allow greater ability to change direction, improve the power-level of weaker classes, and ensuring that the stuff you’re getting is actually good rather than just the random tax you have to pay to keep your options open later.
This is probably still insufficient to keep up with spellcasters, if they’re well-played, but it will help. If your group is not optimizing their spellcasters very hard, you can optimize your weaker class and smooth out the imbalance. It’s not easy, even with DM help, but it can be done to an extent.
But your life will simply be easier as a fullcaster.
Specialization, versatility, and planning
The discussion of whether you should plan on a specialty or plan on doing a bit of everything is, again, sharply delineated along the mundane/magical divide.
Mundanes: almost-enforced specialization
For a mundane character, getting a new trick means starting from the 1st-level feats. Thus, generalizing for a mundane character means starting over. As I hope I’ve established, this is a very bad thing. When you’re level 9, those 1st-level feats are long since obsolete; enemies you face have to be prepared for the 9th-level version of your new trick, which means your 1st-level version isn’t going to impress.
Worse, the thing you had been doing for the first 8 levels is now potentially missing a bonus you could have, and your enemies could be prepared for. Missing one isn’t a big deal; the real problem is if you get to 16th level, and have 8th-level versions of both tricks, neither of which actually works against 16th-level enemies who are prepared for the 16th-level versions of those tricks.
This is what happens when you attempt to generalize as a martial or mundane character, and it’s a huge problem. It all but forces you to specialize, and it means you will inevitably run into situations, potentially many situations, where your trick doesn’t work.
(Note: for simplicity’s sake, I’m describing this as a single trick. In reality, you can probably support a few tricks. But I do mean “a few” – we are talking 1-3, maybe 4 tricks here. And they are likely to be quite narrow tricks with significant scenarios where none of them work.)
Casters: freedom to generalize
On the spellcasting side, you can at 9th level pick up a new trick, that is a spell completely different from any other you’ve ever used before, and it will still be appropriate to a 9th-level character. And spells automatically scale with caster level, which means the spells you already know are still just as good as they would have been. Each spell is independent, which frees you up a lot to generalize. Spellcasting means you can simultaneously support many tricks, and that means you are far less likely to run into a situation where none of your tricks work.
How to plan
As a mundane character, you’re almost definitely going to need to pick a specialty, a few tricks, hopefully related, to support. Otherwise you are not likely to be able to maintain level-appropriate features (barring extremely careful build planning). Worse, you can’t just take your pick; you have to choose your tricks carefully. You need to find out how easily enemies can negate it, and make sure your tricks are as broadly effective as possible, how you are going to handle the limits it does have.
As a spellcaster, you don’t need to plan on a specialty, but you should still be a bit careful. You don’t want all your spells to do the same thing; doing that gives up a huge advantage in that you can have your spells do very different things. That said, you still probably want a specialty. This is what you spend your feats and other permanent options on, this is a major part of your character’s identity, this is a bigger chunk of your spells than others. It will make you have a specialty that you can excel in, that, when it works, can be devastating. But don’t sacrifice your versatility. Even if you’re a “blaster,” you don’t need a direct-damage spell at every level: you need one great one, and one that will work when that one doesn’t, and maybe at higher levels you should get another as the others become obsolete. The other spells are utility, answers when direct-damage isn’t a good one, and whatnot. You don’t need to sacrifice versatility to be effective, so you shouldn’t.
Stealth is fun.
Shadowdancer may be one of the most popular Prestige Classes in 3.5e, and that is solely due to the Hide in Plain Sight feat. Many players enjoy the thought of sneaking invisibly to the enemy and rolling insane backstab/sneak attack damage.
Unfortunately, stealth in D&D is not always that fun.
Now, the backstab part is awesome, and that's why most stealth players enjoy it. The problem is that the D&D mechanics as they are played out in most campaigns do not make much of stealth beyond a canned skill challenge. By looking at some good stealth games for the computer, such as Dishonored or Assassin's Creed, we can take some tips and add them to our campaigns.
This is the biggest change that a DM has to foster in his campaign. As mentioned before, the objective of stealth is almost always just to get in some extra sneak attack damage. Stealth gets boring when, in the end, it's only about combat. There is nothing wrong with sneak attacks, of course. Some of the most memorable moments in my campaigns have been sneak attacks (double crit + 4x backstab damage FTW?), but stealth needs variety.
The purpose of stealth is to remain undetected. Let stealth be a tool for defeating encounters. If the players successfully sneak around an entire group of hobgoblins, give them full XP as if they had beaten the fight. And don't just stop there. If you want great stealth encounters, turn it into a real challenge like Dishonored does. Make enemies move around somewhat unpredictably. Have your players use distractions, or find opportunities to pick off the enemy one at a time. Give them bonus XP or a better reputation for being able to complete encounters without bloodshed, similar to Dishonored. Also like Dishonored, make a few combat encounters really dangerous if you rush right into them, and be sure to make that fairly clear through in-game information.
The world is bigger than a grid. Description helps. Open up the terrain for movement, like Assassin's Creed. Let them sneak past the royal guard by balancing across the rafters of the great hall or by sneaking over the rooftops to bypass the thugs waiting for them in the street. Think in 3D even though the grid is 2D.
Light is a huge factor for stealth in a lot of games, such as Amnesia: The Dark Descent. It ought to be very important in D&D as well, what with all the torches, lanterns, and magical lights often found in its environments. Have players make strategic use of light. One campaign, my players doused a torch while the guard was on the other side of the building so that when he came back, he couldn't see them sneaking inside. Unfortunately, the sudden lack of light alarmed him, which leads to another point:
The Chase Sequence
The way you describe the ninja character as cycling through backstab -> run -> hide -> backstab definitely confirms this as a bad pattern of stealth. One of the biggest flaws of the first Assassin's Creed game was how you could stab someone, run like heck, hide on a bench right around the corner, then go back and stab someone else. Rinse. Repeat. Worst of all, until they introduced notoriety in later games, it seemed like everyone forgot what you did.
Dishonored is a much better example of how to do detection and chase effectively. On the very first detection, the enemy is immediately alert and aware of the fact that you do not belong here. Hostility begins right away, and the chase is brutal. In a chase, NPCs do not let you get away unless you do something really daring. In the TV Show Burn Notice, the main character remarks during a narrative in a chase sequence that the only way to escape a chase is to do something that the people chasing you won't do -- like jumping off a roof.
And even if you get away, the NPCs should not just "forget" about you. They should be on high alert until you die or they are convinced that you have been driven off. Enemies on high alert for a stealth PC should not be easy to catch off-guard. In addition, they should not be splitting up alone if they are even reasonably intelligent.
Have NPCs adopt tactics like the PCs tend to act when encountering stealthy foes.
"But I'm the only stealth character on the team!"
This is roughly the ninja player's position, I take it. I've been there. Fortunately, you don't have to be reliant on stab-and-run to be useful. A number of the former tips are intended for stealth-based encounters, but here's what a stealth PC has gotta do to have fun with stealth while your allies are kickin' down doors in the name of Tempus:
1: Wait for the encounter to get started. Be out of sight on the periphery.
2: Sneak up to a squishy target.
An ultra-stealthy character is ideal for taking out priority targets. Then, using other skills, such as acrobatic-type skills, make a daring escape. Not just running away by pure movement points, but dodging between pillars, leaping onto ledges, or tumbling past enemies to rejoin your allies.
Stealth should get you into the fray. Speed and tricks should get you out.
First, this adds variety to your actions as a stealthy character.
Second, it should be hard to lose detection when enemies are tracking your movements so closely.
Beyond combat, a stealthy character can still be a great asset. Perhaps you can open a gate while the party is fighting. Maybe you can sneak into a camp and rescue a prisoner while the party is attacking from the opposite side. Generally, you should avoid going too lone wolf unless your party wants you to do so, because that's dangerous and slows down the game for others. Performing a stealth mission while the party fights a battle has been the best possible scenario in campaigns I have played. It keeps everyone busy, provides a distraction, and lets your group benefit from stealth simultaneously.
TL;DR version: Stealth should be more than dice rolls. It needs to interact with the environment and the intelligence of the NPCs involved. It should be rewarding, fast-paced, and require cleverness more than just sneakiness. When done right, it should give the party big advantages as a whole.
I have (kind of) the same problem as you
I, too, like to create new, flavourful characters. Most of the time upon reading, or sometimes spontaneously, I think of a new, cool concept that I'm dying to try. I flesh it out, give it a story, connect it to the game.
But the thing that prevents me from switching all the time from a character to a new one, is that (to me) a "good" flavorful character is one you develop and make go from the first to the last page of its story.
I can't do so in the span of a party. What is gonna happen to the kidnapped princess of the adventurous plumber if I decide to play instead a blue hedgehog? Giving them a story, and goals, and relatives is how I get attached to them and resist the temptation to ask for a new character too often.
Your flavour should support a prolonged exposure of the character, and be revealed in all its epic glory over time. (Otherwise, it's probably not that deep, if you've depleted it in a single session?)
PS: Yeah, the comment linking to the other thread was meant to help you in case you find an answer in it. I was about to reply in comment, and I started arguing and... well, I had an answer.