Stealth is fun.

Shadowdancer may be one of the most popular Prestige Classes in 3.5e, and that is solely due to the Hide in Plain Sight feat. Many players enjoy the thought of sneaking invisibly to the enemy and rolling insane backstab/sneak attack damage.

Unfortunately, stealth in D&D is not always that fun.

Now, the backstab part is awesome, and that's why most stealth players enjoy it. The problem is that the D&D mechanics as they are played out in most campaigns do not make much of stealth beyond a canned skill challenge. By looking at some good stealth games for the computer, such as Dishonored or Assassin's Creed, we can take some tips and add them to our campaigns.

Objectives

This is the biggest change that a DM has to foster in his campaign. As mentioned before, the objective of stealth is almost always just to get in some extra sneak attack damage. Stealth gets boring when, in the end, it's only about combat. There is nothing wrong with sneak attacks, of course. Some of the most memorable moments in my campaigns have been sneak attacks (double crit + 4x backstab damage FTW?), but stealth needs variety.

The purpose of stealth is to remain undetected. Let stealth be a tool for defeating encounters. If the players successfully sneak around an entire group of hobgoblins, give them full XP as if they had beaten the fight. And don't just stop there. If you want great stealth encounters, turn it into a real challenge like Dishonored does. Make enemies move around somewhat unpredictably. Have your players use distractions, or find opportunities to pick off the enemy one at a time. Give them bonus XP or a better reputation for being able to complete encounters without bloodshed, similar to Dishonored. Also like Dishonored, make a few combat encounters really dangerous if you rush right into them, and be sure to make that fairly clear through in-game information.

Environment

The world is bigger than a grid. Description helps. Open up the terrain for movement, like Assassin's Creed. Let them sneak past the royal guard by balancing across the rafters of the great hall or by sneaking over the rooftops to bypass the thugs waiting for them in the street. Think in 3D even though the grid is 2D.

Light is a huge factor for stealth in a lot of games, such as Amnesia: The Dark Descent. It ought to be very important in D&D as well, what with all the torches, lanterns, and magical lights often found in its environments. Have players make strategic use of light. One campaign, my players doused a torch while the guard was on the other side of the building so that when he came back, he couldn't see them sneaking inside. Unfortunately, the sudden lack of light alarmed him, which leads to another point:

The Chase Sequence

The way you describe the ninja character as cycling through backstab -> run -> hide -> backstab definitely confirms this as a bad pattern of stealth. One of the biggest flaws of the first Assassin's Creed game was how you could stab someone, run like heck, hide on a bench right around the corner, then go back and stab someone else. Rinse. Repeat. Worst of all, until they introduced notoriety in later games, it seemed like everyone forgot what you did.

Dishonored is a much better example of how to do detection and chase effectively. On the very first detection, the enemy is immediately alert and aware of the fact that you do not belong here. Hostility begins right away, and the chase is brutal. In a chase, NPCs do not let you get away unless you do something really daring. In the TV Show Burn Notice, the main character remarks during a narrative in a chase sequence that the only way to escape a chase is to do something that the people chasing you won't do -- like jumping off a roof.

And even if you get away, the NPCs should not just "forget" about you. They should be on high alert until you die or they are convinced that you have been driven off. Enemies on high alert for a stealth PC should not be easy to catch off-guard. In addition, they should not be splitting up alone if they are even reasonably intelligent.

Have NPCs adopt tactics like the PCs tend to act when encountering stealthy foes.

All Alone

"But I'm the only stealth character on the team!"

This is roughly the ninja player's position, I take it. I've been there. Fortunately, you don't have to be reliant on stab-and-run to be useful. A number of the former tips are intended for stealth-based encounters, but here's what a stealth PC has gotta do to have fun with stealth while your allies are kickin' down doors in the name of Tempus:

1: Wait for the encounter to get started. Be out of sight on the periphery. 2: Sneak up to a squishy target. 3: Stab. 4: ??? 5: Profit.

An ultra-stealthy character is ideal for taking out priority targets. Then, using other skills, such as acrobatic-type skills, make a daring escape. Not just running away by pure movement points, but dodging between pillars, leaping onto ledges, or tumbling past enemies to rejoin your allies.

Stealth should get you into the fray. Speed and tricks should get you out.

First, this adds variety to your actions as a stealthy character. Second, it should be hard to lose detection when enemies are tracking your movements so closely.

Beyond combat, a stealthy character can still be a great asset. Perhaps you can open a gate while the party is fighting. Maybe you can sneak into a camp and rescue a prisoner while the party is attacking from the opposite side. Generally, you should avoid going too lone wolf unless your party wants you to do so, because that's dangerous and slows down the game for others. Performing a stealth mission while the party fights a battle has been the best possible scenario in campaigns I have played. It keeps everyone busy, provides a distraction, and lets your group benefit from stealth simultaneously.

TL;DR version: Stealth should be more than dice rolls. It needs to interact with the environment and the intelligence of the NPCs involved. It should be rewarding, fast-paced, and require cleverness more than just sneakiness. When done right, it should give the party big advantages as a whole.

Improvised attacks with a magic item will overcome resistance

As of a 2018 errata to the Monster Manual, and books from the 10th printing on, monster vulnerabilities/resistances were changed from using "nonmagical weapons" to "nonmagical attacks". The section on "Vulnerabilities, Resistances, and Immunities" was also changed to read (MM page 8, emphasis mine):

Some creatures have vulnerability, resistance, or immunity to certain types of damage. Particular creatures are even resistant or immune to damage from nonmagical attacks (a magical attack is an attack delivered by a spell, a magic item, or another magical source). In addition, some creatures are immune to certain conditions.

The following description is given for improvised weapons:

Sometimes characters don’t have their weapons and have to attack with whatever is close at hand. An improvised weapon includes any object you can wield in one or two hands, such as broken glass, a table leg, a frying pan, a wagon wheel, or a dead goblin.

Often, an improvised weapon is similar to an actual weapon and can be treated as such. For example, a table leg is akin to a club. At the DM’s option, a character proficient with a weapon can use a similar object as if it were that weapon and use his or her proficiency bonus.

An object that bears no resemblance to a weapon deals 1d4 damage (the DM assigns a damage type appropriate to the object). If a character uses a ranged weapon to make a melee attack, or throws a melee weapon that does not have the thrown property, it also deals 1d4 damage. An improvised thrown weapon has a normal range of 20 feet and a long range of 60 feet.

From this description, a magic item can be used as an improvised weapon to make an attack as long as it can be held in one or two hands. As an attack delivered by a magic item, this fulfills the requirements to be considered a magical attack as described in the Monster Manual, and will overcome a monster's resistance to nonmagical attacks.