Surely you mean punishing the character not the player?
The character should have a whole lot of horrible things happen to them. Wake up in a gimp suit? Be kept as "blood on tape"? Raped(1)? At least wake up naked covered in marker pen with inappropriate (and bad) drawing on his face. Certainly, the character should have been robbed of all they had on them.
Maybe that's what the player wants to happen. Are they going for a slow but sure decent into despair? Trying to gain whatever little control they have since the curse is playing havoc with their free will? ...
Alternatively, do have a talk with the player asking them about it. Depending on their reactions, you can either kick them out of the game, let it slide as a "whoops, wasn't I lucky", or commence with the poor character's descent into hell.
(1) Rape is a horrible, violent, and vile thing. It should be reported to the Police if after the fact. Each and every one of us should step in to stop and prevent it, if we ever have the misfortune of being around the event. Most rapist are well known to the victim, this is why "No Means No" is an important message. I hope this state my position on the issue clears: it is a wrong, despicable, vile, and horrible act. However, I have dealt with rape, racisms, mass murder, sociopathy, psychopathy, insanity, torture, paedophilia, xenocide, genocide, and many other horrible things in more than one game. Note that if any of my players objected to a theme, it was dropped without much ado. There is always a full disclosure of themes before I run a game. Some of my players were affected by the above themes and agreed to have them run within my game.
It's pretty reasonable you're annoyed. One of your fellow players secretly plotted to kill your character for revenge (and it worked), the DM - the one guy you pretty much have to be able to trust - was in on it, and your fellow players offered you no emotional support at a point when you clearly needed it and instead made things worse for you.
People have been annoyed over character deaths, but you also have the issue of a betrayal of trust.
Different groups (and different DMs) handle stuff like this in very different ways - some handle it well, some badly. You had a conflict within the party, and a conflict between two people, and some character death and secret plots on your hands. These are often things groups don't talk about beforehand, but should. As the DM, I wouldn't have allowed this secret plot and would've talked to you two outside the game to get this enmity settled at first signs.
(And honestly, if this was real out-of-game racism you were experiencing, I wouldn't have tolerated that either.)
So now the guy who betrayed you is the DM and you want revenge.
Don't do it.
A good plan is to decline the invitation, not play, leave, and find something else to do or another D&D group to play with. This is not advice for how to ruin Bob's day. This is advice for how to avoid having your own next two months ruined, and possibly several weeks or months after that as well, and instead have some degree of peace for yourself.
You're pissed off. But, proverbially:
Holding onto anger is like drinking poison and waiting for the other person to die.
— (Not sure who first said this)
So you're going to have several gaming sessions and two months in which you're busy being pissed off at Bob, stressed out working out how to get revenge, and suffering over the lot of it - and expecting him to suffer for it. Eventually. At some point. Maybe.
That's not working out in your benefit.
Here's how the next two months are going to pan out according to this plan.
- Bob's the DM. The universe of the game you're considering bends to his whims.
- He doesn't like you to begin with, resents you for something you did to him, and apparently is subtly vengeful. He might give you a hard time for the next two months. This is going to feed right back into you being even unhappier in general, and unhappy to be in that game. This may even increase the degree of revenge you want, something you might not even get to begin with.
- You're going to be preoccupied being annoyed at him and not really actually just enjoying a good game of D&D. (Not that this game is going to be a good game of D&D for you necessarily anyway.)
- Eventually, you might actually find a way to get back at him. But, since he's the absolute controller of the game's universe, he can just say: "Oh. Well, my guy's goddess smiles upon him and heals him. Then she teleports your character into the plane of fire. Alright, whose turn next?"
- Or you don't get anything out of it and you're just annoyed.
- Or a month in, before anything even really happens, he or your fellow players ask you to leave because you're not being very fun to play with. You probably won't be. You're here for revenge, not to enjoy a good game.
- And you're playing with this guy as your DM the whole time, again. That's worth repeating. Why would you want that?
This entire plan is toxic to you and you alone. It's going to be really unpleasant for you, and more likely than not won't get anything out of it - and if you do, it probably won't be very satisfying.
Don't do it.
But I really want revenge!
This is an issue between you and another person, not an issue D&D will help with. Don't try to solve it through D&D, and don't try to solve it by playing with this guy as a DM for a few months.
Deal with it out of the game somehow. Maybe talk to him and get stuff off your chest. Consider walking away and leaving it behind you rather than let this weigh on you. Different things work for different people. I don't know what will work for you, personally. But spend these two months of your life doing something else. Find peace, somehow.
A surprising option is forgiving him - and not for his benefit, but for yours. Forgiveness is just as often so you can stop holding onto the negative emotions you have - the ones which are affecting you much more than anyone else - and let them go and find some inner peace. You might not be prepared to do that, but I advise you try it.
D&D is normally not like this.
(But sometimes it can be.)
There are going to be groups more supportive of first time players. There are going to be groups where you can genuinely be friends with most people, get along with all of them, and reliably trust the DM and your fellow players. This wasn't one of them, clearly. I suggest you find a group that suits you if you want to keep playing D&D.
One thing that normally goes unrecognised is that several D&D players at the same table, playing the same session at the same time, are usually not even playing the same game. They have different expectations of what a good game constitutes, different ideas of what's OK and what's not, different understandings of the rules, and different opinions over how issues like loot and character death should be handled. Usually, they assume that everyone else shares a similar view, without realising that everyone probably thinks very differently - to each other, as well.
An analogy is having several people sitting around a table to play a game of cards, but they are playing different games: one person is playing Poker, another Hearts, and another Go Fish. That wouldn't work very well, but somehow that's how D&D gets played without anyone realising it.
This disconnect is the reason why someone authored the Same Page Tool - which, as its name suggests, exists to get people on the same page. Its author also wrote about The Roots of the Big Problems and A Way Out (from which I drew the card game analogy). All three of these links discuss this issue and the situations that arise from it, and how to deal with them.
This group was not on the same page as you, and were not what you were after. Another group might be.
Best Answer
You have several options here, all of which include changing the players' tactics and/or changing your own.
Enforce consequences for the PCs' murderous behavior
If someone came into my castle and started murdering my cooks, I'd have my guards after them so fast they wouldn't see it coming! Just because it wasn't in the written records doesn't mean it didn't happen, especially if the written record is two thousand years old. The castle lord should respond reasonably to having a bunch of murderhobos running around his castle. Putting consequences to their actions should help encourage the players to be less aggressive about meeting new NPCs, which in turn should alleviate some of the problems with their "if I can knock it out it's a servant, if I can't it's a monster" mindset.
Give the party a reason to bring along an NPC
Maybe the party's goal is blocked by an enchanted doorway that will only open for a specific person. Maybe they can't find their way around the castle and need a guide. Maybe they need a hostage to deal with the guards who are coming after them for attacking servants. Whatever the reason, make the players realize that they need one or more NPCs alive and cooperating with them.
Give your NPCs some teeth
You mentioned you tried giving your NPCs some stats, but it's not clear from your question why this didn't work. Not all NPCs should be level-one minions, regardless of what your players want. Guards will have a few levels in some melee- or ranged-weapons class; people in robes are wizards or sorcerers on retainer; the nice woman in a simple dress is a humble but powerful cleric; the nobles are trained in the fighting and/or arcane arts as part of growing up rich. You should also check the various Monster Manuals for low-level commoners - usually these show up under keywords like "cultist" or "guard", but if you reskin them to wield butchers' knives instead of swords, or broomsticks instead of staves, you've got perfectly good servants whose lifetimes of backbreaking work mean they're tough and strong and perfectly willing to fight back when attacked. Once your players get used to the idea that not everyone who isn't a PC or an enemy is a one-HP minion, they'll stop jumping to conclusions based on perceived stats.
Basically, options 1, 2, and 3 are meant to show the players that their current salt-and-burn tactics will cause their mission to fail. They encourage the players to find solutions to problems that don't simply involve a sharp blow to the head. The fact that they also give you more room to use doppelgangers is a handy side effect.
Have the doppelganger play along to deflect suspicion
The PCs knock out everyone they come across? Sure! The doppelganger plays along and pretends to be unconscious long enough to fool the party, then gets back up and goes about its business.
Get one of your players in on the doppelganger action
I did this in my current game and it was beautiful: the PCs didn't think through the consequences of leaving a single guard alone in a hallway in a doppelganger lair, so I sent an IM to the player asking if he'd go along with his character being ambushed, knocked out, and replaced by a doppelganger. The player had a ton of fun playing the doppelganger pretending to be his character, and the rest of the group took considerably longer to clue in because the player was involved in the scheme.
Options 4 and 5 are more about the doppelganger behaving intelligently, plus my favorite DM advice: get the player in on it. The doppelganger wants its mission, whatever that may be (kill the PCs? stop them from completing their mission? something else?), to succeed, and will take reasonable actions to make that happen. Getting one of your players in on this also helps them feel more excited about the plot, and they may even come up with good ideas to fool the other players that you'd never have thought of.
A word of caution: Doppelgangers, by their very nature, will drastically increase the PCs' and the players' level of general suspicion and paranoia. No matter how careful and smart you are about including doppelgangers, you should expect and be prepared for the players to take their own, often drastic, countermeasures. My group, after their one doppelganger encounter, developed the peculiar habit of shooting each other with crossbows to prove they were really themselves.
This paranoia comes because it's frustrating to the players when they can't trust the "rules" of the game world. One doppelganger is problematic enough; when half the NPCs they meet are doppelgangers, the players are likely to lose trust in the DM and the plot (unless you provide a VERY compelling story reason for why there are so many doppelgangers around and impersonating random servants). Once this happens, they're likely to go back to killing everything on sight because they've been burned by doppelgangers a few too many times. I'd strongly recommend being judicious with your use of doppelgangers to minimize this.