How does one build the character of a villain such that the players (playing heroic characters) hate the villain as much as possible without detracting too much from the quality of the game?
What motivations, habits, etc should the villain have to make them particularly despicable?
Should the villain be based off of the PCs in some way or is it more effective to base the villain off of some objective standard of villainy? If the former, in what way should the villain be based off of the PCs?
In order to back up your answers, please either a) provide examples of published works making use of the technique/method you recommend, b) cite real-world RPG experience evidencing the success of your technique/method, or c) cite published material on the subject supporting your position (DMGs, Academic papers, Dragon articles, blog posts, etc)
The best answer will provide unique insight into the causes behind player responses within the character-driven framework, provide critically successful example works exemplifying the particular method advocated, and provide a source where for information on the method/subject, if desired, can be acquired.
Best Answer
My best villains are based off the what the players care about. I'll talk about methods and some recent (in the last 3-4 years) examples. You may want to check out my 7 Types of Antagonists as well.
What the players care about - Flags
So first off, I tend to play games with explicit mechanics for the players to tell me what kinds of conflicts they're into. That makes it easy for me to figure out what kinds of villains will press on those conflicts in interesting ways. As we play, a key point is finding out what exactly hits a player's hot button which is usually near, but not quite on the Flag they've given you.
As you play, pay attention when the players are really serious about winning conflicts, or when they're going above and beyond to protect something or accomplish something - all the physical cues (body language, voice) that tells you they're feeling suspense are key to getting more on point, though games where you can spend hero points can also give you a mechanical cue as well.
The moment of moral bankruptcy
What usually makes the villain click for players is the point when the villain crosses a single line that makes the players angry. What people often confuse is the idea that this moral line has to hurt the player characters, or their allies - it doesn't. This also can be how a character who is relatively neutral or even supportive of the player characters becomes a villain in their eyes.
This works best if the goals are understandable, but the methods are extreme or twisted along the way.
Justice delayed is justice denied
Along with the specifics of a villain being either an obstacle to goals or twisting ideals, etc. a key part of how you play that out is having a villain the players cannot pay back or overcome immediately. The frustration makes you hate them more, and you really can't wait to finally see them get theirs. Of course, when that time comes, it might cost enough that you have to really decide if it's worth going for.
A good extra is your allies or friends also like this terrible character. It turns the battleground into fighting for their hearts - convincing your allies of your cause and avoiding social fallout for taking action.
Troublesome Villains > Fiat Villains
In the games I run, I don't have villains who simply auto-succeed at doing terrible things - they don't just manage to kidnap your loved ones, or get you ousted without the players getting a chance to prevent it. I don't fudge the dice or simply play a "gotcha" on the players.
What this does is establish a position of trust for us as a group - the players know if they protect or save something, they really did it, I didn't hand the victory to them. They also know if they fail or things go wrong, they COULD have possibly succeeding and stopped the villain, which means they really rack their brains over "what could I have done better?"
In comparison, when you have a game where the villains do their evil by GM Fiat, the players don't feel like fighting harder, they feel cheated and angry at the GM, not the villains. I've played those games, they're not fun.