[RPG] How powerful are mass mind control spells

Fundamentally, this is about learning player expectations and limits. You said it yourself, here:

The problem is that I'm afraid if I so much as make an intimidate check against a player, my friends are not only going to stop being my friends, they're going to flat-out crucify me. How, as a DM, can I run a game that includes fear effects, confusion effects, charms, and compulsions without my friends hating me for controlling them?

There's only one real way to answer that.

Ask Them

What type of game do they want to play in? Is it one where effects like mind rape are in play, or one without any form of mind affecting spells, or something in between? You're afraid to use it because you don't know, and the only way to get past that fear is to talk to them about it.

Maybe they really do hate this type of thing. If that's the case, you'll want to seriously consider avoiding using such an effect (making your players uncomfortable with something like this won't help your game). Maybe they're perfectly okay with it. It's not something we can answer for you, because we don't know them. With that kind of effect, there are a lot of different player reactions.

It could be as simple as sitting down as a group, explaining what mind rape does, then asking if they'd be okay with that effect being used in game. If they tell you they have out of character problems with it, then that answers your question. If they say it's fine, that likely also answers your question, so long as they're not just saying that to conform. I'd encourage anyone with an issue to speak up privately if they don't feel comfortable doing it in the group setting. You can also ask them one on one via email or messaging if you have players shy to speak up in front of the group.

There are tools out there to help with that conversation, and this question's answers can help you with that. I'd suggest you take a look there for tips.

Personal Experience

With my players, I would never use mind rape. They invest a lot of time and effort into their characters, and get attached to them. If they die in heroic combat? Okay. If the player does something foolish and that gets them killed? They tend to be able to accept that in time. (One of my players wants me to try to kill him, because he wants it to be a real challenge so the triumph means that much more.)

But what mind rape does is more insidious, and it alters their carefully created character in ways that can very easily make it not-fun to play for them. I won't deliberately make the game not fun for my players, so I just won't use it.

I will use spells like charm, as that's a temporary effect that can be fun to roleplay, and my players readily accept that. They also have been known to use it.

I've played in other games with players who wouldn't have a problem with it, though. It really depends on the player.

I've played in and run evil campaigns of various sorts in both 3.5 and 4e (though not 5e, I think my learning will transfer), and run into a lot of problems: My Guy Syndrome comes up a lot, as does a tendency to default to a regular D&D storyline only with more stealing of spoons and kicking of puppies to remind ourselves we're evil. Sometimes an evil campaign instead descends into over-the-top motiveless violence until there's no story at all. There's a whole host of at-the-table and in-the-story issues, and I tried many different strategies to address them. Eventually I came up with a framing device which works well for us in avoiding these problems:

Provide the PCs with a Master to guide them toward orchestrated works of Evil.

Start the game with the PCs as underlings/minions/hirelings/apprentices/etc of a powerful evil NPC. The Master has a complicated Evil Plan and he tasks his minions to enact various parts as the Plan progresses: "Bring me the soul of a hound archon," "Raze the border keep," "Steal the Apocalypse Gem," "Help a spy infiltrate the paladin's ranks," and so forth, tailored to the PCs' abilities.

This provides the party a reason to work together despite having different agendas (and working together will hopefully bond them as friends so that they want to continue as a group) and establishes small achievable evil goals that accumulate into an Epic Evil Event.

All you need to do is ask the players to make sure their characters have a good reason to work for the Master: The serial killer likes having his rampages subsidised (and the Master protects him from the Law); the necromancer seeks to learn from the Master's experience and gain access to his libraries of forbidden lore; the mercenary's in it for the money and benefits.

Eventually the Apprentices will surpass their Master.

Expect the party to betray their Master at some point, hijacking his Evil Plot for their own gain: this is not only expected, but awesome. It's the Master's Evil Plot, not yours, and the story isn't about the Master--it's about his apprentices. Consider the Master to be training wheels for evil, setting an example which the party can then follow to surpass and overthrow their instructor as they level up.

This works because Evil Needs Goals.

As Ed describes so well and AgentPaper elaborates in the D&D context, evil needs concrete reasons motivating its actions. The Master provides goals and motives while the players find their feet in the new paradigm, channeling and guiding their exploration of what it means to be evil in ways compatible with the D&D paradigm without simply kicking puppies during a dungeoncrawl.

A word of warning: Alignment is tricky.

D&D has a history of the details and nature of alignment sparking major heartfelt arguments, because D&D alignments are not easily (or appropriately) matched to real-world philosophies and moralities; they're narrative simplifications to support the game's conceits and draw their power from storytelling conventions rather than from genuine moral complexity. Exactly what this means and how to deal with it are beyond the scope of this answer (and possibly this site, although there's a LOT of questions on the topic you can look at), but you should be aware it exists and be ready to talk with your players about what "Evil campaign" means to them so there aren't nasty surprises mid-game.