This is an extremely tricky and delicate situation. As a DM, I generally do not allow social skills (Diplomacy, Intimidate, etc) to be used on other PCs, and while I've never had the issue come up, I don't think I would allow mind-control magic to be used either. The reason?

People play RPGs to participate in a group story with a character under their own control.

By taking away that player's autonomy, even just at certain times and even just for specific and generally noble reasons, you're taking away that player's fun.

I played under an extremely controlling GM once, and it was a miserable experience. It was so long ago I don't remember the specifics, but one particular incident that stands out in my memory is when my character did something that all the other players considered reasonable, but which didn't follow the path the GM wanted us to follow. The GM had an NPC knock me out (no saving throw, no defenses, just boom! unconscious), throw me in a bag, and literally drag me along in the plot. He argued that it was the right thing to do because it forwarded the plot, but I stopped playing shortly after, because it's not fun when someone else controls my character. After all, why am I even there, then?

One way you might have handled your situation without taking away the player's autonomy would be to talk to her, out of game, about how her character and her choices were interfering with the group and the game. Ask her for ways that her character could be more closely integrated into the group's goals, and discuss what might happen if she insists on playing in a counterproductive manner.

As an example, in one of my more recent games, I played a chaotic barbarian in a generally good-aligned group, not unlike your situation in many ways. My character had been magically altered to have a "trigger" that would send her into an unstoppable rage when set off. At first it was an interesting roleplay opportunity, but later an unfortunate series of events caused my character to kill an important NPC while in one of those rages, which lost our party major face with key political figures. After that session, the GM suggested to me that my character might not be a good fit for the party, and asked me to consider rolling up a new one. I could see the problems and I was aware that the party was upset about the NPC's death, so I agreed.

In my situation, the GM was on the ball, both about handling things out of game, and about finding a way for my character to exit the party gracefully. Your GM could have dealt with the matter differently, by talking to the player out of character, or by telling the group to find a way to resolve it in-character (and making it possible by finding a way around the "All of you" decree).

So in short, yes, it was wrong to take away another player's autonomy and choices, no matter the reason. If it must be done, then discuss it with the player first, and make sure you have their full consent before doing it.

Charmed is a specific game condition. From Appendix A of the PHB (page 290):

Charmed

• A charmed creature can’t attack the charmer or target the charmer with harmful abilities or magical effects.
• The charmer has advantage on any ability check to interact socially with the creature.

So Fey Ancestry gives advantage on any saving throw to avoid receiving the Charmed condition. Or to put it differently, a creature with Fey Ancestry has advantage on any saving throw made to avoid the effect of a spell, trap, or other effect that inflicts the Charmed condition.