This is an extremely tricky and delicate situation. As a DM, I generally do not allow social skills (Diplomacy, Intimidate, etc) to be used on other PCs, and while I've never had the issue come up, I don't think I would allow mind-control magic to be used either. The reason?
People play RPGs to participate in a group story with a character under their own control.
By taking away that player's autonomy, even just at certain times and even just for specific and generally noble reasons, you're taking away that player's fun.
I played under an extremely controlling GM once, and it was a miserable experience. It was so long ago I don't remember the specifics, but one particular incident that stands out in my memory is when my character did something that all the other players considered reasonable, but which didn't follow the path the GM wanted us to follow. The GM had an NPC knock me out (no saving throw, no defenses, just boom! unconscious), throw me in a bag, and literally drag me along in the plot. He argued that it was the right thing to do because it forwarded the plot, but I stopped playing shortly after, because it's not fun when someone else controls my character. After all, why am I even there, then?
One way you might have handled your situation without taking away the player's autonomy would be to talk to her, out of game, about how her character and her choices were interfering with the group and the game. Ask her for ways that her character could be more closely integrated into the group's goals, and discuss what might happen if she insists on playing in a counterproductive manner.
As an example, in one of my more recent games, I played a chaotic barbarian in a generally good-aligned group, not unlike your situation in many ways. My character had been magically altered to have a "trigger" that would send her into an unstoppable rage when set off. At first it was an interesting roleplay opportunity, but later an unfortunate series of events caused my character to kill an important NPC while in one of those rages, which lost our party major face with key political figures. After that session, the GM suggested to me that my character might not be a good fit for the party, and asked me to consider rolling up a new one. I could see the problems and I was aware that the party was upset about the NPC's death, so I agreed.
In my situation, the GM was on the ball, both about handling things out of game, and about finding a way for my character to exit the party gracefully. Your GM could have dealt with the matter differently, by talking to the player out of character, or by telling the group to find a way to resolve it in-character (and making it possible by finding a way around the "All of you" decree).
So in short, yes, it was wrong to take away another player's autonomy and choices, no matter the reason. If it must be done, then discuss it with the player first, and make sure you have their full consent before doing it.
Plot Issues
1. Play out the sidequests anyway in the back burner
In a way this feels like a Shadowrun situation, and quite frankly if the players are chasing their own (maybe literal) ghosts, the factions are going to hire someone available to do it. Let the party handle their own stuff and a rival party gets the job. You can resolve the quest any way you feel appropriate for where the plot will develop, but now the players need to fight for their reputations. The next player plot hook could be buried in the rival groups.
2. Why not mix it up?
If the party is supposed to be a group of characters with high impact on their world, there is a way to make the character plots intercede with the core plots, even if they happen just a little off kilter from what you need to accomplish like the job is straightforward but Bob's old nemesis just can't see it go well, or George's scorned love demands a favor in exchange for forgiveness, or Nell's long lost sister might be in the mark faction's castle/town/whatever. While they may seem very divergent, the plot device has a setting for looping things together in the strangest of ways such that they all become related and keep the party together.
3. PC the NPC / NPC the PC
Do you have a specific way you want the quests to go? Follow one of the players on their sidequest (and anyone else with reason to be involved) and pre-gen NPCs for the rest of the characters to play - including a core enough personality to keep them involved. Meanwhile, the party does their quest. It might not be the smoothest accomplishment but it would then be up to you as the DM for embellishments but do reward the core party with the quest items for the players performing well in the side arc. This also gives you play room for fuzzy memory parts that you need to ad hoc or retcon.
4. Save this campaign for later
If the players are giving you enough plot to run a full game, it may mean that you should save this campaign for when you have a more attentive party - especially one where you can tell the players what they're in for to make characters in reaction to it. This is my least favorite idea but if they have these in depth characters that run parallel to the world you're running, it may be worth considering.
Motivation Issues
I'm hoping these are not the CN players who just enjoy what I call the Video Game Experience. To me, the Video Game Experience is that to the player, anything in the environment they can interact with needs to be important, monsters are just experience, and the plot itself doesn't necessarily need to matter if they get the levels and the loot. In this case they are almost turning themselves into NPCs if their actions really have nothing to do with the party.
It may be worthwhile to sit down with them to explain that these characters need a few tweaks to make the game harmonic. While a character can be selfish, the fact is that it's hard to run a full party with the truly self indulgent within it. Tell them that they can keep their characters, perhaps even as they are but the player becomes responsible at least up front with why the character is with the group. After encountering a situations where the characters don't mesh or the lone wolf amidst the team, DMs in my area have started telling people that even if they make characters on their own, it's up to them to make party cohesion work. Sometimes it doesn't matter and the DM holds the leash for some major plot hook, but the fact is that in the seemingly sandbox world you have, the party dynamic is player driven.
Best Answer
Establishing a Protocol
Many RPG groups have a protocol for handling potentially uncomfortable situations.
At the start of your game, you give all your players a card with an X on it, and you say: "We might touch on uncomfortable topics in this game. If you encounter a topic that makes you uncomfortable, show us this card. We won't ask why the topic makes you uncomfortable and we won't try to persuade you to be okay with it. All that we'll ask is what we need to remove from the story. We could remove it by just skipping past it and letting it happen "offscreen", or we could agree that it doesn't happen entirely."
You can read more about these protocols in our related question: What do the terms "lines" and "veils" mean?
This protocol was intended for use in very improvisational games, but you could also use it in your game. You'd have to think in advance about how you'd change the story if someone objected -- probably you'd say "okay, so they're not children at all, they're some sort of tree-monster, and the children are safely tied up over there."
My Experience
It happens that I have some personal experience of running a D&D game in which players can kill mind-controlled civilians. I frequently run The Orb Of Storms for strangers at game stores; this scenario involves civilians who have been captured, converted into werewolves, and mind-controlled by evil druids.
The group has the option to kill the werewolves, to set them free of the mind control, or to restrain and cure them. I've had groups do all of these things. Sometimes a single group takes all three of the approaches, with different werewolves, as the story progresses.
Nobody has ever had a problem with this. Some groups try to rescue the werewolves; some groups don't care. Nobody has been upset by it.
Your Situation
Your game might be worse, because you're specifically using children as the captured NPCs. There's a different scenario I run which involves a mind-controlled child attacking the group, and most of the groups I've run this for have gone to great effort to avoid killing the child.
What this experience suggests to me is that your group will probably also try pretty hard to avoid killing any children; if you make sure to include a well-clued way to avoid doing this, I think it's very likely that the group will take it.
The other thing this suggests to me is that, if you worry that the children thing is too dark, it seems to me that you could replace them with adult peasants. Then my experience of the werewolf game would directly apply.