My vote is for the broad-strokes banhammer.
In short, don't make a big deal of it and people mostly won't notice.
Just because something exists in the world does not mean it should be an option for the players. Why not let one of them play as a Mage or Werewolf? Because it doesn't work in the world the GM has defined. Why not Malkavians? Because they're insane. Just because it isn't the reason doesn't mean it isn't a valid reason to ban them.
In addition to simply disallowing certain rules (like Level Adjustments in D&D), I've seen or heard of artificial GM limits such as:
- no evil
- no non-evil
- no lawful
- must have half levels in Bard
- must not play <these> classes/races
- must not take some ability or other
and they've all worked out well from what I've seen.
I favor very open, liberal character creation rules, in both the games I run and in the games I play. I chafe quickly at too many restrictions, and feel that some DMs who enforce too many are shooting themselves in the foot.
But even from that perspective, your rules are really very minimal. Even with my preferences, I still expect more rules than that—you really are being very open here. Allowing a player to play two PCs? That’s quite generous (but not so uncommon for small groups—I suggest offering the same to other players, though!). And wanting to know what the source of material they use is really pretty much a given for any DM. I’ll have to take your word for the balance problems of Yuan-ti purebloods and bugbears, but really nixing just two, obscure races for PCs is quite tame as far as a banlist goes.
Blocking the chaotic evil alignment bugs me slightly more, in that really any alignment can cause “My Guy” problems, and I feel like it is better to address that problem itself, but OK, new players, I can see it.
And that’s it, as far as requirements you’ve described. That’s really quite minimal: none of these requirements are too severe individually, nor is there overly many of them. Meets my preferences, personally, and with quite a lot of room to spare! What is OK to control and what isn’t is massively subjective (and depends on more than just the group, but also the setting, campaign, and so on), but I find it very hard to believe that anyone would feel you’ve gone overboard here.
The real issue, it seems to me, is that you have new players; they don’t know what is or isn’t expected. They seem to have real buy-in and enthusiasm, since they seem to be going online to find options (presumably where they found this homebrew, and quite likely also where at least some of the ideas of the two-character combo came from)—which is great. Except that they don’t know how to judge what they read online, and they don’t know how much of it is stuff they have to work with the DM on versus stuff they can reasonably just “expect” to be available.
This is why I tend to prefer to do character-creation with new players in person, together. Craft a party together, focusing more on story and character, with you and/or the books there to provide relevant rules for bringing that character to life. Even when you have to say “no,” it’s much better when you can do it immediately, rather than after the player has chosen something and gotten emotionally invested in the idea. In the future, that’s the approach I recommend.
In the meantime, I suggest that if your players resist any of these rules, you work with them to try to come to a method of representing their character that satisfies you both. If that means toning down the bugbear they had their heart set on, so be it. If that means finding analogous options to the homebrew they like within the official rules (including by “refluffing” the official options—teaching new players to not feel restricted by the characters described by the books is a great thing!), then OK. If that means allowing the CE character, replacing your ban with stern warnings about getting along with the party, well, like I said: it doesn’t have to be a problem, and any alignment can be a problem, so really, maybe it will work out (and if not, well, you did warn the player).
But don’t feel that you’re being overbearing! The only reason I suggest you work with these players at all is because they’re new, you don’t want to damage their excitement and enthusiasm, and the things they didn’t know have to be excused. If these players had been in even a single campaign before, I would instead be telling you that they should know better than to try to complain about these minimal rules, and recommending that you stay firm with them.
Best Answer
I have often reduced the options available to my players in my D&D campaigns of every edition I've been playing, from 3e to 4e, and I am currently playing in a 5e game where I'm limited to the PHB only.
So, what am I trying to achieve when I impose these limitations? Well, it's several things and maybe not all at once.