You're trying to railroad the game when the players are telling you very loudly where they want the campaign to go instead. Take them there.
If the NPCs are boring you, that's a different problem. Be sure you're making NPCs that engage you and not just your players. You have to enjoy the game too.
To run an interesting socially-focused game, you might need to learn how to use a few tools.
Challenging situations made from social connections can be made using PC-NPC-PC triangles—but don't intentionally make life-or-death situations out of them unless you want a bloodbath; save life-and-death for external threats to the social group.
In a game with a big stable of NPCs in a complex set of social relationships, maps of the land become far less important than maps of the social situation, and you can use relationship maps (like this one) to keep track of how everyone is connected and feels about each other.
And if, in the end, you don't want at all to play a game that is heavily focused on personal relationships and social interactions, sit down and talk with your players about what kind of game you can all enjoy. Putting a collar around their neck and dragging them toward the plot won't be fun for them or (as you've been learning) for you either.
I do several things to keep the player characters interested and invested in each other.
- At the start of the game, I insist that players coordinate backgrounds (subject to my approval) such that each character know at least one, and preferably two or more, of the other characters. In general, I prefer these connections be positive; the most negative I will usually tolerate is on the order of a friendly rivalry. In general, I also prefer that if you start from any one character, you can get to any other character by following these pre-arranged links. (In graph theory language, the players are all connected, although indirect connections are fine; the alternative would be two more more sub-groups connected internally but not to each other.) And finally, I try to ensure that each connection is more than trivial, but not necessarily life-binding.
So for instance, "We met in a bar twenty years ago and never saw each other again," is trivial. "We are cousins who are best friends and we are rarely separated," is life-binding and more than I look for (although it's fine if that's what they want.) Things like the following are what I look for, and/or what I've seen in the past:
- Our characters served in the same unit years ago, and knew each other, but haven't kept in contact...
- I served as a mercenary escort once, while he was travelling with his master from here to there; along the way, this happened....
- We weathered the siege/plague/earthquake of wherever together some time back....
Now, some players are genuinely not wired that way-- if you ask them for backstory, they freeze; if you given the one, they can't connect to it. When I run into a player like that, I have to respect that, but I try very hard to get everyone to adhere to the guidelines.
That does not directly solve the problem. (It actually solves the problem of getting the characters all on the same page at the start of the game.) But it does often give me enough to work with to do the following:
- With enough insight into character backgrounds, and with overlapping backgrounds, I try to give every character an mid-term to long-term goal or plot arc, and then I try to modulate that by giving at least one other character a minor to moderate interest in how the first character's arc plays out.
It's important (to me, for the games I want to run) that these arcs not be strictly opposing: If one character has sworn blood-vengeance on an NPC, I won't give another character the goal of keeping that NPC alive. But I might give another character the goal of getting something from that NPC before his death, or getting the NPC to do something, etc.
And I also try to modulate this in another direction by giving other characters-- ideally, not the same one-- influence over the plot lines. So continuing that thought:
- Player A has sworn to kill Sir Odious, his parents' killer
- Sir Odious has information that will help Player B in her quest to do something else
- Player C knows someone who can be bribed into giving up information about where Sir Odious will be
In that way, for each of the various player sub-quests going on, at least one or two others will be involved somehow, even if only at the periphery. Ideally, Player B has some motivation for something to happen, and Player C has something he needs-- something at least moderately costly or risky. They are invested.
One thing I would not do-- at least not again-- is what you tried:
I've had players create characters (with backgrounds) completely secretly from each other with the hopes of allowing the character interaction to be heavily role-played at the table. Didn't work because of very incompatible characters.
I've never done that, specifically, but I've inadvertently done similar things and it never worked well. It seems like it should work, especially if you pattern it similar to what I've outlined above, but there's a structural weakness to it: If the players, starting out with the relative blindness of only knowing their little part of the background, they just might not see those connections you built in for them, and won't give themselves the incentive to start sharing information. And if your players were the sort that would do that naturally, you wouldn't have to go through these acrobatics in the first place.
Best Answer
Start every character with at least one relationship to another character:
The blacksmith is opposing the baron and that's why there is a white rose in his window. The apothecary joined the cult of Orcus and will sell poison. The guard is in love with the girl selling apples at the southern market. The beggar hates One-Eyed Tim for destroying his marriage to Lilly Hill twenty years ago.
I'm trying to phrase these such that they'll offer something to explore like the flower in the window, something to talk about like One-Eyed Tim, a little mission like carrying a message by one of the two lovers while passing through, or a trigger like the attempt of buying poison (or in my campaign, seeking a way to raise the dead and treat the injured).