[RPG] Players get frustrated when they can’t solve a hard diplomatic problem. How to get them to think out of the box

Are you familiar with the Same Page Tool? It sounds like you had expectations that you tried to convey subtly to the group in-game, but this wasn't overwhelming enough to overturn their existing expectations or the conflicting messages being sent by your campaign kickoff's dominant tropes, so you weren't on the same page.

Getting on the same page is the first step toward your players having their PCs act like real people – you can establish your expectation, and the possibility, that they can ignore the common game-like meta structures that are often taken for granted around a roleplaying game. Things like they are all a party and cooperate and they're aiming for survival of their PCs rather than character embodiment are common assumptions that are often necessary for certain kinds of campaigns, because it is easy and expedient to say that some play possibilities are "off limits" in order for the group to focus their game time on the fruitful voids of the campaign.

What appears to be the problem is that you have tacitly broadened the fruitful void, without notifying the players of this strongly enough to make them set aside their (productive, functional) RPG-playing conventions. On top of that you set up the campaign with a very standard "form party, loot dungeon" structure, which strongly conveys a standard "dungeoncrawl campaign" set of expectations that are the opposite of what you seem to have hoped for.

How to get them play their PCs like real people is then a two-step process: first, clearly grant them the breadth of allowed play possibilities necessary to be able to play them like real people. Second, cultivate a group value of character embodiment.

The second is the hardest part actually, and the how can't be covered here because the barriers to doing that are personal and depend on your players. Given that they haven't even yet become aware of the possibility with the first step, I have no data to even begin giving advice on the second.

So, that makes the first step very important: get on the same page, eliminate the assumptions about how to play a dungeoncrawl-type fantasy RPG, and replace those assumptions with explicit understanding that you're aiming for humanist drama in a fantasy context. Once you've had this conversation, only then can you even find out whether your players are interested in embodiment-focused play and what their individual barriers for that are, if any. Be aware that they may not be interested in this kind of play; be prepared to have a conversation that is about negotiating a common ground, and it may not lead to the sort of play you're looking for. It's possibly you'll all get on the same page, but if you can't, that may mean the group can't continue – but that's better than forging ahead with conflicting expectation and play goals.

It sounds like your players are engaging with the NPCs, they're just not asking the questions you want so they can get the information you've prepared that will help them. Doesn't sound like disinterest so much as they interact differently than you would, or than you'd expect. It's unlikely that you'll be able to change the way they act, at least not directly, so you're probably going to have to change your style a bit to correct the mismatch.

In general terms, you may have the classic problem that often arises in mystery RPGs; the players miss the clues, don't ask the obvious questions, and otherwise fail to extract the appropriate information from the scene to move the plot forward. It's a common disconnect, usually due to the fact that the GM has a lot more information, so connections that seem obvious are actually much less so to the players. To counteract this difference, you'll need to put a bit more window dressing around the points that they need to discover that are critical to the plot.

I recommend reading Justin Alexander's essays on the Three Clue Rule. It gives some great advice on how to improve the players chances to ferret out what they need from the setting without just handing them the info. Ultimately, if there's a piece of information that was important enough that you've actively called it in your setting notes, you're going to need a couple of back-up opportunities for the players to discover it, because they're going to miss the first two glaringly obvious opportunities to do so.

A last tidbit that I find works for some groups is profiling,, i.e. put stuff where they look. Pay attention to the details the players engage with and try to get a feel for their patterns. Maybe they tend to engage with shopkeepers but not people on the street. Maybe the wizard is always looking for cool wands. If you want them to ask the locals for more info on the ants, put a cool wand proudly displayed in the local shop. The shopkeeper will tell them it's not for sale, but he'll ramble on a bit about the wizard that created it, and how he lived here to study the physiology and tactics of the ants. As one of three clues that they can get more info to help in their fight, it starts to give you a chance to get the message across.