The fluff is there to help you as the DM build a coherent fictional world.
First, without any explanation even to the GM, "there's a bearded devil with an intelligent glaive in a cell" seems to not make a lot of sense. "But why didn't they disarm him?" "These guys are devils, why is a devil in a cell?" etc. PCs tend to investigate things and want to know "why," so many times when they do I go to the fluff and let them uncover some of it, it builds the sense in them that the world is a logical one and it's not, as in 1e sometimes, "a huge dragon in a room with only 5' corridors going out of it." Frankly, even if they don't ask, by comforting you the GM that there is a reason (instead of just having a list of rooms with a monster listed in each) it makes you more confident in the game's fiction as well.
Second, they try to do this when there is a chance the PCs will interact with a NPC. In this case, they are in a cell, and can be spoken with without opening the cell door (in fact I am not sure why someone would open the door...). In cases like that, it's nice to have something to hang your hat on. (Not to harp on this specific encounter because you're asking in general, but it just says he's "mad" and "full of rage," it does NOT say he attacks the PCs on sight, which means he could potentially be enlisted as an ally against the villain - a crazy, evil ally, but hey). And don't forget they then get his intelligent glaive, which was a witness to those events. "Hey glaive what was the deal with that guy? "Uhhh...." Or maybe they capture someone else, a guard or whatnot, or they free someone else, and ask them "what was the deal with that devil down there..." There are many, many mouths information like this can issue from.
Third, even a detailed AP is meant to only be the starting off point for your own game. You are expected to expand on it, change it up, mash it up with other stuff, alter it to incorporate your own plots and characters, etc. By adding a little more info to the characters (in the case of the Howling Fiend, the fluff is exactly two sentences long) it provides a plethora of additional expansion hooks that a GM can use during adventure planning. When another devil shows up later, maybe he's not just a random encounter, maybe he's looking for his buddy Szasmir who went MIA a long time ago and now the PCs have a glaive that reeks of him.
Anyway, the actual killing of monsters and NPCs forms a reasonably small part of the overall action in a game of Pathfinder, the investigation and roleplay and all that form a huge part as well, and the fluff is there to fuel it.
I've been on the receiving end of a bunch of bad negotiations in RPGs. Real life negotiation training helps, but there's also some RPG specific aspects to keep in mind.
Often, the problem is that there's some adventure hook that requires the PCs to do something that's totally stupid. "Hi, you're level 10, would you like to go on a fetch quest for 100 gp?" Or the classic module Against the Giants, where the locals decide your high level party is guilty of unspecified crimes (vagrancy?) and must go kill 100 giants to atone. Even LG parties respond to that with "You're going to look mighty funny with that longsword sticking out of your ass." Your non-derogatory use of the term "railroad" (generally considered bad) leads me to believe this may be the case - if you expect the PCs to "take the adventure hook" you present them regardless of its desirability and they are trying to live in character and not go for some awful deal their characters at their current wealth and power level would never contemplate, you'll be disappointed.
Now assuming there's at least the makings of a deal, meaning each side has something the other person reasonably wants and might indeed exchange (goods, services, money, etc.) you need to remember that people want to make deals and that they can be win-win.
Allow me to use the framework presented in Getting to Yes, a book on negotiation everyone should read.
Separate the people from the problem. Put yourself in the PCs' shoes. Listen more than you talk. Get them involved. Help them save face at giving in on things. They should ideally be doing the same to the NPC.
Focus on interests behind positions. Ask "Why" and "Why not?" Be hard on the problem, soft on the people. How can everyone's goals be achieved? Don't attack the other person's position, look behind it. When they attack your ideas, ask for advice. Ask questions and pause. "How can we make this deal happen?"
Invent options for mutual gain. Generate a range of options, Use imaginative procedures. What other things does someone have that they'd be willing to do or give away that the other person might value more? It doesn't have to be a flea market "Five dollars! One! Four! Two! Three! Deal!". That's a one-dimensional negotiation. Timing, loot, favors, future goods are all possible. "What else can I throw in?"
Use independent standards. What is this really worth? If someone's being unreasonable, you can show what the item or job is "worth" to others. "These guys will do it for half the rate, but I want you to have the first crack at the loot..."
Develop a best alternative to a negotiated agreement. If one side has to have the deal or they're completely screwed, they are in a bad position. As a result most people have a plan B. As a DM, you need one too, besides "tell them out of character to take the bait." Maybe it's some kind of setup or extortion (e.g. the PCs' rooms get ransacked and a convenient clue is left behind pointing at the people the adventure hook is trying to sic them on. Of course the other negotiating party did it, but PCs are usually dumb and easily pointed in a direction.)
Of course, you can just let events transpire. In your example, the PCs got suspicious of the sage. So? Can they not get the information any other way? Are there no other people they can go to? Do they not plan to just stage a home invasion and interrogate him (normal PC response to something like this)? Your problem is less about negotiation and more about poor adventure design and railroading. Should there really only be one way to proceed in the plot? It's best if not; but if it is then the PCs will eventually have to give in or beat it out of him... I find you always get the best results from handling things in game, not with metagaming and railroading. It may seem expedient now but it's a long term poison for your game.
Best Answer
Since you say you're a first-time DM, I might exceed the scope of your question slightly in my answer. You're up against a philosophical question: Are the players co-tellers of the story, or "passengers" in it? This affects whether or not a particular quality is a Pro or a Con.
Players as Co-Tellers
In this approach, you treat the players as storytellers, and your job as DM is to arbitrate disputes between them (like an editor working with multiple writers) and to provide a general framework for this to occur. You keep the 'grand vision' of the world, while they cooperatively tell stories within it.
Players as "passengers"
In this approach, you treat the players as mostly reactionary inhabitants of the narrative, rather than near-equal participants in crafting it. This more closely resembles the 'traditional' DM/player relationship, with the players and characters both being surprised by in-game events. It allows more authentic reactions on the part of the players, but can sometimes create tensions since the DM decides how the characters' arcs will unfold rather than the players getting to develop their own visions.