[RPG] How to optimize this Dwarven Inquisitor build for effectiveness of overrun attack and intimidate strategies

Demoralize

There's various feats that add to demoralize, and the DC is usually fairly low. However, investing in it properly is a matter of considerable feats and even potentially class levels. A good standby in this situation (or any optimizing situation) is to look at what you'd be doing otherwise, compare the two, and decide which you like better.

I'd personally go with Demoralizing because as a melee character having as many mechanical things to do in combat as you can is always good - attack attack attack gets boring over 20 levels.

That said, after briefly looking at the options, Demoralize appears to be a genuinely useful thing to be doing, with the right feats and class options.

Using demoralize on the same creature only extends the duration; it does not create a stronger fear condition.

This is where I start to get interested. Not because being Shaken for longer is great, but because of this ability I found when I searched for 'is demoralize any good pathfinder'.

Frightening (Ex)

Whenever a thug successfully uses Intimidate to demoralize a creature, the duration of the shaken condition is increased by 1 round. In addition, if the target is shaken for 4 or more rounds, the thug can instead decide to make the target frightened for 1 round.

That's for the Thug rogue character archetype, and it receives it at first level. Let's take a look at what Frightened does for us in terms of mechanical effect.

Frightened: A frightened creature flees from the source of its fear as best it can. If unable to flee, it may fight. A frightened creature takes a –2 penalty on all attack rolls, saving throws, skill checks, and ability checks. A frightened creature can use special abilities, including spells, to flee; indeed, the creature must use such means if they are the only way to escape.

That is amazing. Takes someone completely out of the fight, and the best part is that penalties usually stack - so a Frightened, Shaken foe takes -4 to literally everything.

Better than that. The general rule is that if someone would make a Frightened character Shaken or Frightened, it instead becomes Panicked.

Becoming Even More Fearful: Fear effects are cumulative. A shaken character who is made shaken again becomes frightened, and a shaken character who is made frightened becomes panicked instead. A frightened character who is made shaken or frightened becomes panicked instead.

Panicked is another -2, and even if they can't flee, they don't fight, they cower.

Intimidate doesn't allow demoralize to stack with itself to create further levels of fear, but sources of Shaken, Frightened, and Panicked from other sources do stack with Demoralize (this is explained in the FAQ).

So if you can find a non-demoralize source of Shakening Baconing, you can stack that all the way up to 'they cower in a corner while you all wail on them'.

But that's broken!: Eh, not really. Note the 'Try Again' section in Intimidate. Every subsequent Intimidate check takes a -5, regardless of success. So you can't just spam this to victory - you need to succeed quickly on a few checks to frighten them for a round or two, they need to be vulnerable to fear, and anything too big or strong is likely to just resist your intimidate anyhow - it's still a good tactic, but for feat and level expenditure you're putting in it's not overpowered.

Now, this is already pretty alright, despite requiring a level in Rogue and skill points and intimidate boosting feats (maybe?) to pull off. But ideally you want to be applying this condition on round one, not after several rounds of increasingly ineffective intimidate checks. It would also be nice to be able to target more than one person.

Enter Enforcer.

Benefit: Whenever you deal nonlethal damage with a melee weapon, you can make an Intimidate check to demoralize your target as a free action. If you are successful, the target is shaken for a number of rounds equal to the damage dealt. If your attack was a critical hit, your target is frightened for 1 round with a successful Intimidate check, as well as being shaken for a number of rounds equal to the damage dealt.

With the Blade of Mercy trait (fluff it as hitting people with the flat of your blade, slapping them, kicking them etc - standard swashbuckler stuff), you can turn every attack into an intimidate attempt (yay). Combined with your high crit range (rapier), and TWF (off hand main gauche), and you could potentially fear multiple weak opponents or a single strong one in a single round.

Don't forget that using Frightening from Thug just makes the new intimidate check make them frightened - it doesn't 'upgrade' or 'remove' the Shaken condition, so...

Shatter Defenses, as it makes Shaken foes flatfooted to your attacks for this round and the next if you hit them when they're already Shaken. a) this is great for getting off sneak attack from that Thug level you took, b) this lowers their AC and also their CMD (trip them! kick them down stairs! swashbuckle! swashbucklers win the battle in the hearts and the minds before the first foe is struck!).

Shatter Defenses requires Dazzling Display, which is actually a good feat to have for this build regardless. It allows you to make an Intimidate check against everyone in 30' as a Full-Round Action, and you do it by showing off your sword skills! Talk about Swashbucklery. Normally that's a Waste of a Turn since Shaken isn't that great and if you're entirely surrounded by enough foes to make it worthwhile you should be tumbling to safety. But against weaker foes, with Frightening from Thug, you can show off a dazzling display of swordsmanship, Errol Flynn style, and watch an entire room of mooks lose their crap and flee in terror! Talk about awesome.

To optimize your Demoralize, I suggest you startize with Taunt. That way, instead of growling death threats, you can send them packing with witticisms about their mother! Or puns!

If you have backwards compatibility, the skill trick Never Outnumbered might come in handy, but you should steer away from Imperious Command, is it will result in flying DMGs.

Also, if you go this route, specialize your gear vs common fear-immune enemies like undead, and Paladins. Since that way you won't be a one-trick pony (Although this trick is cheap and leaves you room to develop your melee damage and whatnot).

It's worth noting that intelligent undead and the like actually seem to be affected by Demoralize - pathfinder uncoupled 'morale' from 'fear', so you can Shaken a Paladin or a Vampire, you just can't use Cause Fear on him.

Let's work out some of the math here; the details of your build and your DM are going to influence how these numbers work out. Some assumptions I am making based on what you have stated.

Assumptions

1. You said DM is letting you use flanking rules so I am assuming you are getting advantage on each attack so I am going to state that you are getting advantage on your rolls 75% of the time due to other skills and tactics you could be doing.
2. I am assuming that your attack stat at this point is a 20 in any of the builds. I am ignoring barbarians right now who can get more then a 20 in a stat also I am ignoring belts or other items that can raise your stat above a 20.
3. Finally we are going to ignore the Sun Blade because getting a hold of a specific magic item should not be a requirement for a build.
4. This also assumes that your DM lets you roll your X number of attacks and decide to apply the sneak attack to the critical attack. The once per term requirement is very vague on this and should be clarified.

The Math

My first thought is that the thief build is going to be really top heavy; you are trying to get as many dice rolls as possible because you want that natural 20 critical to double the number of dice you are rolling for damage. I did a little look at monsters vs stats and I am trying to figure out a good threshold for when a die roll will hit and I feel like 13 on the die should hit based on average AC of monsters at that level. So 13 for die roll, 2 for weapon bonus, 6 for proficiency and 5 for stat. Now one could argue these numbers one way or another but I am going to be using them as the base because I need something to compare apples to apples.

So 13 + 2 + 6 + 5 = 26 for target AC to hit or 35% chance to hit and a 5% chance to crit. I used http://andrewgelman.com/2014/07/12/dnd-5e-advantage-disadvantage-probability/ as a reference for determining advantage so that translates to a 63.9% chance to hit and a 9.8% to critical. That makes it so 54.1% of the time you are going to do normal damage and 9.8% time you are going to do double damage. I basically am saying 2/3 of the time you are going to hit with your attack, really anything that gives you 2 attacks is going to let you get a sneak attack in during that round so long as you are within 5 feet of your target. This is also a pretty high number, some could argue that 10 or above would be a more realistic number then you are talking almost an 80% chance to hit with advantage.

Since you decided to dip into fighter instead of go 20 rogue you don't get Stroke of Luck but that would only be a factor depending on how much resting vs combat you would do. To ease math on calculating average damage I am assuming your weapon damage for rogue is a D6. If you are getting a D8 or better this just slightly makes your numbers better it just helps me to do my math faster.

Critical Hit - 18d6 + 2d6 + 5 + 2 = 77 Avg

Normal Hit - 9d6 + 1d6 + 5 + 2 = 42 Avg

GWF Critical Hit - 18d6 + 2d6 + 5 + 2 = 85 Avg

GWF Normal Hit - 9d6 + 1d6 + 5 + 2 = 46 Avg

The math was kind of surprising because of the limited reroll only on 1 or 2 and you could get a 1 after you rerolled a 2 but on average on 10d6 rolling a 1 or 2 and getting 4 extra damage on average.

Polearm Master

After looking at the polearm master the numbers are basically the same for attack and damage except that you get another attack that you will get your critical damage against. Even though the base dice is a d4 that extra attack is what gets you the key advantage — do you want that extra damage from the reroll or the chance to attack again and be able to sneak attack? Frankly I'd rather take the 63.9% to do my sneak attack dice again then the extra 4 damage on average.

Crossbow Expert

This basically follows the same rules as Polearm master: the feat is letting you get an extra attack with the a crossbow and I still think it is better then Polearm master because being able to attack at range is crucial; however, that depends on your flavor and what you like to do. However where this starts to break down is flanking and how your GM rules on flanking with a crossbow. If your GM rules you can't with a range weapon then you are stuck with polearm.

Bottom Line

Bottom line your GM house rule about flanking is getting you the most bang for your buck here because you want to maximize that rule. The winner between all 3 is really hard to decide, and also depends on how your GM house rules critical hits and determining which one was a sneak attack. If you get to pick the extra attack is going to be always worth it. If you can't pick which attack to apply sneak attack damage to then they are really almost a wash because your extra critical chance vs the on average higher damage from your rolling is going to end up being almost a wash in the long term at high level.