I read a passage in the DMG the other night that without having the book at hand I think I can roughly paraphrase as
"Each NPC should be built with flaws, bonds, ideals and [whatever the
last one is] in the same way as a PC. An insight check gives the PC an
insight into one of these so they can use them to craft a convincing
roleplay against the NPC"
So you simply don't allow insight to be a lie detector test, you let them find the personality of the individual they are talking to and decide for themselves if the person is trustworthy and if they can use those features to convince themselves.
You also make all conversations grow organically, when they reach a suitable point to ask for an insight check you can allow them, but they can't have insight if they don't engage in meaningful conversation.
Example:
Flaw "The mayor is a compulsive liar"
Bond "The mayor will do anything to keep his pet Kangaroo safe".
The PC's chat to the mayor and ask him about something they know. You roleplay the mayor as a liar and say something like "The mayor looks away from you for a second before replying with [wrong answer here]". This is the clue for the PC to ask for an insight check which will reveal the flaw. They now know they are dealing with a liar and have to find a way to get the truth.
You know the only way to get that truth is to threaten the Kangaroo, so the PC's have to do something to either ask about the Kangaroo and insight check the relevance, or see the huge painting of the mayor and his kangaroo frolicking in the pastures and deduce it for themselves.
This relies on you giving more depth to the NPCs however and properly roleplaying them according to those characteristics. How hard this is I couldn't tell you.
TL/DR:
Insight isn't a lie detector, it is a clue to the personality of the NPC which will allow them to deduce for themselves. If the NPC has no personality it is hard to do and almost has to be a lie detector test which is where the bad habit comes from.
Ask the players when they want to intervene, and narrate until that point. Don't get bogged down in mechanics.
The normal flow of the game is for the DM to describe the scene/scenario, the players to declare their actions/intentions, then listen to the DM describe the result (with dice rolling as necessary). If the players choose to sit out of a conflict between NPCs, there isn't much else to do. The players are there to have fun, and if they want to sit out of a fight, they must not think that that fight would be fun, so hit the highlights and move on.
In my experience the best way to handle this is to make your best judgement on either what the outcome would be, or what it should be, either what makes sense or what makes the best story, then start describing how you get there from here. Roll one or two dice if you want help deciding some major turning points, but the story isn't about them, it's about the players, so keep it brief. Focus on the highlights and things of particular interest to the players.
If you're concerned that the players will want to intervene eventually, then describe the conflict in stages, giving the players easy/sensible times to do so. This can be done either explicitly, where you ask "Do any of you want to jump in yet?" or implicitly, where you pause and gauge the players by their expressions, body language and questions. If they still don't want to act, then move on.
In my opinion going through the full combat rules isn't necessary here. As DM you should be familiar enough with both the rules and the story to be able to make a sound judgement call on what the numbers should be if the players decide to surprise you during or after the fight. If the players do decide to jump in, reroll initiative from scratch, as the addition of a new fighting force is enough to throw everyone scrambling.
Remember that experience points should be granted for overcoming obstacles. If the players orchestrated events to cause one band of monsters to eliminate another, then I would reward that success. If this conflict was already in motion and the players sat idly by, then at least their time wasn't wasted while getting to the parts they were interested in.
I know that I've handled situations this way at least a few times, but the most recent was when I was running Sunless Citadel for a group of new players. They had formed a cautious alliance with the kobolds that are found earlier in the dungeon, and after softening up the goblins' defenses further in the kobolds drove the goblins out. My players, being both new and cautious, decided to stand back while a great melee broke out in the goblin warrens.
The players had moved on and taken out the hobgoblin chieftain, and then taken a look back at the rank and file. While I could have spent over an hour letting the mechanics play out, I instead decided that the side that the players' chose should win, and started describing the results. Within five minutes the players were back to the interesting bits: experience points, the hobgoblin's treasure, and this mysterious, vine-lined well (my players wouldn't let me call it a shaft) that descended deeper into the dungeon.
Best Answer
You have two options (and hey, they're the ones you mentioned in your question!), and it sort of depends on what exactly you're wanting to do.
First and foremost, most likely the advice I give here is only any good until we see the full NPC creation rules in the DMG where hopefully there will be much easier shorthands than what I'm going to suggest.
The by far and away easiest thing to do is to take the Mage NPC, spell swap for find familiar (something you're given full license to do in the rules, and even if you werent...well you're the DM) and just be done. Also add a racial trait to him to make him feel more like a dwarf.
Your other option is to build up a Wizard as if he was a PC. I'd advise against feats and just take the most basic options and equipment.
The first approach is way easier. You have a set CR, you don't have to rummage for the right spells or anything else really. The cons here are that you don't have any scaling rules for the NPC (he's set at CR 6), and he doesn't necessarily feel enough like a dwarf. These may or may not be problems for you.
The second approach has the pros of being fully scalable, and can be built as a dwarf so it will have a dwarf's racial traits etc. On the flip side, we don't have a PC->CR guide yet so you'll have to guess and it's a ton more work than just pulling up an NPC that's already created.
In a lot of ways what you do probably depends on context. If this is a one-off villain for an encounter, use the pre-gen'd NPC. But if this is a long running villain who is worth more of your time, build him as a PC and maybe even level him with the party.
Given that this character is going to be a quest giver, I'd be inclined to create him with PC rules as the customization offered in that set is going to be far better than what's available for monsters right now (in a month or so, I might change this answer, but I just don't know yet). The only caution is that if you make him too much higher level than the PCs then I wouldn't ever (or maybe only on super rare occasions) let him fight with them. (DM PCs aren't a great idea anyways, though, so there's that)