Personally, I’d say you’re about done with fighter and swashbuckler
I try to avoid taking more than 3 levels of swashbuckler (insightful strike is its best feature) or 2 levels of fighter (a feat per level is OK; a feat every other level is pretty weak). I’d say it’s probably a good time to leave those classes behind.
The exception is if you take, e.g. Daring Outlaw (see below) to improve these classes. The Zhentarim Soldier ACF can also do wonders for the fighter class (up to 9th level anyway) if you have the Charisma and Intimidate ranks for it.
Classes
The best answer is warblade (but you don’t have that allowed)
Just for the sake of anyone else reading this, the warblade (Tome of Battle) is perfect here: he’s flashy, highly-skilled, and makes good use of Intelligence (and he gets 4+Int skills). White Raven makes him an excellent battlefield leader.
You already have Int to damage: factotum adds Int to (almost) everything else
(I just noticed you don’t have factotum listed either; that’s a real shame. Again, leaving this here for future users who do)
Factotum (Dungeonscape) sounds really perfect for your character here. They do gain a few spells (and some Supernatural abilities), though there are homebrew variants that can remove them. More importantly, the factotum’s whole schtick is “I add Intelligence to everything.” Like all Strength and Dexterity skill and ability checks (e.g. Athletics, Acrobatics, Stealth, Initiative) at level 3. In fact, the only notable thing that factota don’t add Intelligence to is damage: that you already have from swashbuckler’s insightful strike.
And a factotum is always the man with the plan, even if he’s actually just making it all up as he goes along. Seems like a pretty good guy to follow.
See this question for more information on using the factotum class well.
You may already qualify for chameleon, and it’s pretty good
You may qualify for the chameleon prestige class (Heroes of Destiny or here) based on Able Learner (depends on whether or not you’ve been putting points in the skills you need). While the best feature of chameleon is their spellcasting options, you don’t need to use them. Two levels of chameleon gets you the wonderful “floating feat,” a feat you can change every day. A third level can get you some sneak attack damage, which is useful (see below re: Daring Outlaw).
If you qualify for that, you might also make a good master of masks
This prestige class (Complete Scoundrel or here) isn’t very good and is probably way too specific for you: you want to be a leader of men, not hiding behind constantly-changing masks. But if you take just one level (i.e. only get one mask), and choose the gladiator mask, you get to be Maximus. You become proficient in every weapon ever, which is pretty awesome, and we all know how Maximus leaves everyone entertained.
Dread commando is fitting, but not very strong
The dread commando prestige class (Heroes of Battle or here) is full-BAB and gets sudden strike, which is the weak version of sneak attack (and stacks with sneak attack for qualifying). Requires Dodge and Mobility, which is a shame since they’re pretty weak and you should have ranks in Tumble to avoid AoOs anyway. Mobility, at least, can be gotten as an armor special ability (and the rules do allow you to qualify using items, but make sure DM allows it), but it’s quite expensive for your level.
Note that there are several feats (Midnight Dodge from Magic of Incarnum, Expeditious Dodge from Races of the Wild, Desert Wind Dodge from Tome of Battle) that count as Dodge and are better. Still not good, but better. Worth considering.
Anyway, dread commandos are skilled fighters, and the bonus damage is good for a finesse build, and you get some team-based bonuses. So fitting. But it’s a weak form of bonus damage and the team bonuses are pretty small, so not very good.
A little bit of bard could work
Bards require ranks in Perform, but unless you use their spells they don’t actually need Charisma. Just one level for inspire courage would not be a bad idea for the sake of being all leader-y. You can just ignore the cantrips that you get.
See this question for thoughts on making the most of the bard class.
Knight is OK for a few levels
Knight (Player’s Handbook II or here) is heavily-armored and full-BAB, and makes some solid attempts to protect others (e.g. test of mettle). Unfortunately, the knight’s code is extremely strict, even stricter than the paladin’s in a lot of ways.
It’s often said that knight has five good levels: 1, 2, 3, 4, and 20. If you go for knight, test of mettle and bulwark of defense are probably your best bets.
If you can, refluff a barbarian
Instead of rage (or ferocity or whirling frenzy, which are probably more appropriate to you), call it something else: In the Zone, Focus, something. Refluff it as bullet time or something; the mechanics still work for that (particularly ferocity or whirling frenzy, which actually do make you move faster). One level in barbarian gets you these great features, and if you have Complete Champion available, you can also swap fast movement for pounce via the lion spirit totem alternate class feature, which is a great deal.
This doesn’t directly impact leadership skills, but it gives you a lot of martial prowess to back up your leadership.
Marshals are OK if you’ve got great Charisma
If you already have very-high Charisma (16-18 before items), marshal (Player’s Handbook II) is worth it for the minor auras that add that to various things. But since you can change your minor aura whenever you like, and the major auras are weirdly weak, I don't think you need more than one level.
Legendary leaders certainly fit the bill—if your DM is using the Commander rules
Legendary leader (Heroes of Battle) gives some solid abilities on an OK chassis. The problem is that it depends on the DM using the Commander rules found in Heroes of Battle, and in my experience, most don’t. Without them, a couple of abilities don’t really make sense.
If you do go this route, note that Iron Will can be bought for 3,000 gp (Otyugh Hole, Complete Scoundrel). Many DMs won’t allow it, but it’s definitely worth asking if you can. 3,000 gp is a bit much for a +2 bonus to a single save, but as something that counts for prerequisites, it’s a steal.
Feats
Daring Outlaw would improve your damage
The Daring Outlaw feat (Complete Scoundrel) stacks your swashbuckler and rogue levels for grace and sneak attack. You need 2d6 sneak attack damage to qualify, which means three levels in rogue, or a level in rogue and a level in something else that gives sneak attack (or an analogue like the ninja’s sudden strike or the scout’s skirmish; both from Complete Adventurer).
Note that a dread commando (above) could take Daring Outlaw because sudden strike stacks with and counts as sneak attack for prerequisites. This will give you +2d6 sneak attack dice from swashbuckler levels, which is nice, and allows you to consider going back to swashbuckler for more.
Daring Warrior isn’t too good
It stacks swashbuckler and fighter, two classes you already have, which makes it seem appropriate, but they only stack for the purposes of grace and qualifying for fighter-only feats. With the exception of Weapon Supremacy (Player’s Handbook II, requires Fighter 18), the fighter-only feats are pretty weak, and grace isn’t too great.
Knowledge Devotion is pretty solid
If you have Knowledge skills, it’s an attack and damage bonus. Just worth mentioning because it’s a good feat that might fit the character. Depends whether or not your wit is backed up by real knowledge, though, since Knowledge Devotion does require a fair amount of, ya know, devotion to knowledge.
Conclusion
A bunch of dips for useful abilities
I like Barbarian 1, Bard 1, Chameleon 2, and Master of Masks 1 best of the options presented that are on your list of legal material. I wouldn’t take more levels of any of these classes, however. That gets you to 10th-level, anyway. After that, I suppose dread commando and/or legendary leader are probably your best bets.
The simple option: Daring Outlaw and dread commando
The other option is to not bother with any of that hodgepodge and just go Swashbuckler 13/Fighter 2/Dread Commando 5 with Daring Outlaw. That gets you full BAB, 7d6 Sneak Attack, and 3d6 Sudden Strike. Of course, that assumes you can get Dodge and Mobility... Anyway, it will make you a reasonably effective melee combatant and give some minor bonuses to those around you. It’s not the best build available but it does have a certain elegance.
I really do have to mention warblade again
If you can get it allowed, I have to mention again how perfect warblade is here. With a start of swashbuckler 3/fighter 2, you can do really well by taking three other classes, and then taking warblade at 9th to start with an Initiator Level of 5, thereby starting with 3rd-level maneuvers and stances. My preferences for those three levels would be Factotum 3, Chameleon 2, Barbarian 1, Master of Masks 1, and/or Bard 1 (such that they add up to three, of course). For bard, Song of the White Raven becomes your obvious choice as 9th-level feat.
Paladin 2 / Wizard (abjurer) X
You said you are willing to take at most a 2 level dip in another class for proficiencies. Have you considered taking these levels in Paladin? Assuming you want to keep the "I'm a Wizard!" flavour, taking your first 2 levels as a Paladin of the god of magic (assuming there is one in your campaign, of course) isn't really out of character. So here's my take on this build. I'll mostly cover melee options, since this is what the question is about, but you should still be able to do anything that another wizard could do, just 2 levels later. This gap can be great at certain levels (Fireball is a great damage upgrade at level 5), but it shouldn't be noticed too much on single target damage (Smite!). It's not the strongest build out there, but it's definitely good enough to be enjoyable!
Benefits of 2 paladin levels :
- Proficiency with all weapons : opens up many solid weapon choices
- Proficiency with all armors and shields : although dex is definitely the best stat in 5E, a melee wizard will require spending spell slots on defence to keep up. Being able to wear a full plate and a shield gives great defensive benefits and costs no daily ressources.
- 6 more HPs : While 2 more level wizard levels would've given you 4 more arcane ward HPs, those only replenish on a long rest or by spending spell slots. Arcane ward is an awesome feature, but real HPs trump it anytime.
- Lay on Hands : Yay for 10 free HPs per day! More than makes up for those 4 lost ward HPs, and can be used on other people, too.
- Fighting style : There are many good options here, but I'd go for protection or defence to increase your utility/survival. Great weapon fighting is a good offensive option if you forgo your shield and grab a great sword, especially since you're going to stack lots of dice on those attacks. This can be somewhat replaced by the Elemental Adept feat, however.
- Access to 1st level paladin spells : One thing you'll have as a wizard is lots of low-level spell slots rather early. Smite spells might not always be the best options, but they can add good damage and other effects on your melee attacks. Paladin also has a few abjuration spells, which gives you back some arcane ward HPs, but the good ones are higher level, sadly. If you have the warcaster feat and can handle your constitution saves, Shield of Faith is also very good as a low-cost defensive boost. While Cure Wounds is a paladin spell, you can cast it using your full spell slots, which means that, although not a cleric, you still have access to some great healing powers if they are needed.
- Divine Smite : Convert spell slots into damage. Given that you can elect to smite after you know the attack hits, and that there is no save for this damage, it's a great way to increase your melee damage. Especially good if you crit...
That being said, given equal gear, you should have the same AC as any other tank, since you're wearing the same stuff they are. As for your damage, the new Sword Coast Adventure Guide cantrips are there for you. Sure, they're not going to out-damage a fighter or a rogue, but they're still very good damage options. Since they require a melee weapon attack, they also trigger class features such as Divine Smite, so you don't miss out too much. Having two attacks could technically mean 2 smites per round, but the added damage of your cantrips will compensate for this quite decently.
- Green-Flame Blade : when fighting multiple enemies, this attack basically gives you the benefit of a second attack without costing anything. Since you're not going to have multiple attacks from class features, this is pretty much as good as you're going to get.
- Booming Blade : Assuming you can make sure your opponent moves willingly each round, this cantrip will give you the same benefits as GFB except on a single target.
Other thing to note is that the secondary damage from these spells automatically hits, no save and no attack roll required, which means that you can strike a minion to land free damage on the BBEG. These spells will thus give you decent damage round after round, leaving you with all your other spell slots to use as you see fit. Your 2 levels of paladin will give you the opportunity to convert your low-level spell slots into more damage as an option, and your wizard powers will allow you to turn these same spell slots into defensive abilities. Shield will grant you +5 AC for one round, which is awesome if you attract the BBEG's attention (especially given that your AC is already high). In Elemental Evil, there is also the Absorb Element spell, which gives you resistance against a single element, also for one round. This gives you defensive options for both physical and most magical attacks. As an interesting addition, both of these spells are abjuration-based, and will thus heal your ward for 2 HPs every time you cast them.
And after all this, you're still an almost full wizard. Your access to spells and features will be delayed for two levels because of multi-classing, which shouldn't reduce your melee effectiveness, but you will still have access to 9th level spells and have spell slots as a 19th level caster. While the Signature Spell capstone is good, the real treat is at wizard level 18 (character level 20 for you), so you don't miss out on much. Indeed, spell mastery will allow you to have 2 low-level at-will spells, which gives you infinite Shield spells, for instance. This should allow you to use your slots for more offensive options, since your defence is free. Infinite Shield spells also means that your arcane ward is always topped off after every combat if you take a minute or so to recharge it after each fight.
Your melee power also scale quite decently if we compare them to a tank fighter (longsword + shield, defensive fighting style) of equivalent level :
- Level 1-4 : Fighter [1d8 + str] vs GFB [1d8 + str + (int secondary)]
- Level 5-10 : Fighter 2x[1d8 + str] vs GFB [1d8 + 1d8 + str + (1d8 + int secondary)]
- Level 11-16 : Fighter 3x[1d8 + str] vs GFB [1d8 + 2d8 + str + (2d8 + int secondary)]
- Level 17-19 : Fighter 3x[1d8 + str] vs GFB [1d8 + 3d8 + str + (3d8 + int secondary)]
- Level 20 : Fighter 4x[1d8 + str] vs GFB [1d8 + 3d8 + str + (3d8 + int secondary)]
Of course, a fighter has abilities to increase his damage, but so do you (thank you, Divine Smite)! Since the fighter has more attacks, he's more likely to deal some damage each round, whereas your damage all hinges on your single attack landing. This will be especially important if the fighter can get his hands on a weapon that deals extra damage (like a flame tongue), an ability that would be mostly wasted on your single attack per round. The fighter also only requires only one good stat to be 100% effective, while you will need both strength and intelligence. Since your intention is to go melee, I'd consider maxing strength first, especially considered you'll use your spells mostly defensively or in a utilitarian fashion (so lower save DCs are not as bad).
Feats to consider :
- Warcaster : For pretty obvious reasons (casting and adv. on constitution saves). Combined with Booming blade, it's also a boost to damage: when an enemy tries to get away, you booming blade it, which will deal almost double damage if it hits, as the enemy will likely leave the area right after. If they don't, well, your party gets to kill them anyways.
- Shield Master : If you are using a shield, it's a solid defensive feat.
- Heavy Armor Master : +1 strength, reduce most physical oncoming damage by 3. The drawback is that it doesn't work on your arcane ward, as it has it's own HP pool. As such, any damage the ward takes is not "damage that you take" and does not trigger this feat. Still a great option for a tank, especially one that has a wizard's HP pool and requires strength for his attacks.
- Elemental Adept : You're going to cast a lot of fire cantrips, and this feat allows you to ignore resistance and to treat 1s as 2s. This feat does not specify that only the fire damage dice are affected, so it works on your weapon die, your GFB dice, but also on your added Divine Smite dice. Definitely not bad.
- Mage Slayer : Disrupt enemy spellcasters from melee range. The attack granted by this feat is not as strong as a full opportunity attack (as warcaster will allow for a GFB or BB cantrip), but if you can prevent them from casting, it has a lot more utility. Advantage on saves isn't exactly bad either.
Other things to consider :
- Transmuter instead of abjurer : possible proficiency with constitution saves or resistance to one element is a good trade-off for a melee wizard. Not sure that I'd sacrifice arcane ward for it, though. Since Arcane Ward has it's own HP pool, all the damage it soaks isn't counted towards the concentration save DC. Advantage from warcaster should be more than enough, especially if you have some stat bonuses.
- Bladesinger instead of abjurer : A tough one... Since you're going to use cantrips for offense, you're wasting the extra attack feature with that build. But... a dex-based character wearing light armor will have only 1 less AC than a plate wearing character (equal if you cast mage armor instead), so your bladesong will leave you with a net +2 to AC during a fight once you get 20 intelligence (since you're also forgoing your shield). Even if not going for a strength build, you lose none of the advantages of a paladin (smite, armor proficiencies, healing, spells, etc.), but also gain Intelligence bonus to concentration saves and increased combat speed, as well as overall better skills since dexterity is the best stat. I'd personally favour bladesinger, but both options are very interesting.
Best Answer
Avoid "all or nothing" spells, especially against boss type enemies.
By "all or nothing", I'm referring to spells, like Disintegrate, that have zero effect on a creature that saves against them. These spells are especially risky against bosses that tend to have high saving throws and/or Magic Resistance, and may have Legendary Resistances to save themselves even if they roll poorly.
Instead, consider spells like Fireball, Wall of Ice, or Sunbeam that deal half damage even if the target saves. Or, there are a few rare spells like Otto's Irresistible Dance or Power Word Pain that don't offer an initial save and just work initially but give the target a chance to save once they've experienced the effect for at least a turn.
Spells that use attack rolls, like Scorching Ray, Steel Wind Strike, or Crown of Stars can also be more effective, depending on the situation. Many enemies that have great saving throws don't necessarily have a great armor class, and attack roll based spells don't have to deal with Magic Resistance (which grants advantage on saving throws vs spells) or Legendary Resistances.
Alternatively,
Don't blast your enemies, buff your friends (or yourself)!
Unless all your enemies are equipped with Counterspell, you never need to worry about a buff failing. Spells like Enlarge/Reduce (choosing Enlarge), Haste, or Greater Invisibility cast on yourself or the party's Fighter, Paladin, or Barbarian can help them tank or dish out more damage. Fire Shield, Blink, and Mirror Image are some nice spells to help with your own survivability, and none of them even require concentration.
And,
Control the Battlefield.
Create difficult terrain to keep enemies from moving as they please with spells like Earth Tremor, Erupting Earth, or Ice Storm. Cut off enemy lines of sight with spells like Fog Cloud or Darkness. Block enemies' paths or lines of sight to your allies with the various "Wall of ..." spells. Use Telekinesis or Bigby's Hand to hold an enemy in place (these spells both use contested ability checks instead of saving throws, so you can bypass Magic Resistance and Legendary Resistances) or move aspects of the terrain.