The TWF fighter does less damage than a two-handed fighter on a single attack action (about 35% less), but virtually ties with a full attack action. Letting the TWF fighter always get his full attack damage significantly unbalances the equation in the other direction.
The thing is, whichever configuration you take, you can work on infinitely twinking it out. Two-weapon fighting is better when you want many more attacks. Being a rogue with sneak attack is the number one example of this. Or if you bleed per attack (like with Bloody Assault). Or use poison. Or use the various critical effects from the critical feats. Or want to use some of those attacks for trip/disarm/sunder attacks. There's a bunch of ways to increase your TWF damage by stacking "per hit" things on top.
Two-handed is better when something has DR or you're only getting one attack action in. Per-attack damage plusses and minuses and additional conditions affect a two-weapon fighter much more than a two-handed fighter - a "bard bonus" is multiplied in value by number of attacks. And as you note, you have to get a full attack to get those other attacks... But that's the same thing with a L11 two-handed fighter, they lose out on a lot of attacks when they can't stick and hit.
A straight fighter will do more dps in an abstract sense with a two-handed weapon than two-weapon fighting, if all he does is straightforward hitting without any other interesting feats. But if you change the TWF rules, your player is just going to turn around and layer the other TWF twinks on top and suddenly it's the more powerful thing, and you have to start worrying about your two-hander player... Because he who inflicts the most damage wins the game, apparently.
Also, there's a difference between optimization and real play. Real adventurers find gear, they don't "have 32000 to spend on it." I've played and GMed a lot of Pathfinder, and players have run and enjoyed two-weapon characters as much as two-hander characters. (Sword and board, however, sucks.)
Since the peanut gallery demands "math," here's some reasonably maxed out L11 characters, using all the Paizoey goodness. Each has more special abilites than one can reasonably enumerate, but I tried to stick to "jack up the numbers" options as much as possible. Full builds are below, but the relevant stuff here is that:
- 2HGuy, attack action: +23 (4d6+29/19-20 plus 1d6 fire plus 1d6 cold) 19-20 x2 - power attack, overhand chop is giving double STR to damage, vital strike, furious focus
- 2HGuy full attack: +23/+15/+10 (2d6+25/19-20 plus 1d6 fire plus 1d6 cold, 2d6+29 for attacks 2 and 3 from backswing) 19-20 x2
- 2WFGuy, attack action: +15 (1d6+15/17-20 plus 1d6 fire)/+15 (1d6+12/17-20 plus 1d6 cold) 17-20 x2 - two attacks from Doublestrike, piranha strike is like Power Attack for Dex, if he hits with both then 1d10+1 rend, if he crits +2d6 bleed, agile weapon enhancement gives +dex to damage
- 2WFGuy, full attack: +17/+12/+7 (1d6+17/17-20 plus 1d6 fire)/+17/+12/+7 (1d6+14/17-20 plus 1d6 cold) 17-20 x2, if he hits with both then 1d10+(1.5x Strength modifier) rend, if he crits +2d6 bleed, agile weapon enhancement gives +dex to damage, +2/+2 during full attack from twin blades, and also +4 AC bonus from two-weapon defense and defensive flurry)
Versus let's say AC 25, the average DPS (including crits and rend) for each is:
- 2HGuy, attack action: 55.67 (plus maybe shaken)
- 2HGuy, full attack: 87.06 (plus almost guaranteed shaken)
- 2WFGuy, attack action: 35.55 (plus two points of bleed)
- 2WFGuy, full attack: 81.80 (plus 5 points of bleed ongoing)
So yes obviously, in high mobility situations the 2WF guy loses out (though he does get two attacks, not one, from his archetype), but he's pretty much neck and neck in a full attack situation, any slight change to any of these numbers makes the winner of the full attack follies go back and forth. Both these characters have loads of other fun stuff to do, they have more feats than they know what to do with. The TWF guys is obviously much tighter on feats and stuff. Keep in mind this is a straight fighter compare, and there's other ways to fill out TWF and all those rogues with sneak attack can stack a lot of damage on... Go look at some build guides and there's plenty of L11 twf builds out there that do a lot more than this.
Does the TWF guy lose? Yes. But the question is, "is the answer to let him have his 81-point full attack with a standard action?" The answer to that is no, it unbalances it back the other way.
Full builds
Because everyone has stat boost items I usually don't add them, but in this case I decided to make up the wealth differential caused by the weapons with a +4 belt of strength.
2HGuy
Human (Kellid) Fighter (Two-Handed Fighter) 11 (Pathfinder RPG Advanced Player's Guide 0)
N Medium humanoid (human)
Init +1; Senses Perception -1
Defense
AC 11, touch 11, flat-footed 10 (+1 Dex)
hp 114 (11d10+44)
Fort +10, Ref +4, Will +2
Offense
Speed 30 ft.
Melee +2 cruel flaming frost greatsword +20/+15/+10 (2d6+25/19-20 plus 1d6 fire plus 1d6 cold)
Special Attacks backswing, overhand chop, piledriver, shattering strike, weapon trainings (heavy blades +2, pole arms +1)
Statistics
Str 24, Dex 13, Con 16, Int 9, Wis 9, Cha 9
Base Atk +11; CMB +15 (+19 bull rush, +18 sunder); CMD 29 (31 vs. bull rush, 32 vs. sunder)
Feats Bull Rush StrikeAPG, Cornugon Smash, Dazing AssaultAPG, Dreadful CarnageAPG, Furious FocusAPG, Greater Bull Rush, Horn of the Criosphinx, Improved Bull Rush, Power Attack, Pushing AssaultAPG, Vital Strike, Weapon Focus (greatsword), Weapon Specialization (greatsword)
Skills Acrobatics +12, Intimidate +13
Languages Common, Hallit
Other Gear +2 cruel flaming frost greatsword, belt of giant strength +4, 150 gp
Special Abilities
Backswing (Ex) Attacks after the first in a full attack receive 2x STR bonus.
Bull Rush Strike Critical hit's confirmation roll is a Bull Rush maneuver check.
Cornugon Smash When you damage an opponent with a Power Attack, you may make an immediate Intimidate check as a free action to attempt to demoralize your opponent.
Dazing Assault (DC 21) -5 to all attacks and maneuvers but struck foes are dazed 1 rd (Fort neg).
Dreadful Carnage If you reduce an enemy to 0 or fewer HP, you can make an intimidate check to demoralize all enemies within 30' as a free action.
Furious Focus If you are wielding a weapon in two hands, ignore the penalty for your first attack of each turn.
Greater Bull Rush When bull rushing, foe's movement provokes AoO from your allies.
Horn of the Criosphinx Add 2x your Str bonus to damage rolls on charges while wielding a two handed weapon.
Improved Bull Rush You don't provoke attacks of opportunity when bull rushing.
Overhand Chop (Ex) Single attacks with two-handed weapons receive double STR bonus.
Piledriver (Ex) Standard action: attack with a two handed weapon, if successful, free bull rush or trip w/o AoO.
Power Attack -3/+6 You can subtract from your attack roll to add to your damage.
Pushing Assault If you are wielding a weapon in two hands, push the target 5' back instead of dealing power attack damage.
Shattering Strike +3 (Ex) +3 Sunder and damage vs. objects.
Vital Strike Standard action: x2 weapon damage dice.
Weapon Training (Blades, Heavy) +2 (Ex) +2 Attack, Damage, CMB, CMD with Heavy Blades
Weapon Training (Pole Arms) +1 (Ex) +1 Attack, Damage, CMB, CMD with Pole Arms
2WFGuy
Human (Kellid) Fighter (Two-Weapon Fighter) 11 (Pathfinder RPG Advanced Player's Guide 0)
N Medium humanoid (human)
Init +5; Senses Perception -1
Defense
AC 16, touch 15, flat-footed 11 (+1 shield, +5 Dex)
hp 114 (11d10+44)
Fort +10, Ref +8, Will +2 (+3 vs. fear)
Defensive Abilities bravery +3, defensive flurry
Offense
Speed 30 ft.
Melee +2 agile flaming cold iron shortsword +15/+10/+5 (1d6+15/17-20 plus 1d6 fire) and
+2 agile frost mithral shortsword +15/+10/+5 (1d6+12/17-20 plus 1d6 cold)
Special Attacks doublestrike, improved balance, twin blades
Statistics
Str 13, Dex 20, Con 16, Int 9, Wis 9, Cha 9
Base Atk +11; CMB +9; CMD 27
Feats Bleeding Critical, Critical Focus, Double Slice, Greater Two-Weapon Fighting, Improved Critical (shortsword), Improved Two-Weapon Fighting, Piranha Strike, Two-Weapon Defense, Two-Weapon Fighting, Two-Weapon Rend, Weapon Finesse, Weapon Focus (shortsword), Weapon Specialization (shortsword)
Skills Acrobatics +16, Intimidate +13
Languages Common, Hallit
Other Gear +2 agile flaming cold iron shortsword, +2 agile frost mithral shortsword, 150 gp
Special Abilities
Bleeding Critical Critical Hits deal 2d6 bleed damage.
Bravery +3 (Ex) +3 to Will save vs. Fear
Critical Focus +4 to confirm critical hits.
Defensive Flurry +3 (Ex) +3 AC vs. melee when making a full-attack with both weapons.
Doublestrike (Ex) Standard action: Attack once with each weapon (normal two-weapon penalties).
Improved Balance -1 (Ex) Reduce the penalties for two-weapon fighting or count off-handed one-handed weapon as light.
Piranha Strike -3/+6 You can subtract from your attack roll to add to your damage with light weapons.
Twin Blades +2 (Ex) +2 to hit and damage when making a full attack with both weapons.
Two-Weapon Defense +1 to AC while wielding 2 weapons. +2 when doing so defensively.
Two-Weapon Rend Deal extra 1d10+(1.5x strength modifier) if you hit a foe with both main and off hand weapons.
Light Weapons that the rogue has proficiency in
These are really the only significant concerns. You need a light weapon for Weapon Finesse and/or Two-Weapon Fighting, and you need too many feats to burn one on proficiency in something fancy. You could go with a rapier in one hand and a light weapon in the other, if you wanted, but if you have any feats or class features that work for only one weapon, it’s better not to (most of those feats suck, though, so it’s totally valid to not have them).
The differences in base damage, critical ranges, and critical multipliers barely matter at all. The difference in damage between a 1d4 and a 1d6 weapon is, on average, 1. Even at first level, that will probably never matter. Criticals happen much too rarely to put any significant investment in them, so the critical range or critical multiplier mean quite possibly less.
Unless you are taking Shadow Blade (which is an excellent idea and highly recommended; see below), just pick whatever light rogue weapon strikes your fancy. It probably will never make any difference.
About feycraft weapons
Rogues tend to be feat-starved, and Weapon Finesse is often an extremely desirable feat for rogues. If you can afford it (or can afford to wait), a feycraft light weapon is treated as if the wielder had Weapon Finesse even if he does not. Feycraft is an item template from Dungeon Master’s Guide II, and costs 1,500 gp on top of whatever the weapon costs. If you’re concerned about things that require that you actually have Weapon Finesse, like a feat or prestige class, note that this requirement can also be covered by Shadow Blade, which as noted, is highly recommended.
About Shadow Blade
The only other concern is validity for Shadow Blade, an excellent feat from Tome of Battle. Shadow Blade gives you a damage bonus equal to your Dexterity, but only if you use a dagger, sai, short sword, siangham, unarmed strike, or spiked chain. Since you don’t get proficiency in sai, siangham, or spiked chain, and unarmed strike requires Improved Unarmed Strike, you should ignore those, leaving the dagger or short sword. Both are light and rogues get proficiency in both, making distinctions between them fairly trivial.
Daggers can be concealed more easily, though. That’s fairly nice, particularly for a rogue. Compared to a paltry ~1 damage, I’d take it.
Note that to take and benefit from Shadow Blade, you must have and be in a Shadow Hand stance. Both island of blades (1st-level stance, flank as long as you and an ally are adjacent to the same opponent) and assassin’s stance (+2d6 Sneak Attack damage) are excellent for rogues, so this is no great burden.
To get either as a pure-rogue requires a Martial Study feat for a Shadow Hand maneuver (cloak of deception, 2nd level boost: swift-action greater invisibility for your turn only, and shadow jaunt, 2nd level maneuver: teleport 50 ft. as a standard action, are available at character level 6th, and are excellent choices), and then a Martial Stance feat (you need to be character level 10 or higher to qualify for assassin’s stance, but island of blades is available immediately). That’s three feats for Shadow Blade; that’s a heavy investment.
A better choice is to dip swordsage at character level 9th. This gets you 6 maneuvers (which can include both cloak of deception and shadow jaunt, as well as the excellent wolf fang strike and sudden leap from Tiger Claw) as well as your choice of stance, which can include assassin’s stance. Once you get these things, you can take Shadow Blade straight-away as your 9th-level feat. This is an excellent option.
Best Answer
The Barbarian is known to be a more powerful class than the Swashbuckler. The Barbarian is a so-called "Tier 4" class, the Swashbuckler a "Tier 5" class (see the Tier System for Classes). Parts of the definitions of the involved tiers are "Capable of doing one thing quite well" (Tier 4) and "Capable of doing only one thing, and not necessarily all that well". In fact, the two classes are a particularly good example of the difference between the involved Tiers, since both are primarily melee classes that have a similar "thing" to do in combat - dealing damage. So you really have a textbook example of a power disparity within the 3.5 class system. And this isn't all that easy to fix. Some things you can do:
Tailor some of your encounters to the weak player
Difficult to pull off gracefully, but it can be done. The Swashbuckler has a few things the Barbarian doesn't, including social skills (Diplomacy, Bluff and Sense Motive) and the Tumble skill. The Barbarian having more trouble at a king's ball shouldn't be too difficult to envision. Creating combat encounters where Tumbling plays a huge role is usually an exercise in cleverly positioning enemies (long reach helps) and making said enemies good at taking advantage of Attacks of Opportunity.
Problem is, of course, that it doesn't do much for the problem of the Swashbuckler not being competitive in the damage department. The players would effectively be taking turns at being relevant - not something that ages well in a campaign. But once in a while to keep spirits up should be fine.
A more lasting solution would be to...
Steer the Swashbuckler towards more powerful options
There are a few options that make Swashbuckler more palatable.
Daring Outlaw: Multiclassing and taking the Daring Outlaw feat (from Complete Scoundrel) is a good way to bring a Swashbuckler's damage output up - up and a bit beyond that of a Rogue, at least. This would be my recommended approach if only there weren't a Rogue in the party already. As-is, it might just move the competition from the Barbarian to the Rogue, but perhaps that allows more branching out.
Arcane Stunt: This alternate class feature from Complete Mage grants Swashbucklers a few uses per day of a spell-like ability, at the cost of their Grace. It's a tiny improvement, but usually an improvement.
Factotum: Multiclassing into Factotum (from Dungeonscape) provides immediate extra combat use out of the character's Intelligence. Perhaps more importantly, it can provide considerable utility in the form of skills (and other class features), making the comparison with the Barbarian less direct.
Tome of Battle: Multiclassing or prestige classing into a Tome of Battle class (probably the Warblade for Int synergy) would make the Swashbuckler an enormous lot better, no question. And level 4 is an excellent time to do it (if not before). Problem is, Tome of Battle works for more or less all melee characters - including the Barbarian.
Some Retraining (Player's Handbook 2) may be necessary or beneficial to make the new options fit.
If it won't...
Abandon ship!
The whole situation might not be salvageable. See K.L.'s answer.