I've played 3.5 for quite some time, and this has been something I've never quite understood, at least with respect to rules as written.
On pp 311 of the 3.5 Player's Handbook, under the entry for 'off-hand', it mentions you incur -4 penalty when using your off-hand when attacking and you only apply 1/2 your Strength bonus to damage. The Strength bonus to damage is covered elsewhere; under the Strength ability entry, several places under equipment, in the combat section and in many examples. However, this particular off-hand penalty is not mentioned elsewhere. Under Two-Weapon Fighting (pp. 160), the normal off-hand penalty is -10. This penalty also did not make it into the SRD. Oddly, in the Rules of the Game articles, "Two-Handed Fighting (Part One-Three)," this penalty is mentioned up front, and used in one attack of opportunity calculation. [Please note: this is only true in the pdf version, evidently the html version has been updated and does not include this] In the main FAQ, there are mentions of off-hand, primary and secondary, with respect to iterative attacks (not two weapon fighting), but again, no mention of this penalty specifically. There are several references to "all the penalties for attacking with two weapons (see Table 8-10 in the PH)".
How and when does this penalty apply? Does it stack with the penalties from table 8-10? Is it an oversight and should be ignored?
Best Answer
It gets factored in as part of the calculations only for the offhand if you don't have the Two Weapon Fighting feat, but honestly you should just ignore that wording as it's confusing. Two-Weapon Fighting spells it out much more clearly:
If you look carefully, the extra -4 is showing up in the first part, where your main hand gets -6 and your off-hand gets -10. It's still there with a light weapon (-4, -8). It disappears if you take the two weapon fighting feat, which reduces the main hand penalty by 2 and the off-hand by 6 (the difference is the -4 you noted being removed).
So while everything is being applied correctly, the wording of the part you noted is bad. Just use the Two-Weapon fighting entry itself, as it's wording is clear and the table summarizes very well what's going on.
The total penalties go like this, if you're curious:
Main-Hand: -2 for using two weapons, -2 for non light weapon, -2 for not having Two Weapon Fighting Feat = -6
Off-Hand: -2 for using two weapons, -2 for non light weapon, -2 for not having Two Weapon Fighting Feat, -4 for off-hand penalty (the rule you were curious about) = -10
edit - Using the example from the RotG article, you could argue (as the author does) that the off-hand penalty applies if you're wielding two weapons but only attacking with one of them (whichever weapon you picked as your off one). The off-hand would take the -4 for being offhand, but not the two weapon fighting penalties (as you're only attacking with one weapon). That's the cited example.
It never applies if you're only using one weapon, no matter what hand it's in, because "off-hand" is something that only appears under the two weapon fighting rules. The rules don't care which hand you use if you're only wielding one weapon.
By my reading, it also doesn't apply even if you're wielding two weapons but only using one of them to attack. RAW treats that as your main hand, no matter what. But if you wanted to use the Rules of the Game article's interpretation, then you'd have to pick an off hand as soon as you pick up a second weapon. Either way, when only wielding a single weapon it doesn't matter which hand you use.