Class specific weapons can provide some class only enchantments that you cannot get on standard weapons. Whether or not this is worth a trade off for damage, that is entirely up to you to decide.
For instance, you can find a demon hunter only 1 handed crossbow that provides hatred regeneration or a wand for mages that provides improved arcane pool.
Like James mentioned in his answer, the demon hunter is the only class that has a unique situation where you MUST have a ranged weapon equipped or you can't use his ranged attack skills. And his one handed crossbow is the only ranged weapon you can dual wield.
I decided to test this by removing my gear and all increased attack speed. I then test the weapon I chose for my main hand, over 400 base damage with 1.60 attack speed. During Rapid Fire I was averaging 800 never below 450. I then tested weapon #2 which i later put in offhand barley over 200 damage and used rapid fire. I was averaging around 200, never over 375. I then duel wielded each weapon. #1 in main and # 2 in offhand. I recorded myself shooting creatures with short bursts of rapid fire. The outcome was this.
Barrage #1:
253,
133,
58,
301
Barrage #2:
898,
882,
884,
715
Barrage #3:
636,
100,
469 (crit),
156 (crit)
Barrage #4: 532, 649, 544 (crit), 490
Barrage #5: 80, 185, 84, 122
Barrage #6: 220 (crit), 1200 (crit), 823, 743
In barrage one you can see that all 4 shots were fired from the offhand due to none of them deals over 400 damage. In barrage #2 you can see all 4 shots are from the main hand, due to all the weapons damage being over 500, but in barrage #3 you can see both the main hand and the off hand dealing damage. the main hand being the 636 and the off hand being the rest of the damage in the barrage. In barrage #4 I believe that the main hand did all of the damage due to the fact the the off hand never crit for over 375. This also makes Barrage #4 mimic #1 but for the opposite hand. In #5 it is all off hand damage that mimics #2 except for the opposite hand and in #6 it is mixed again with the offhand dealing the 220 crit and the main hand dealing the rest, but still mimics #3.
Answer:
As you can see each weapon has its own consistent damage. It also appears that each weapon has some sort of a cast time and a rhythm that barrage 1 and 4, 2 and 5, and 5 and 6 mimic each other, but is even. The only differences is which weapon was dominate in the set of attacks. If each alternate weapon did damage for each alternate hit the damage would look more like 800,200,800,200. I suspect that weapons with different attack speed would have a much different outcome, and the dominate weapon being the one with the most attack speed.
Best Answer
The behavior of some skills to use only main hand damage was decided by Blizzard since the beta patch 13 so players wouldn't have to time attacks for higher damage (by using skills right after weaker weapon had been used). If you want to read the notes, the Blizzard link doesn't work any more but I found a copy at Incgamers.
I'm not going to try and provide a list of skills; instead I will quote Bashiok (Blizzard CM):
Interestingly, someone created a thread on battle.net about this very recently - I agree with stormcontrol's post there that spammable skills without CD alternate damage, while others don't.
Thanks for asking this question, it was an eye-opener since I had a very different impression on the rarity of skills using only the main hand for damage.