From the standpoint of your weapon: Weapon Damage is king. Above all else, you want the highest possible amount of raw damage and DPS possible. Beyond that, you'll want to take a look at this formula to understand the value of other stats.
In short: at low levels, nothing is going to beat raw +Damage on any slot where you can get it (Rings, Neck, Offhand). As you increase in level, you'll eventually reach a point of inflection where the multiplicative scaling gained from adding +Dex is more valuable than the straight + Damage. This point of inflection will vary with other stats, and particularly with the base DPS of your weapon - your best bet is to simply keep an eye on the +Damage portion of the tooltip when comparing items.
As for stats like Crit and Attack Speed, the same basic rules apply. In general, .1 attack speed is worth 10 Dex, but this ignores the defensive benefits of Dexterity, and since attack speed bonuses are provided as percentage multipliers, you'll need to do some math. You'll need to run similar calculations with +Crit and +Crit Damage, but in general, unless you use skills or passives that rely on Crit, you can assume those stats are less valuable than the others.
Really, all of this comes back to one key point: Trust the Damage Stat on your Character Sheet. So long as you're doing the basics right (using a level appropriate, high DPS weapon), it isn't going to steer you far wrong. Beyond that, Dexterity is King, and everything else depends entirely on the magnitude of the stat that you get. An item with +Crit might be better than an item with +Attack Speed, based purely on how much of it is there.
When you're comparing armor and weapons, there's a number of things you should consider.
The first is the basic stat of the item - this is typically in large print. On armor, this is the "armor" value, and on weapons, this is damage per second (or "DPS" for short). Generally speaking, the higher the level/tier of weapon or armor, the larger this stat will be, although there's a bit of variance in each tier. It's a good general bit of information, but once you start adding magic, rare, and legendary items to the mix, you're going to have more things to consider.
The second thing you should note is the effect the item has on the three stats that are called out in the item's tooltip. These are life, damage, and protection.
- Life represents how much additional health the item will give you if you equip it. Clearly, the more life you have, the better off you are.
- Damage represents the damage change you'll experience when you equip this item. Higher damage means you're killing monsters faster, so this is frequently important.
- Protection represents how much overall armor you've got, and it decreases the damage you take, and it therefore makes surviving fights easier.
If you're just playing on normal, and aren't serious about maximizing your character, you can probably just look at these three numbers - they're highlighted in green if they're better, and red if they're worse. If some are green and others red, you'll have to decide if it's more important that a fight ends quickly, or that you're more likely to survive if things go south.
Watching these three numbers is likely enough to get you through the game on Normal. When you start finding gear with multiple enchantments, or you start shopping the auction house, you're going to want to be a bit more specific. Here's a few things to keep in mind:
Watch for items that increase your primary stat.
Your primary stat varies depending on your class, (Strength for Barbarians, Dexterity for Demon Hunters and Monks, and Intelligence for Wizards and Witch Doctors) but it's directly tied to how much damage you deal. Increases to this stat are generally taken into account in the comparative "damage" stat I talked about previously.
Note in the photo above that my damage is increased by equipping pants - this is because I am playing a monk in this screenshot, and the pants have +6 Dexterity. I'd have to decide if this damage boost is worth the loss of +6 vitality (= 60 life) or not.
The other stats will tend to give you secondary bonuses, but your primary stat is one of the most important things a piece of gear can give you.
Watch for useful enchantments.
What's useful is going to depend heavily on your character, but things like bonus experience or additional gold from enemies are almost universally good. Depending on how and what you're fighting, you might prefer a pair of boots with faster movement speed to get away from enemies, or perhaps something with high Vitality so that you can stay longer in a fight without needing healing.
Rarity isn't everything.
Although increasing rarity from magical (blue) to rare (yellow) and to legendary (orange) generally indicates better items, this is mainly because they are able to have more enchantments, or they have special/unique properties.
If the enchantments don't match your character or your build, they may not be better, even though they're more "special."
Best Answer
Life: Yes, yes it does. This is easy to test, just toss an Amethyst into a socketed helm with no Vitality on it. Since there are no other stats or affixes on items that affect HP beyond those two, there's no need to go into more detail on the formulae here I hope.
Damage: See here for all the details, but in short: Yes, it accounts for everything that matters. The only damage boosting stats it does not take into account are bonuses to the damage of an individual skill, or increases to resource capacity or regeneration.
Protection: It includes Armor, Dodge, and Resistance, but this is new. As of patch 1.03, "Resist values will now be taken into effect when calculating the Protection comparison stat (the Protection value is an average protection estimate of all your resists)".