So, You Want to Keep One Hand Empty

Spellcasting is very close to the only good reason a creature keeps a hand empty, and even then taking a free action to keep one hand on a weapon the creature usually uses two-handed still permits the creature to cast spells, as does toting (but not employing) a buckler. Besides spellcasting, though, only a handful of reasons exist ever to have a hand empty of a weapon or shield.

• You want to steal stuff. "[T]o snatch an item worn by the target" can be a disarm, but, instead, "[i]f you try to take something from another creature, you must make a DC 20 Sleight of Hand check to obtain it" (PH 82) as a standard action. Some feats improve the Sleight of Hand skill option--the best are City of Stormreach's Master Pickpocket (95) and Dragon Compendium Volume 1's Cutpurse (95). The interesting thing about the Sleight of Hand skill check is that it can be performed as a free action by taking a -20 penalty, which is a lot, but until the DM limits your free actions, once you've a Sleight of Hand skill bonus of +20 or higher (and can avoid the attacks of opportunity that both feats eliminate) you can just steal all of every adjacent opponent's stuff... by taking, potentially, hundreds of free actions and dropping things after swiping them. This becomes even funnier if you've reach. This tactic is even tacitly encouraged by the feat Master Pickpocket, which reduces the free action Sleight of Hand check penalty from -20 to -10.

  This is actually a good reason to have an empty hand as a non-spellcaster. It is interesting, works against any gear-laden foes, stays relevant largely until the game ends, and is an appreciable combat debuff. Your DM may start limiting you, though, especially if you're doing this at low levels when the spellcasters aren't yet controlling reality, so you might want to sandbag if you can do this at, like, level 5, and--very seriously--folks will trap their gear after rumors circulate that you make enemies naked then dead instead of the reverse.

• You've something important in that hand that's not a weapon. Wands are a good choice, but your main weapon should already have a wand chamber (Du 30, 34) (100 gp; 0 lbs.). And so should your gauntlets (PH 117-8) and your spiked armor/armor spikes (PH 116, 123-4) and, if using Complete Scoundrel, your boot blades, elbow blades, knee blades, and sleeve blades (109-10). In other words, you've enough wands all the time. Here are some other choices.

  Note that even a light shield "lets you carry other items in that hand" (PH 125), therefore many options below are also permitted while employing a light shield or a buckler (PH 124).

  • The hands item gloves of endless javelins (MIC 194) (7,000 gp; 0 lbs.) makes a +1 javelin made of force appear in your hand as a free action. I know you're not looking for weapons, but if you don't have the feat Quick Draw (PH 98) and want to mix melee attacks and ranged attack during a full attack, the gloves are cool, and it's a real reason not to carry a light shield.
  • The hands item casting glove (MIC 84) (20,000 gp; 0 lbs.) functions like an overpriced glove of storing (DMG 257) (10,000 gp; 0 lbs.) or vastly overpriced glove of the master strategist (Gh 71) (3,600 gp; 0 lbs.) until you store in it a magic item requiring activation. You can activate that item as if it were in your hand. So, yeah, if you're serious about having a hand free and keeping it free, you can. Be Spider-man and store within it a rod of ropes (CS 113, 116) (4,000 gp; 4 lbs.)
  • The infinite scrollcase (MIC 162) (2,800 gp; 3 lbs.) organizes your scrolls, makes retrieving a scroll less dangerous, enables you to hold the retrieved scroll in the same hand as the case, and grants a bonus to Concentration skill checks to cast defensively the spell on the retrieved scroll. Great if you've enough Use Magic Device skill and the DM bans one-use wands.
  • The talisman of the disk (MIC 188) (500 gp; 0 lbs.) is stunning in its versatility. The standard action employment time is miserable, however, and cuts deeply into the item's usefulness, but the price is unbeatable.
  • The sphere of awakening (MIC 186) (1,800 gp; 1 lb.) is great for what it does, but normally the party's in serious peril by the time someone has a chance to use it. Hey, look at you and your free hand.
  • The talisman of undying fortitude (MIC 188) (8,000 gp; 0 lbs.) as a swift action grants the wielder a host of immunities as if the wielder were undead. My monsters recommend this.
  • The truelight lantern (MIC 190) (36,000 gp; 3 lbs.) essentially grants your side true seeing [div] (PH 296), which you'll need by the time you can afford this.

• You want to disarm a foe and keep his weapon. The disarm maneuver is difficult, but "[i]f you attempt the disarm while unarmed, you end up with the weapon in your hand" (PH 155), which, in addition to looking cool, in my experience also renders many warrior NPCs useless. I have never seen the disarm special attack used in a campaign to such a degree that any creature actually uses a locked gauntlet (PH 124-5), but a DM confronted with a master disarmsman will probably have the NPCs start using it. The feat Snatch Weapon (Song and Silence 40--I know!) lets you end up with the weapon in hand after a successful disarm attempt with a weapon and stab the dude from whom you took it.

• You want to use a feat requiring a free hand. The feats Deflect Arrows (PH 93) and Snatch Arrows (PH 100), the feats Einhander (PH2 94) or Single Blade Style (Dragon Compendium Volume 1 108), and the feat-intensive Return Shot (XPH 51) all require a free hand. All but the feat Return Shot are feats on the fighter bonus feat list, accessible via the 2nd-level Sor/Wiz spell heroics [trans] (SpC 113), which has a long duration and fits cheaply in a wand.

But, yeah, you really should weaponize that empty hand. If your feats are spoken for, gloves of the balanced hand (MIC 105) (8,000 gp; 0 lbs.) allow the wearer to fight as if he had the feat Two-weapon Fighting (PH 102). If that hand still must be empty, get a fanged ring (DM 101) (10,000 gp; 0 lbs.) or bracers of striking (Mag 110-1) (1,310 gp; 1 lb.); the bracers can explicitly give your unarmed strike weapon special abilities, so go ahead and add to your hands the weapon special ability defending (DMG 224) (+1 bonus) or whatever.

For Vision, Daylight and Bright Are The Same

For vision and hiding purposes, there are only three levels of light, as you mentioned. Bright, Shadowy, and Darkness.

"Bright" in this case means it's bright enough that there is no hindrance to vision at all. "Shadowy" grants concealment, and Darkness creates effective blindness if you can't mitigate it.

So yes, for these rules, a torch you're holding and the sun are the same thing, within the "bright light" range of the torch (20'). Outside of that range, the torch stops providing bright light. Get some distance on someone with a torch and you move into shadowy illumination against them instead. (If you have different vision modes than they do, you could actually have different conditions to see them then they have to see you, depending on the light.)

example:

If you're holding the torch, you're in bright light and can't Hide without Cover.

If someone else is holding the torch and you're 5' away from them, you're in bright light and can't Hide without Cover.

If someone else is holding the torch and you're 25' away from them, you're in shadowy illumination. That grants you Concealment, and you can Hide.

The same rules apply for any light source, you can use the Vision and Light table for the effective ranges of different light sources.

So what's Sunlight do?

The sun creates both bright light and bright sunlight. The difference is only in the case of monsters or effects that mention something related to that, like an Orc:

Orcs are dazzled in bright sunlight or within the radius of a daylight spell.

A torch doesn't generate bright sunlight, so in this case it's different.

This Creates Lots of DM Interpretation

As you noticed, this gets confusing pretty fast when Shadowdancers get involved. Just what is "some sort of shadow", and where is it? Ask your DM. It's relative to the light, the position of the light source, the size of the thing casting the shadow, and the rules don't have anything to say on the matter whatsoever.

How big a shadow do you need to use it? Doesn't say. How much shadow is enough shadow to shadow jump? Doesn't say. How long is the shadow being cast by the enemy in front of me? Doesn't say. (This is probably why Pathfinder changed the wording on some of these abilities to be near "dim light" instead, which is more of a known rules thing than "some kind of shadow.")

As a DM dealing with a player who uses these abilities, it can get pretty confusing to try and sort out. It's actually easier in a dungeon with no light of its own, because if some player is carrying a light source, we can map it out on the board pretty easily relative to them and see where the shadowed areas would be (and which direction the player shadows are going, if you want to hide in the shadow of the Wild Shaped Druid).

But in an outside area at 5pm? Where are the shadows in that? It's a lot of work to sort out exactly how it all works, and it's often easier to come up with a simple rule of thumb and apply that rule of thumb consistently.

Hide

The one exception you mentioned is "how dim is dim enough?" In order to use Hide, you need concealment. Shadowy Illumination provides that. So you need to be in Shadowy Illumination, which is going to depend on what the area's light sources are (but most of those have a radius of providing light, and the table in the first link I provided has those distances).

You also need to be not being observed. If they're watching you, even with Shadowy Illumination, you can't use Hide. "Watching you" basically means they can see you at all, because in D&D vision is omnidirectional: characters are looking in every direction on every turn.

That's part of what makes Hide in Plain Sight so good (with HiPS, you can use Hide while being observed).