The 4-tier difficulty level is tied to progression throughout the game. To unlock "Nightmare" (the 2nd difficulty), you must first complete act IV on "Normal" difficulty. Nightmare is balanced based on the level you would be by beating through Normal, so it's not feasible to just skip ahead, even if you could.
The other factor is Hardcore, which is unlocked at level 10. Hardcore is, well, harder, and is separate from the game difficulty presented elsewhere. You want more of a challenge? Play Hardcore and try not to die!
I've been playing Inferno in a group and solo. This is my build: http://us.battle.net/d3/en/calculator/wizard#aZcXlQ!WTX!aZbZaZ
Act 1 Hell isn't really a good gauge if your solo build is doing really well... My build has changed several times because the later acts and difficulties required more CC and escape mechanisms.
You need a lot of control if you want to solo the hard champion packs, but they are all entirely doable with a good wizard build. The 60% slow on blizzard is a godsend and all but 'fast'-modified champion packs will be hard pressed to close the distance on you if you kite properly.
The main problem is fast, or mortar monsters. With fast you just need to do more running than usual.. Diamondskin-Frostray to single target the close ones when skin is off cooldown (Diamond skin with the reduced cost rune makes these ray abilities take almost no arcane power), teleport around if you're getting chased too close.
With mortar the best way is to KEEP RUNNING as far away from them as possible. Drop blizzards and arcane orbs behind you (If you pop diamond skin you can get off 5 or 6 arcane orbs instead of like 3 or 4, due to the cost reduction...it's amazing). If you have a circular area you can kite, use that; if it's just a single hallway when you reach the end you can wormhole-teleport to the other side and resume kiting.
Never stand still against mortars, those things hurt. I use magic missile & frost ray to pick units off. Again...diamond skin + frost ray is amazing.
I cannot emphasize enough how important something that controls the monsters is. I like blizzard but you might be able to get away with the 30% passive slow for arcane spells (but that's half as much as blizzard provides) or other forms of slow/cc.
Good luck!
Best Answer
There are a few differences that are known and probably more as well:
Higher difficulties mean higher tier loot. Jay Wilson stated that over 70% of all of the armor items in the game cannot even be found in Normal difficulty. So as the difficulty raises so do the armor tiers. One can assume 4 per act totaling 16 tiers. Remember, tiers don't necessarily mean better, just different looking.
As the difficulty increases, the boss monsters become more powerful. More specifically, in each difficulty, the monsters gain one more special ability. By Inferno, they can have up to 4 special attributes which can make them very difficult to take down.
More obviously, the standard enemies will scale with the difficulty. This video says that monsters scale much like they do in diablo 2. They all have set stats which they carry with them as they level. The monsters also have more awareness and abilities as they scale through the difficulties.
Also, you progress through the difficulties, you can upgrade your artisans more through items that drop only in higher difficulties. However, as noted by @Kexlox you can also buy the tomes from the AH and upgrade your Artisans early.
Keep in mind while playing that Normal is made to basically be the tutorial and break you into the game. Experienced D2 players will cut through Normal with ease. As you progress, the game does get very difficult, and you'll be wearing your butt on your head once you reach Inferno. Inferno was basically built to murder you. Jay Wilson stated that they took the hardest difficulty that their testers could handle and agreed was fair and they doubled it.
FROM BRYSONIC'S ANSWER -
The difficulty also scales by having additional players affect monster HP. Monster damage used to scale, but no longer does as of patch 1.0.3.
Note: Because of patch 1.04 monster health per additional player now is not effected by difficulty.
Here is a table showing the amount of scaling that occurs based on the number of players in the game:
This table is now technically useless, but I will leave it here as a placeholder.