You can go exploring some corrupted/crimson areas (randomly picked at world generation) to destroy the shadow orbs/crimson heart waiting for you at the bottom of chasms.
Each shadow orb/crimson heart destroyed will provide you some loot, and make a meteorite land on the world. Once the meteorite has fallen, go find the site, and mine the meteorite ore. This will allow you to craft some meteorite gear and weapons.
Once you've destroyed three shadow orbs/crimson heart, it will make the Eater of Worlds/Brain of Cthulhu spawn, allowing you to fight two different bosses.
Note: The shadow orbs are protected behind some ebonstone/crimstone, which you will not be able to mine with your current pickaxe. Just buy some dynamite from the Demolitionist and use it to access the shadow orb. Or you can buy purification powder from the Dryad, which turns the ebonstone/crimstone into the easily destroyable standard stone.
You can use ten of the same gemstone, plus 20 Meteorite Bars, to craft a phaseblade. All phaseblades are identical except for color, which is based on the color of the gemstone. You only need ten total gemstones, all of one type. You do not need ten of each.
As of 1.05, they have other uses! Six to ten each of two gemstones are required to craft Jungle Armor set pieces. You will need ten emeralds and sapphires, eight amethysts and topaz, and six rubies and diamonds. (for a total of 48 gemstones.) Additionally, rubies are used to create the Robe and Gold Crown, while topaz are used to create the Sandgun. (Source: http://wiki.terrariaonline.com/)
As of 1.1, each of the gemstones can be combined with 3 torches to produce 3 colored torches (shows off Terraria's new lighting engine!).
As of 1.2, each of the gemstones has an associated ore that it can be crafted with to create a "___ Rod" item, which is a rod made out of 15 of your gemstones and 10 bars of a particular ore. The Diamond and Ruby rods are First-and-second best, and use Platinum and Gold bars respectively.
All gemstones can now also be used to craft new grappling hooks that don't require a Hook to create, making them ideal for new characters. The Ruby and Diamond hooks also have a longer reach than the Ivy Whip.
In addition, these gems can be used to create Large Gems for the Capture-The-Flag game, so it would be wise to hold onto some in case you ever wish to host said game for some of your friends.
Best Answer
In my worlds, any pick can do it. Try a copper one to start with. Remember though, stronger pickaxes can mine faster.