You killed it, man!
The solid-state switching devices in dimmers tend to fry when the dimmer's output is shorted. Replace the dimmer, and this time, don't peel the insulating sticker off to avoid another short. The breaker may have tripped eventually, or it may not have; the good news is that what you did is OK for now as the smell is simply lingering as odors are wont to do.
Typically there are 3 connections involved for lights, receptacles and small appliances (some larger appliances like ovens, air conditioners, heaters, clothes dryers are different).
Normally, hot is switched. Neutral and ground are not switched. Normally, that results in a safe environment where this is nothing "live" when hot is switched off.
This safety is based on neutral being the same as (or very close to) ground and the exposed metal of appliances, light fixtures, etc. being connected to ground. There are a number of things that need to be done correctly for this to be the case, including (but not necessarily limited to):
- Neutral and ground connected together (bonded) in only one place (typically the main breaker panel).
- Ground (and by virtue of the neutral/ground bonding, neutral as well) being connected to the actual ground via ground rods and/or connection to rebar and/or connection to metal cold water pipes.
- Fixtures wired correctly so that the wires that should be hot are hot and the wires that should be neutral are neutral.
- Ground wires used correctly so that there is really only one connection between neutral and ground.
- Ground and neutral wires used correctly so that neutral is never piggy-backed onto ground or vice versa.
Anything that can go wrong, will. That's why when working on even something as simple as replacement of a fixture that is controlled by a switch, we don't rely on the switch to provide full protection but instead turn off the breaker or pull the fuse to make sure that the circuit is off and will stay off.
All that being said, with a properly wired house, changing a light bulb should be a very safe activity.
It sounds like you have a number of possible problems, which could include swapped wires (hot vs. neutral), neutral/ground not connected properly (too many places or none at all) or other problems.
In a case this extreme, I suggest getting a professional electrician to check things out.
One quick test you may want to do is to check various fixtures & receptacles with that multimeter. The values should be similar throughout the house - e.g., in the US that would be in the 110 - 120V range for most things. If you get wildly varying numbers then you likely have a very serious problem such as a "lost neutral" that would require immediate professional assistance.
Best Answer
You have probably burnt out the dimmer due to the inrush of current during the short and I am guessing the on/off switch on the dimmer is mechanical and didn't burn out. The common cure for this is to replace the dimmer. When you do that you need to connect it up with the power off. Then turn the power on an see if everything is working.
Good luck