The best way that I've seen to handle this is to draw the map in chunks. Before the session, draw the first several rooms of the crawl, either as part of your pre-game prep or as people are coming in and getting set up.
If you have a break in your session (say, to order pizza), spend some time during that break erasing the map and drawing out a new set of chunks based on the players' location.
Most of the time the layout of the map isn't actually important (since the PCs can only really see the shape of the room). If it is important, just cover the map with paper, and remove sections that covered areas explored by the PCs.
You can also print a copy of the map without details on it, and hand that to a PC. Have them draw the map as you're doing the initial narrative.
This will work well for tactical games, like D&D 3 or 4, where the players move through the dungeon relatively slowly (room to room). For games where the players might fly through rooms quickly, use a small-scale map for the overall dungeon layout, and use the battle mat for the rooms where the fighting actually takes place.
If you have the option to go with some kind of model, toys, etc. that allow you to see it, that's best.
If you don't, here's some options:
1) Two layers, two colors
On your map, use two colors of markers - one is the upper level, one is the lower level. For your mini's, you'll want something like colored rubber bands, flags or to stack them on top of a base/token indicating which ones are up top/lower down.
2) Breakout map
This only works if you don't have too much overlapping area - you can put the upper areas parallel and further out on the same grid map to the lower areas. I've used this for a fight on a series of platforms/construction scaffolding, and it worked well enough.
3) Parallel maps
Get several separate map grids, each one representing a different level.
This works best if there's a lot of overlapping areas, where people might run/climb up and down, but not where they'll be fighting, shooting arrows, etc. between floors very often. You want to keep the action on a single map and not have a mini on one map shooting ranged attacks at someone down stairs, which means running over to the other map and seeing how they line up.
All of these require you, and your group have good spacial visualization and communications skills, otherwise you just end up with people MORE confused. Usually terrain made even of cardboard or legos, works best for visualization and for more people keeping track of their positioning.
Best Answer
1 inch squares, corresponding to 5 feet.
The DMG suggests that you make the squares on your battle map 1 inch on each side. These squares then correspond to 5 feet for game purposes.
DMG 250:
1 inch squares are consistent with standard D&D miniatures, which have roughly 1 inch circular bases.