Spiders are spawned in the centre of a block, and because they are larger than a single block, part of them overlaps the other eight blocks surrounding it.
This means that when a spider is placed to be spawned they must not have any solid blocks around them in a 3x3 area in.
An area like this will prevent spiders from spawning.
ooooooooooo
ooooooooooo
ooxooxooxoo
ooooooooooo
ooooooooooo
ooxooxooxoo
ooooooooooo
ooooooooooo
ooxooxooxoo
ooooooooooo
ooooooooooo
o = air
x = solid block (half-slab)
Update for 1.8 and after
Update 1.8 added a 1/16 meter high hitbox to carpets, which made natural mob spawning on carpets impossible. Carpets are now safe.
Thanks @TomášZato for bringing this up!
(There still are some types of "unnatural" spawning that are able to force mobs into carpets, like Village Sieges.)
Before 1.8
Carpet both does not prevent and does not facilitate mob spawning, but it does help in a way.
Carpet is a non-solid block, so, in terms of algorithm, mobs do not spawn on carpets, but rather in. Basically, you can replace all carpets with signs for the same, but more obvious, situation.
Mob spawning alrotihm, simplified, is the following:
- Pick a totally random place (in 3D, not only on the surface) to spawn a pack. It must be a block of air (not carpet).
- Try to spawn several mobs around the pack on the same height. Block underneath must be solid, and the block mob spawn in may be any non-solid block, like a carpet.
What it means regarding carpets: if you have a level floor in a room, and room is surrounded by solid stone, no mobs will spawn even in the dark, as there are no air blocks on floor level. If you have a building surrounded by air, though, mobs will be able to start spawning outside and "leak" onto your carpet floors.
So, glass floors are as safe as they ever were.
Detailed algorithm is on the wiki.
Best Answer
Mob spawning happens in two stages.
The culprit in your case is the second stage - individual mobs can spawn inside non-solid blocks like Carpets, Pressure Plates, open Fence Gates and even Doors - tall mobs are thin enough not to collide with a door when they are placed in the center of the block.
Had your installation been underground, with no Air block at the level of doors, it would be secure. But it being in the air, packs start spawning outside, and "leak" inside through door blocks. Oh, plus there's air above glass floor.
P.S.: My test world shown on the screenshot is a snow-covered superflat. Though mobs definitely can spawn in snow cover blocks, there is never (if I'm careful) an Air block on ground level, so the world is safe.