First and foremost, consider the nature of the works you cite. When 1st edition Advanced D&D (aka 1e) was written, roleplaying was still new and error-correction and (extensive) rules extrapolations had not yet occurred. Worse yet, this rules set was written while Original D&D was being played by everyone, including the 1e author Gary Gygax. Several errors (see footnote) occurred during this overlap phase, and a general lack of specificity is common, most rules being left to the DM's logic and discretion.
Any question demanding a "by-the-book" (BtB) 1e ruling is thus subject to these errors and omissions, and the answer may be contradictory and/or illogical.
The first-level illusionist spell color spray operates by overloading the senses/neural networks of its subject.
Sorry Rich, but 'overloading the senses' is your deduction, and is not BtB. We can only say (BtB) that the 1st level spell (i.e. minimum power) causes unconsciousness, blindness, or stun, and that some victims get no saving throw.
Our view is that the same lack of conventional consciousness that makes the undead immune to sleep and charm also makes them immune to color spray.
While this is a logical sentiment it again is not supported by the Rules As Written. Nowhere do the rules refer to a 'lack of conventional consciousness'. Although it is a logical deduction, your BtB emphasis again precludes it.
It is true that most undead are immune to sleep and charm effects. But note that a Ghost is not explicitly given those immunities. This was probably a mere omission -- but again we have the BtB issue. (Deductions about immunities of undead as a class may also be faulty; many undead are also immune to hold and cold effects, but the Ghost is not, nor are ghouls or ghasts.)
THUS: given the descriptions in Monster Manual (1977) and lacking any explicit prohibition in the subsequent volumes, the Players Handbook (1978) and Dungeon Master's Guide (1979), the BtB ruling should be that Color Spray affects all undead mentioned in those works (Skeleton, Ghoul, Shadow, Wight, Ghast, Wraith, Mummy, Spectre, Vampire, Ghost, Lich).
In my own games I apply additional deductions and agree with your stance. But strictly BtB I must respectfully disagree with the other answers offered here.
Corollary: Don't insist on BtB answers for 1e. ;)
- One of the most famous contradictions of the OD&D/AD&D crossover period represented in the 1e rulebooks is one At-Will ability of the Demon Yeenoghu, a "magic missile... having a +2 to hit" (MM1 p20, 1977) although the magic missiles (per spell) "unerringly strike their target" (PH p67, 1978).
There's no need to house rule this — you just need to start enforcing its restrictions. Allow me to draw your attention to its area of effect:
4 square″ + 1″ square/level
(Aside, this is taken from PHB page 95, since the Illusionist version of the spell has different range and area of effect.)
These are table inches, which we could convert to feet/yards, but the following will be simpler if we just compare table inches directly with fireball's numbers.
A fireball has a radius of 2″. Let us ask then, “what level would an illusionist have to be to cast a phantasmal force large enough to contain a 2″ radius sphere?” Let's make a first-order approximation using the minimum number of 1″ squares necessary to create an area in which the image of the sphere could appear. The area of effect of fireball is three-dimensional, but the area of effect of phantasmal force is two-dimensional, which presents an odd issue. However, phantasmal force is described as being able to create images “within the boundaries”, so let's be as generous as the spell seems to be intended, and say that the vertical space it can create the illusion within is “good enough” for our purposes and we just need the footprint of the illusory fireball to fit inside the phantasmal force's area of effect.
We'll assume as a first-order approximation that each 1 square″ of area of effect can't be divided, so they have to be tiled as whole square table inches. Inscribing a 4-unit sphere's footprint (a circle) inside a pixelized shape requires a 4×4 square of squares, or 16 tiled squares.
So, as a first approximation, an illusionist needs to have 16 square table-inches of area of effect to play with to be able to make a phantasm of a full fireball. That requires being level 12 to pull off.
But maybe you think tiled squares aren't generous enough. Let's allow the table-inch squares of area of effect of the phantasmal force to be subdivided infinitesimally so that the area can be a cylinder that just barely contains the footprint of an illusory fireball and see where that gets us. (Some DMs might reject this move since the area of effect is given in squares and they may require a straight-edged area of effect, but some others might allow it with the reasoning that it's fine as long as the final area matches regardless of shape.)
A 2-unit-radius sphere has an area through its widest point of just over 12½ units, rounding up to 13 square″ required. So even being super-generous, it would still take an illusionist of level 9 to use phantasmal force to create an illusory fireball.
Clearly, creating illusions of fireballs with phantasmal force isn't something that your players should be doing at first level, or even 5th level when their magic-user friends are starting to throw real fireballs.
Conclusion: You don't need to house rule, just give the players a reality check
So, there you have it: you don't need to house rule this at all, you just need to pay proper attention to the relatively small area of effect, and break the news to your players that you've been accidentally letting them create a 9th-level illusionist's phantasmal forces before they were actually 9th level, and won't be doing that any longer.
If they are 9th- or 12th-level, then this isn't overpowered at all, because they have access to equally (or more) powerful effects already and cleverly using their resources like this should be letting them paste low-HD enemies by the truckload. So again, there simply isn't a problem if the area of effect is enforced.
Addendum: Don't overlook the audio limitation either
The above is all completely ignoring the impact of the stipulation that a phantasmal force is silent. Is a fireball silent? Not at all. An illusory fireball with no sound effects is not very convincing, and unlikely to fool even stupid opponents, even stupid opponents who have already been hit by a real fireball. In fact, having just experience a real one with all the sound and fury of real fire, they'd be more likely to notice that a silent one is not quite right.
Best Answer
We don't know; the spell text was not converted properly from OD&D and no errata was ever published. One suggestion is to add the word "additional" to get
"The spell caster is able to affect 1 additional level or hit die of creatures for each of his or her levels of experience."
so a 5th level caster can affect d6 creatures +5 levels.
Edit: I've not playtested this interpretation myself; it's merely one suggestion I've seen. Check with your DM.
More editing: Here's the original text from the very first issue of The Dragon FYI:
A sheet of bright conflicting colors. They affect 1-6 levels of creatures, rendering them unconscious through confusion. (Note: for every 5 levels above Trickster the caster has obtained, add one to the die roll for amount of levels, the number never to exceed 6.) The distribution of the effect if there are more target levels than spell levels is semi-random, first one creature is fully affected, then another, till all the levels are assigned, there being no more than one partially affected creature. There is no saving throw vs this spell if the creature is fully affected, if all but one level is affected, it gets a normal saving throw, for every level unaffected beyond the first, it gets an additional +2 on its saving throw, in any case, it will not affect any creature above the 6th level. Range 24”.