I've been DMing 1st edition for about 12 years now, and while I've experimented with different editions I always go back to 1st. Over the years DMing I have written a number of my own house rules to fix the many balancing issues 1st edition has. But, there is one spell/mechanic that I have yet to find a good way to balance, and that is the Magic User/Illusionist spell phantasmal force.
For magic users the spell is a 3rd level spell (can't be used until the magic user reaches 5th level), and generally speaking my players who play magic users don't tend to rely on it too much and the power level of the spell doesn't feel overpowered when you have other spells of the same level like fireball or lightning bolt.
The problem comes with players who play an illusionist. For illusionists, phantasmal force is a 1st level spell. My groups have gotten smart enough to know that at least one person should play an illusionist and memorize as many phantasmal forces as they can each day. Very rarely do my player illusionists use other illusionist spells. The reason why is that phantasmal force is such a powerful spell if used correctly (deadly illusions are fatal to those who believe it). The most common illusions I see from it are a continuous barrage of illusory fireballs or magic missiles, which can clear out entire hordes of dumb enemies who believe it. (To increase the chance that an enemy would believe the illusion, my players like to do a trick where they have the magic user cast the real thing, then have the illusionist step up and continuously replicate it. Since the enemy has been hit by the real thing they have no reason the disbelieve the fake one.)
Given the rules for the spell:
Phantasmal Force (Illusion/Phantasm)
Level: 3
Range: 8″ + 1″/level
Duration: Special
Area of Effect: 8 square inches + 1 square inch/level
Components: V, S, M
Casting Time: 3 segments
Saving Throw: SpecialExplanation/Description: When this spell is cast, the magic-user creates a visual illusion which will affect all believing creatures which view the phantasmal force, even to the extent of suffering damage from phantasmal missiles or from falling into an illusory pit full of sharp spikes. Note that audial illusion is not a component of the spell. The illusion lasts until struck by an opponent — unless the spell caster causes the illusion to react appropriately — or until the magic-user ceases concentration upon the spell (due to desire, moving, or successful attack which causes damage). Creatures which disbelieve the phantasmal force gain a saving throw versus the spell, and if they succeed, they see it for what it is and add +4 to associates’ saving throws if this knowledge can be communicated effectively. Creatures not observing the spell effect are immune until they view it. The spell can create the illusion of any object, or creature, or force, as long as it is within the boundaries of the spell’s area of effect. This area can move within the limits of the range. The material component of the spell is a bit of fleece.
What are some ways I can modify the rules or reinterpret the rules to limit its power and stop players from overusing it, while at the same time not limiting the power so much that no one wants to play an illusionist?
The only thing I've done so far to limit the power of the spell is to "fudge" saving throws for targets. So, in a case where the spell would normally clear a room of enemies, I usually make sure at-least 1/4 disbelieve the spell (or simply weren't paying attention) and survive just so the remainder of the PC's have something to do.
Best Answer
There's no need to house rule this — you just need to start enforcing its restrictions. Allow me to draw your attention to its area of effect:
(Aside, this is taken from PHB page 95, since the Illusionist version of the spell has different range and area of effect.)
These are table inches, which we could convert to feet/yards, but the following will be simpler if we just compare table inches directly with fireball's numbers.
A fireball has a radius of 2″. Let us ask then, “what level would an illusionist have to be to cast a phantasmal force large enough to contain a 2″ radius sphere?” Let's make a first-order approximation using the minimum number of 1″ squares necessary to create an area in which the image of the sphere could appear. The area of effect of fireball is three-dimensional, but the area of effect of phantasmal force is two-dimensional, which presents an odd issue. However, phantasmal force is described as being able to create images “within the boundaries”, so let's be as generous as the spell seems to be intended, and say that the vertical space it can create the illusion within is “good enough” for our purposes and we just need the footprint of the illusory fireball to fit inside the phantasmal force's area of effect.
We'll assume as a first-order approximation that each 1 square″ of area of effect can't be divided, so they have to be tiled as whole square table inches. Inscribing a 4-unit sphere's footprint (a circle) inside a pixelized shape requires a 4×4 square of squares, or 16 tiled squares.
So, as a first approximation, an illusionist needs to have 16 square table-inches of area of effect to play with to be able to make a phantasm of a full fireball. That requires being level 12 to pull off.
But maybe you think tiled squares aren't generous enough. Let's allow the table-inch squares of area of effect of the phantasmal force to be subdivided infinitesimally so that the area can be a cylinder that just barely contains the footprint of an illusory fireball and see where that gets us. (Some DMs might reject this move since the area of effect is given in squares and they may require a straight-edged area of effect, but some others might allow it with the reasoning that it's fine as long as the final area matches regardless of shape.)
A 2-unit-radius sphere has an area through its widest point of just over 12½ units, rounding up to 13 square″ required. So even being super-generous, it would still take an illusionist of level 9 to use phantasmal force to create an illusory fireball.
Clearly, creating illusions of fireballs with phantasmal force isn't something that your players should be doing at first level, or even 5th level when their magic-user friends are starting to throw real fireballs.
Conclusion: You don't need to house rule, just give the players a reality check
So, there you have it: you don't need to house rule this at all, you just need to pay proper attention to the relatively small area of effect, and break the news to your players that you've been accidentally letting them create a 9th-level illusionist's phantasmal forces before they were actually 9th level, and won't be doing that any longer.
If they are 9th- or 12th-level, then this isn't overpowered at all, because they have access to equally (or more) powerful effects already and cleverly using their resources like this should be letting them paste low-HD enemies by the truckload. So again, there simply isn't a problem if the area of effect is enforced.
Addendum: Don't overlook the audio limitation either
The above is all completely ignoring the impact of the stipulation that a phantasmal force is silent. Is a fireball silent? Not at all. An illusory fireball with no sound effects is not very convincing, and unlikely to fool even stupid opponents, even stupid opponents who have already been hit by a real fireball. In fact, having just experience a real one with all the sound and fury of real fire, they'd be more likely to notice that a silent one is not quite right.