I could find no developer commentary explaining why the spells needed to drop a prismatic sphere's or prismatic wall's layers were picked for any edition of Dungeons and Dragons that preceded Pathfinder, and Dungeons and Dragons 3.5e is whence Pathfinder takes its prismatic sphere, spray, and wall.

The prismatic spray originates with Vance…

Vance's short story "Mazirian the Magician" (1950) is the prismatic spray spell's source:

Mazirian made a selection from his books and with great effort forced five spells upon his brain: Phandaal's Gyrator, Felojun's Second Hypnotic Spell, The Excellent Prismatic Spray, The Charm of Untiring Nourishment, and the Spell of the Omnipotent Sphere. This accomplished, Mazirian drank wine and retired to his couch.…

[Much later] Mazirian shook off the [foe's] spell, if such it were, and uttered a spell of his own, and all the valley was lit by streaming darts of fire, lashing in from all directions to split Thrang's blundering body in a thousand places. This was the Excellent Prismatic Spray—many-colored stabbing lines. Thrang was dead almost at once, purple blood flowing from countless holes where the radiant rain had pierced him.

As can be seen from the description, Vance's the Excellent Prismatic Spray spell doesn't much resemble the effects of the D&D spell prismatic spray (from which Pathfinder drew its spell of the same name). Further, I've found nothing to indicate that the prismatic spray spell's effect was changed to distance it from Vance's creation, or, instead, that the spell was named as an homage to Vance's invention, or, really, anything much about the spell's development at all. But it's an almost inconceivable coincidence that the name's accidental, given Gygax's affection for Vance's works.

…But the prismatic wall and prismatic sphere don't seem to

While in the same story Mazirian uses the Spell of the Omnipotent Sphere, the D&D spell prismatic sphere doesn't seem to owe Vance's spell anything:

He called his charm, the Spell of the Omnipotent Sphere. A film of force formed around his body, expanding to push aside all that resisted. When the marble ruins had been thrust back, he destroyed the sphere, regained his feet, and glared about for the woman.

To this reader, that sounds closer to (but still different from) Otiluke's resilient sphere or something.

Thus it's my understanding and the understanding of others far more well-informed than I that the spells prismatic sphere and prismatic wall were created created wholecloth for Dungeons and Dragons, with no fiction serving as inspiration. Some things are just new, I guess.

---

Research notes

Originally—that is, in the Player's Handbook (1977) for Advanced Dungeons and Dragons, which is as originally as I can muster—the necessary spells were, in order of layers dropped, cone of cold, gust of wind, disintegrate, passwall, magic missile, continual light, and dispel magic.

However, in order of spell level for magic-user—AD&D's wizard—, these are as follows: 1st—magic missile; 2nd—continual light; 3rd—dispel magic, gust of wind; 5th—cone of cold, passwall; and 6th—disintegrate. This comes really close to a magic-user needing to devote one spell per spell levels 1st through 6th to drop a prismatic wall or sphere. (The last layer's being dropped by dispel magic remains an outlier but a reasonable one.) It'd be great if I could find a version of passwall or cone of cold that was, instead, a 4th-level spell, but, alas, I can't. (Even the 1974 Dungeons and Dragons puts its spell pass-wall as a 5th-level spell and lacks the spell cone of cold entirely!)

Were the spells needed to bring down a prismatic sphere or wall to equal one spell per spell level, this would be a predictable and interesting design choice, but there appears to be no point in mandating two 5th-level spells and one 6th-level spell: A level 10 magic-user can cast two 5th-level spells but no 6th-level spells, yet a level 12 magic-user can cast four 5th-level spells and one 6th-level spell, more than enough to drop a prismatic sphere or wall and have spells remaining!

Thinking this might be campaign-dependent, I looked for AD&D liches. While liches are (thankfully!) rare, encounters with level 17 magic-users in AD&D are rarer, and I thought perhaps an early adventure module might be geared toward level 12 or higher PCs encountering foes who could cast prismatic sphere or wall. But, for example, Asberdies, the infamous lich from Descent into the Depths of the Earth (1978) that's for levels 9–14, doesn't memorize prismatic sphere or wall, nor do the two liches (and, incidentally, the ki-rin) from the Rogue's Gallery (1980) (which includes PCs from Gygax's campaign). The spells needed to drop prismatic layers appear oddly—but, perhaps, appropriately—random. (I mean, seriously, what magic-user memorizes continual light instead of stinking cloud?)

In Dungeons and Dragons, Third Edition makes some changes, but keeps the same weird non-pattern. The spells, in order of layers dropped, become cone of cold, gust of wind, disintegrate, passwall, magic missile, daylight, and dispel magic. While some names changed, spell levels didn't, so that, in order of wizard spell level, this is as follows: 1st—magic missile; 2nd—daylight; 3rd—gust of wind; dispel magic; 5th—cone of cold, passwall; 6th—disintegrate. The Dungeons and Dragons 3.5 revision swaps the spell levels of daylight and gust of wind, keeping intact the same non-pattern.

Spell level isn't the connection. Spell school isn't the connection (a spell from each school would've been just as convenient!). Expected character level seems to have no impact. It doesn't appear to be a puzzle, either, as there's no reasonable acronym or initialism or apparent code. Some stuff we may just never know unless a designer deigns to reveal his secrets.

Firstly, it's worth mentioning that the wording "or a similar spell of equal or higher level" and "or another spell of equal or greater level" is likely future-proofing to a certain degree, in case they bring out something like (or someone homebrews something like) "greater dispel magic" or something in the future, then if this specifically said "This layer is destroyed by dispel magic." then RAW, greater dispel magic wouldn't work, even though it clearly should.

Nonetheless, some of these layers have existing spells that would bring down the layers in question besides using the examples given. Let's tackle them one by one:

Orange Layer

This layer "is destroyed by a strong wind." A strong wind? Well, firstly, I'll include Joel Harmon's comment:

Incidentally, "Strong Wind" is a term defined on DMG 110. It describes some effects on flames, fog, and fliers, but doesn't have any hard numbers.

Unfortunately, as the comment says, there are no "hard number"; this appears to be describing an environmental effect, so yes, if someone cast prismatic wall outside on a very windy day, a DM could rule that this layer might go down immediately, but in the middle of the dungeon, that's not going to happen. So we need to use magic.

The gust cantrip isn't going to cut it; gust of wind, wind wall, warding wind and whirlwind, on the other hand, all create strong enough wind to take this wall down.

From gust of wind (PHB, pg. 248):

A line of strong wind 60 feet long at 10 feet wide blasts from you in a direction you choose for the spell's duration.

From wind wall (PHB, pg. 288):

A wall of strong wind rises from the ground at a point you choose within range.

From warding wind (XGtE, pg. 170):

A strong wind (20 miles per hour) blow around you in a 10-foot radius and moves with you, remaining centred on you.

From whirlwind (XGtE, pg. 171):

A whirlwind howls down to a point that you can see on the ground within range.

Ok, this last one doesn't contain the phrase "strong wind", but it's a whirlwind! I think it counts...

Green Layer

This layer requires "A passwall spell, or another spell of equal or greater level that can open a portal on a solid surface". Unfortunately, the only other portal related spell I can think of that uses a surface is teleportation circle, which specifies "the ground" (PHB, pg. 282):

As you cast the spell, you draw a 10-foot diameter cricle on the ground inscribed with sigils that link your location to a permanent teleportation circle of your choice...

I imagine the intent of this wall's requirement is to cast the spell similar to passwall on the wall itself, not on the floor next to it, so I don't think teleportation circle can help us.

There is demiplane (thanks @Nick), which says (PHB, p. 231):

You create a shadowy door on a flat solid surface that you can see within range. The door is large enough to allow Medium creatures to pass through unhindered.

I think this would count as creating a spell of greater level that can open a portal on a solid surface, so demiplane should bypass the green layer.

(Thanks to @illustro for suggesting the next three spells)

Mordenkainen's magnificent mansion could be an option, since it somewhat ambigiously says (PHB, pg. 261):

You conjure an extradimensional dwelling in range that lasts for the duration. You choose where its one entrance is located.

Gate says a similar thing (PHB, pg. 244):

You conjure a portal linking an unoccupied space you can see within range to a precise location on a different plane of existence. The portal is a circular opening, which you can make 5 to 20 feet in diameter. You can orient the portal in any direction you choose.

These entrances or portals could be on the green wall (gate less obviously, but it could be oriented to be against the wall), thus serving as an alternative to passwall, although they're not explicitly cast against the wall like passwall is, so it's a DM's call as to whether this would work.

Arcane gate could also be an option, given that you could align the free-floating portals with the wall (PHB, pg. 214):

The portals are two-dimensional rings equal to 10 feet in diameter, hovering perpendicular to the ground. A ring is visible only from one side (your choice), which is the side that functions as a portal. [...] On your turn, you can rotate the rings as a bonus action.

The fact that it's hovering, it could be argued that this is therefore definitely not interacting with the wall, but the mentioning of rotation could imply that it could be aligned with the wall, as with gate.

Another suggestion (thanks to @CapnZapp) could be using the Nolzur's Marvelous Pigments (DMG, pg. 183), a very rare magic item that can:

allow you to create three-dimensional objects by painting them in two dimensions.

...

Thus, painting a door on a wall creates an actual door that can be opened to whatever is beyond.

I, as DM, would personally allow this as it seems similar enough to passwall for the purposes of taking down this layer, although it must be said that RAW this magic item does not cast passwall, nor does it cast "another spell of equal or greater level", so strictly RAW this one probably wouldn't work.

Overall, many of these options would be up to the DM, since passwall and demiplane seem to be the only spells that explicitly say that it's cast on a solid surface.

Indigo Layer

This layer requires "bright light shed by a daylight spell or a similar spell of equal or higher level." Those that spring to mind (or @illustro's mind) are dawn, wall of light, sunbeam and sunburst, all of which create bright light, although I should mention that clearly the light cantrip will not work (even though that's obvious) since it's not of equal or higher level to daylight.

From dawn (XGtE, pg. 153-154):

The light of dawn shines down on a location you specify within range. Until the spell ends, a 30-foot-radius, 40-foot-high cylinder of bright light glimmers there. This light is sunlight.

From wall of light (XGtE, pg. 170):

A shimmering wall of bright light appears at a point you choose within range.

From sunbeam (PHB, pg. 279):

A beam of brilliant light flashes out from your hand in a 5-foot-wide, 60-foot-long line.

From sunburst (PHB, pg. 279):

Brilliant sunlight flashes ub a 60-foot radius centered on a point you choose within range.

Also, if we include magic items, it's worth mentioning that the Driftglobe can cast the daylight spell, and that there's a magic item called the Gem of Brightness, which says (DMG, pg. 171):

The second command word expends 1 charge and causes the gem to fire a brilliant beam of light at one creature you can see within 60 feet of you.

Unfortunately, this won't help us since it wants to target a creature, and we need it to take down a wall. However, the next bullet point says:

The third command word expends 5 charges and causes the gem to flare with blinding light in a 30-foot cone originating from it.

The only trouble with the Gem of Brightness is that whether the effects of an uncommon magic item are considered "of equal or higher level" to a daylight spell, and the fact that the description of this layer of the prismatic wall specifically says "or a similar spell", and the effect of a magic item is not a spell (although a DM may allow it anyway, since it's light from a magical source with a similar effect to daylight; I'd allow it).

Driftglobe is probably alright RAW because it mentions the daylight spell specifically.

Violet Layer

"This layer is destroyed by a dispel magic spell or a similar spell of equal or higher level that can end spells and magical effects." Besides dispel magic, what else can "end spells or magic effects" in such a way?

Possibly greater resortation, since it can remove certain conditions like petrified and the like, but I'm not sure if it is similar enough to dispel magic, since it just stop a creature being affected by such spells or effects. This one might have to be a DM's call as well.

On the other hand, remove curse should be valid RAW, as it is considered to end curse spells; from bestow curse (PHB, pg. 218):

A remove curse spell ends this effect.

Hex also says this about it (PHB, pg. 251):

A remove curse cast on the target ends this spell early.

So remove curse is called out as being able to end spells early at least twice, meeting the criteria for being a spell "that can end spells and magical effects".

It's also worth pointing out that an antimagic field wouldn't work, since prismatic wall specifically says (PHB, pg. 269):

The wall can be destroyed, also one layer at a time, in order from red to violet, by means specific to each layer. Once a layer is destroyed, it remains so for the duration of the spell. Antimagic field has no effect on it, and dispel magic can affect only the violet layer.