Many areas of effect, like wall of fire, do damage to creatures when they enter it.
One side of the wall deals 5d8 fire damage to a creature that […] enters the wall for the first time on a turn.
That wording seems to imply that the creature takes the damage even if it was unwilling to move there (such as if pushed by a Repelling Blast or grappled and moved there).
However, prismatic wall's wording is different :
When a creature attempts to reach into or pass through the wall, it does so one layer at a time through all the wall’s layers. As it passes or reaches through each layer, the creature must make a Dexterity saving throw or be affected by that layer’s properties as described below.
Here, the words "attempts to" are used, instead of just "enters".
So, if a creature is unwillingly moved into a prismatic wall, does it have to make the saving throw(s)?
Creatures can be thrown through the walls, but would have to make the saves one at a time as they attempt to pass through them. Willingness to do so doesn't matter, unless specified in the spell.
I believe the wording is made in that way because a creature might not pass through all of the walls, because they're stopped short by an effect, leaving the rest of the walls up. I don't believe that the wording means you must intend to go through the walls to be affected by them, rather, you attempt to make it through all of them because there is a chance you might be stopped (or killed).
You make the saving throws one at a time, as you pass through it. You might pass through every wall, eventually, but only one at a time. Imagine time slowing down for each save, because the effect from the first might prevent you from traveling through the second.
Another thing to consider, most of the spells in the game specify if the creature needs to be willing or not. A quote supporting this from an unofficial source, lead rules designer Jeremy Crawford: