Rules as Written are Unclear
The Force Cage spell states (PHB, pp243)
When you cast the spell, any creature that is completely inside the cage's area is trapped.
And then separately:
Creatures only partially within the area, or those too large to fit inside the area, are pushed away from the center of the area until they are completely outside the area.
Given that these are are two separate sentences, the "too large to fit" clause may or may not be directly tied to the "when you cast the spell" clause. If it applies to both at time of casting as well as during the life of the spell then the creature is shunted (if it is two sizes larger than the cage - see squeezing rule.) If it only applies to the cage creation, then it raises the issue of a creature being in too small a space (see below.)
If those two sentences are supposed to be directly related, it changes the interpretation and then allows for use of Force Cage to autokill something. There are no real rules for this, and this question on polymorph may be related with regard to too small a space. See the Concerns section below for more information.
Squeezing into a smaller space
On the chance that the polymorphed creature is just one size smaller then the creature itself, the Squeezing rules would come into play (PHB, pp192)
A creature can squeeze through a space that is large enough for a creature one size smaller than it...It has disadvantage on attack rolls and Dexterity saving throws. Attack rolls against the creature have advantage while it's in the smaller space.
Concerns
There are two concerns that you should consider when determining use of this at your table:
If you choose to read the Forcecage text as not applicable post-casting, then you have created a neat effect to instantly kill a creature. This could become very powerful if you have a group that is able to do this. However, this is also using a 7th level spell slot (along with a 3rd) - and may not be as unbalanced as it seems. This is similar, but not identical, to the idea of polymorphing into a tiny creature to be eaten or go into a larger creature and then end the polymorph so you "burst out" and kill the creature. There are no RAW rules around this and allowing tactics like this potentially creates a slippery slope (although not as great when you consider the 7th level spell cost for Force Cage.)
If you choose to go with the interpretation of being shunted out, you are risking devaluing the Forcecage. The odds of this happening are small as it is unlikely to be used against the party by an enemy.
Squeezing
what if you arrange your panels so there's a section only 2" wide? Is the creature squeezed between the walls?
First, the space a creature takes up is not the same as the size the creature controls.
A creature's space is the area in feet that it effectively
controls in combat, not an expression of its physical
dimensions. A typical Medium creature isn't 5 feet
wide, for example, but it does control a space that
wide. If a Medium hobgoblin stands in a 5-foot-wide
doorway, other creatures can't get through unless the
hobgoblin lets them. (PHB 192)
The rules on squeezing are pretty clear for most cases:
A creature can squeeze through a space that is large
enough for a creature one size smaller than il. Thus, a
Large creature can squeeze through a passage that's
only 5 feet wide. While squeezing through a space,
a creature must spend 1 extra foot for every foot it
moves there, and it has disadvantage on attack rolls
and Dexterity saving throws. Attack rolls against the
creature have advantage while it's in the smaller space. (PHB 192)
So, a medium creature can squeeze into a space that is Small, which according to DMG page 6 is a 2.5x2.5 ft square it controls.
Squishing
2 feet is less than 2.5 feet so what happens? The rules simply don't say, and a ruling is necessary. So here are some options the DM might rule.
The Creature Takes Bludgeoning Damage
The DM could rule that like in real life when getting stuck in a space that's too small, the creature gets squished into the space, all the things of being squeezed applied, and take some amount of bludgeoning damage. They may also say the space so cramped he's not just losing a foot of movement, but maybe he's in difficult terrain or even restrained.
The Creature is Pushed The Other Way
The DM could rule there isn't enough room and spell isn't a damage spell, and the spell simply shunts them the other way instead.
The Creature is Squeezed
The DM could rule that sizes are an approximation and abstraction. All medium creatures fit inside of a 5' square, but maybe the elf has a slight enough frame can squeeze not only into a 2.5' square but even into a 2' square. The DM could continue that logic to even smaller spaces if they felt appropriate.
Best Answer
The rules assume you already know how walls work
I've looked everywhere I can think of, and as far as I can tell there's nothing in the rules explaining that you can't walk through walls.1 Despite this, I've never met or heard of a DM that allows their players to walk through walls. Walls are commonly understood by everyone to be barriers to movement, in addition to providing total cover. This includes both voluntary movement, such as walking and flying; as well as involuntary movement, such as being pushed, pulled, or carried; but generally not teleportation and similar effects that allow one to arrive at their destination without movement.
(It might seem obvious or even a bit patronizing to go through all of this, but it's important to establish it as a baseline that we all agree on, since as far as I can tell "walls block movement" isn't actually an explicit rule. Also, JC has stated in a tweet that walls "work the way we expect them to, except for when the rules say otherwise.")
Magical pushing still causes physical movement
So if we take it as a given that walls are impassible, the question becomes whether there is anything special about a push effect caused by a spell or magical effect, compared to something like a mundane shove attack. In general, I would say there is no difference. Just because the thing that launched the target backward is magic doesn't make the movement itself magical. Even if you were to pick up the target with a Telekinesis spell and try to carry them through the wall, it wouldn't work, whether the wall belongs to a castle or a Forcecage spell. Regardless of the force doing the pushing, the target is being physically moved, so any barrier to physical movement will block them.
You need the right "magical means" to escape a Forcecage
The Forcecage spell says:
This establishes that you can't escape the cage by hacking your way through the wall with a axe, picking a lock, using a crowbar to separate the bars, etc. However, it doesn't mean that any magic spell or effect will get you out of the cage. Even when you're using magic, the wall of the cage is still a wall. You're going to need a spell that can get you through a wall. Teleportation is one way, subject to the limitations described in the spell. Disintegrate can also punch a hole in the cage for you. However, Passwall isn't going to help you, because the cage isn't "wooden, plaster, or stone", and Etherealness won't get you out either, since the cage extends into the Border Ethereal. There may be other spells that can get you out, but the point is, a spell will only allow you to escape the cage if it helps you pass through a wall made of magical force.
1The closest I can find to an explicit statement that walls are impassible is this rule from the grid variant rules:
There is also this excerpt from the DMG about movement in the border ethereal plane:
Both of these assume that walls are impassible, but don't say so on their own.