The folding boat explicitly says that it takes an action to speak the command word. If your player is using his action to cast catapult, he doesn't have an action left to properly incant the command word before the attack finishes.
The catapult spell says that it targets an item weighing "1 to 5" pounds, and it propels the target "in a straight line". This isn't throwing the item, it's moving it magically. If the item ceases to be a valid target for the spell (because it weighs too much), the spell fails and the item falls to the ground.
The catapult spell deals equal damage to the thing being thrown as to the target. The foldable boat probably can't take too much damage before it stops folding properly.
Finally, it's not clear from the folding boat description how long it takes to gain all that mass. If you don't think explosive decompression is reasonable, you might rule that the unfolding process takes a minute or two.
The players have already used this once and seen that it worked. I recommend being honest with them: "Guys, the boat-catapult thing worked once because it caught me by surprise, but I went and looked at the rules more closely and I have the following concerns. We'll leave the result of the previous combat as it happened, but if you try it again, we'll follow the rules correctly."
No.
The simplest way to approach this is to take the rules literally. The interplanar effects only happen when the sphere touches a planar portal or an extradimensional space. An antimagic field is none of these, so it will not "go off".
Given that there's no specific interaction between an antimagic field and a sphere of annihilation, we then fall back on your first interpretation, which is that the sphere winks out of existence only while the antimagic field is active, in the same way that any other magic item is deactivated.
Best Answer
Nothing. The boat becomes a mundane version of whatever shape/state it is currently in (unless the DM says otherwise)
The folding boat is classified as a magic item, despite the fact that its description doesn't indicate that its different configurations are switched or maintained by magic:
Page 18 of the Sage Advice Compendium lists the criteria for determining if something is magical:
The first and last apply to the boat. The first, as stated above, obviously true but it doesn't offer any guidance with regard to how the Folding Boat interacts with an Antimagic Field.
The last bullet is also relevant: Nothing in the boat's description says "magic." We know that the boat is magic and therefore its transformation abilities must be magic, but we are left in the lurch with regard to whether or not the box/boat/ship shapes are maintained by magic in some way.
Ryan C. Thompson aptly points out that the boat seems to violate the laws of physics through its capability of containing everything in a 12 x 6 x 6 cube. But then again, D&D is not a physics simulator.
The Folding Boat's description does not give any explicit indication that magic is required to maintain a given shape, only that the transformation between shapes does. This is important because the antimagic field spell says:
Therefore, if we assume that the Folding Boat's magic only drives the shape changing, it is subject to the third part of antimagic field's effect: it becomes a mundane version of its normal magical self. To wit: a box or boat or ship, depending on whichever state it happens to be in.
The only difference between the folding boat being within an antimagic field and outside of one is that the command words that would normally cause it to change would have no effect while it was within the field. It would still function as a mundane boat or ship (or box) while within the field.
Under this interpretation, it's also worth noting that the boat might be capable of being manually converted to a shape (inside or outside of an antimagic field). In other words, inside an antimagic field, it's functionally a mundane collapsible boat/ship kit. Deployed this way, it would be up to the DM to decide if this is possible and how long it takes to assemble or break down since the rules offer no guidance in this matter.
However, a DM might look at the unexpected physics of the Folding Boat and think that magic is required for it to achieve/maintain a given shape. If this is the case, that DM would need to come up with their own explanation for what happens to the ship in an Antimagic field. Perhaps it simply collapses into a pile of parts?