There is no RAW answer. It must be decided by the DM.
Firstly, this ambiguity is likely an intentional, direct consequence of 5e's design ethos of "rulings over rules" and the prominent role of the DM as the person that must make those rulings in the case where the rules are silent. Illusions and illusion magic are one area in which there is essential no general rules guidance, and thus falls squarely into the domain of DM-rule.
Jeremy Crawford said as much when asked about this exact question on Twitter:
Q: Can a creature under the effects of Phantasmal Force be restrained by it, i.e chains wrapping around a creature?
A: Phantasmal force details the few game effects that it can reliably create. A DM is free to allow additional effects.
Because illusions are a complicated issue, I will outline two of the major DM approaches to this issue. Since they take opposite stances to each other, any DM reading this will have to figure out which way makes the most sense, seems the most fun. and/or which way aligns better with how they've already been handling it.
The core of the disagreement is whether the DM rules that an illusion fools the mind so thoroughly that the body will react physically to it or not.
Interpretation 1: No, Phantasmal Force cannot chain someone and force them prone
In this argument, the spell looks and feels like a chain, but the chains (not being real chains) would not be able to restrain the creature since they would offer no resistance when the creature tries to struggle out of them.
The target rationalizes any illogical outcomes from interacting with the phantasm
Because of the above effect from the spell, the mind of the creature would try to rationalize this in some way (the chains broke, the chains loosened temporarily, etc.) but they would still believe the illusion to be true.
In this way the creature would not gain the prone condition because their movement is not actually inhibited in any way even though they believe the illusion is real.
See this answer for a more thorough take on this interpretation.
Interpretation 2: Yes, Phantasmal Force can chain someone and force them prone
In this interpretation, the chains really do restrain the creature because the creature treats the chains as real chains, even though they are not.
While a target is affected by the spell, the target treats the phantasm as if it were real.
The chains also sound and feel real to the target in every way.
An affected target is so convinced of the phantasm’s reality that it can even take damage from the illusion.
So, in this case, when the creature tries to escape from its restraints, its mind, thoroughly convinced that the chains are real, will act under that assumption. Thus, the creature's limbs will be bound. And this creates the case for the fact that the creature is able to gain the restrained condition from this spell.
See this answer for a more thorough take on this interpretation.
Decide but stay consistent
So I have provided very brief overviews of two interpretations of the spell's effect. There are certainly more of them out there. As a DM, the key is to decide on one and apply it fairly, transparently, and consistently. Not only across this spell but other similar ones.
Depends on the situation.
There are two things to consider.
The target rationalizes any illogical outcomes from interacting with the phantasm. For example. a target attempting to walk across a phantasmal bridge that spans a chasm falls once it steps onto the bridge.
and
The target can use its action to examine the phantasm with an Intelligence (Investigation) check against your spell save DC
Given that the targeted creature failed the first save**, no matter how smart a creature is, if it rationalize any illogical outcome it would not use any action to investigate the illusion. This hold true for any direct interaction between the illusion and the target.
But, if there is a situation where the affected creature has a reason to believe it might be an illusion, it can use its action as the rule states. For example, an ally seeing him fighting with a ghost might help him yelling "it is an illusion" or "it is in your head". In this case, the creature has a reason to suspect is an illusion an use its action accordingly.
** The description of phantasmal force says:
The target must make an Intelligence saving throw. On a failed save, you create a phantasmal object, creature or other visible phenomenon of your choice that is no larger than a 10-foot cube and that is perceivable only to the target for the duration. This spell has no effect on undead or constructs
Best Answer
(Disclaimer: the power of the illusions are controlled by the DM unlike most spells, the reason is because illusions might be too powerful and game changing for so little. Therefore, the best course of action is to ask the DM about your question, if you are the DM here are some guidelines but at the end is up to you)
Narrative wise
From the DM to the players, the way they present the illusion is the way it works (mostly, rules of the spell still apply). This is very important, specially in how the magic works in the world, for example, shield: does it make sound when you cast it or while it is active? What about light or shimmers? Some illusions like silent image and minor illusion+ don't make sound. A beast probably won't question why the wall that suddenly appear didn't make sound, but an intelligent creature familiar with a tiny bit of magic, yes, it would.
An interesting spell, narrative wise, is major image. Since major image can create illusion with smell, temperature, and sound, etc, the narrative of the spell should reflect the illusion presented. A wall emerging from the deeps of the earth making crumble sounds and the characteristic smell of earth, dust and cut grass will be way harder to disbelieve than a wall that just appear.
It just odd
This is the most basic way of disbelieving an illusion. Something might be misplaced and should not be there. For example, a cardboard box in the middle of a room undoubtedly will attract attention but the same box in a corner won't be noticed as much. Another example would be that the caster was too greedy and created an illusion so big that there wasn't room for a fake shadow. A giant guppy floating in the middle of the battlefield is a tell too. An example list would be:
Failed roll
Some DM will ask for roll checks when you want to create an illusion of a creature that you have not seen or that you usually do not use. These checks vary from illusion to illusion, the most common are arcane, nature and religion. A failed roll just increase the oddity of the illusion, it does not make an automatic failure, the "enemy" still have to pass the corresponding check if the illusion seems odd for him. For example, you know of a giant floating fish, you have seen one once, but you fail (nature roll) to recall the details, but the Orc that you are facing only heard of tales of such creature, he have to pass an history check to remember details of such creature. On a failed, the big fish is safe. This goes for hallucinatory terrain and other spells that require some degree of knowledge.
Phantasmal Force
This is a special case, mainly because of two particular things:
If we consider the initial save roll as failed, there is very little that the affected target can do for itself. Number 1 take care of things like thinking it is to odd for it to exist or to be real. Number 2 is particular problematic, since it make any interaction feel real and in order no matter how illogical the reaction or action was. But, there is a solution to this problem, third parties. Since any third party won't see what the affected creature sees, it is very easy for them to connect the dots and establish that something is very wrong with him. They can yell it's in your head or it's an illusion, they can ask what is wrong and latter say there is nothing there or the like. In that case, the affected target have a reason to suspect that it might not be real and act accordingly.