The cloak has a property that causes creatures to have disadvantage on attack rolls against the one wearing it (emphasis mine):
While you wear this cloak, it projects an illusion that makes you appear to be standing in a place near your actual location, causing any creature to have disadvantage on attack rolls against you. If you take damage, the property ceases to function until the start of your next turn. This property is suppressed while you are incapacitated, restrained, or otherwise unable to move (DMG, pg 158).
The way Shell Defense is described
You can withdraw into your shell as an action… While in your shell, you are prone, your speed is 0 and can't increase, you have disadvantage on Dexterity saving throws, you can't take reactions, and the only action you can take is a bonus action to emerge from your shell (EGW, pg 181).
is effectively the same as being both incapacitated and restrained, with the only notable difference being that an attack against a prone creature
has advantage if the attacker is within 5 feet of the creature. Otherwise, the attack roll has disadvantage (PHB, pg 292).
While extremely similar mechanically, the tortle using Shell Defense is technically neither restrained nor incapacitated. Furthermore, the ability to emerge from its shell as a bonus action implies the capacity to then move during that turn if the tortle chooses to, so neither are they otherwise unable to move.
As such, would the Cloak of Displacement work for a tortle using Shell Defense (and thus cancel out the advantage melee attackers would normally have due to the prone condition)?
Best Answer
The cloak won't help you while you are in your shell
While you are hiding in their shell you have a speed of 0 so you are unable to move. While you are like that the cloak won't work. You mention the fact that you can use a bonus action to come back out of your shell. But that is only possible on your turn and it is only relevant when you are actually do it. You aren't generally going to be attacked on your turn, aside from attacks of opportunity, but that requires you to move which you can't do while hiding in your shell. So if you end your turn while hiding in your shell, the cloak of displacement won't cancel out the effect of prone if it grants advantage on an attack.