It depends.

The most common way in reality is to do so in plain sight. Often there's a distraction, e.g. a light bump or such used as a misdirection for the theft. This kind of thing is part of the Sleight of Hand skill; that's what it's for.

If your player doesn't want their target to be able to think back to what happened and infer who likely robbed them, then you may ask for an additional Stealth check to determine whether they were seen.

But failing the Stealth check wouldn't fail the theft - though your player may, finding their target looking at them, decide against continuing.

Additionally, if it's a while since the robbery occurred, it might make more sense to have the target do a "memory" check (using a general Intelligence check) to see if they remember the circumstances, though as far as I'm aware there are no rules governing this.

Can you continue to be not 'clearly visible' while moving and remain Hidden?

Yes, unless you are travelling for long distances at a normal or fast pace.

Can you become 'clearly visible' while moving and remain Hidden?

Yes, if the circumstance allows it, but usually not when you approach someone in combat.

Movement matters for hiding in two general cases

When you move over a distance measured in days, hours, or miles, this paragraph in Activity while Traveling applies:

While traveling at a slow pace, the characters can move stealthily. As long as they're not in the open, they can try to surprise or sneak by other creatures they encounter. See the rules for hiding in chapter 7.

While you are in combat, this paragraph of the Hiding sidebar applies:

In combat, most creatures stay alert for signs of danger all around, so if you come out of hiding and approach a creature, it usually sees you. However, under certain circumstances, the Dungeon Master might allow you to stay hidden as you approach a creature that is distracted, allowing you to gain advantage on an attack before you are seen.

There are also a handful of specific features and traits which tie movement and stealth together to varying degrees, such as the Ranger's Hide in Plain Sight feature or the Gargoyle's False Appearance trait.

You can't be both clearly seen and hidden from a creature

The Hiding sidebar states:

You can't hide from a creature that can see you clearly, and if you make noise (such as shouting a warning or knocking over a vase), you give away your position.

This applies throughout the whole time your are hidden, not just the moment when you begin to hide. Moreover, the sidebar states:

Ultimately, the DM decides when circumstances are appropriate for hiding.

Which implies a change in circumstances can cause you to become unhidden. This is consistent with the Ability Checks chapter, which states:

An ability check tests a character's or monster's innate talent and training in an effort to overcome a challenge. The DM calls for an ability check when a character or monster attempts an action (other than an attack) that has a chance of failure. When the outcome is uncertain, the dice determine the results.

If the outcome is certain, then dice do not determine the result. If the circumstance changes such that you are clearly seen, then by definition the outcome of hiding is certain failure and the stealth check no longer determines the outcome.

That said, whether being clearly visible means you are clearly seen depends on the circumstances. For instance, if you are out in the open and approaching a sleeping creature, you are clearly visible, but it's uncertain whether that creature wakes up and sees you, so the stealth check determines the outcome.