No, the first two points don't hold for any caster with Mage Hand.

Pickpocketing isn't just taking something from a pocket; it's a subtle skill which requires doing so unnoticed, after all. This is more than just "doing it while the victim's back is turned" - the human body registers slight touches and subtle sensations, like the weight of an object. A pickpocket knows how to beat these senses - by touching the victim in other places to confuse the senses, for example.

But the Mage Hand is implied to be clumsy, with limited dexterity - not conducive to stealth.

(Note that nothing in the Mage Hand description says that the hand has significant dexterity - it can "open an unlocked door", for example, or "pour the contents out of a vial", but apparently not manipulate lockpicks or disarm traps, which requires more precision. It's limited almost exactly to simple tasks that you don't have to practice.)

So it's not that the Mage Hand eliminates the normal skill check as such - instead, the rules as written do not contain a "normal" skill check for picking a pocket with a Mage Hand. (A standard thief can't pick locks with a Mage Hand either, even if they somehow have one.)

Most people can undo the buckle on a bag, or shoe. But can you do so so swiftly and subtly that the holder of the bag (or wearer of the shoe) doesn't even notice? Try it. It's not as easy as it sounds. It needs significantly more manual dexterity than just being able to undo a buckle - it's more like being able to play a piano.

Under the old-school skill system of 3e or 4e, the correct way to handle this would have been with a skill roll and an extremely hard DC - as GM, personally I'd have set the DC 10 or even 20 points higher than the usual for that kind of pickpocketing. The old skill system would then have allowed top-level characters to pull it off anyway.

But 5e discourages this "everything is technically possible with the right roll" approach, in favour of limiting skills to "actions anyone could attempt". "Pick a pocket with a magical force" isn't something anyone could attempt, and there's no obvious RAW reason why being able to pick pockets by the normal means would help you use a Mage Hand to do so.

(And it is typically next to impossible to pickpocket successfully with a fully visible, somewhat clumsy, disembodied hand. So even a disadvantage is not enough penalty - it should just be impossible.)

Legerdemain clearly gives the hand more dexterity, not just invisibility. This allows for more complex tasks.

Short answer

If the standard hand doesn't have enough manual dexterity to pick locks or disarm traps, it doesn't have enough for the equally tricky task of picking pockets.

Point of clarification: the question as you pose it in your title asks if gust can move "someone" under the effects of levitation, but in the body of your question you are specifically asking about moving yourself. I will focus on the latter.

By Rules as Written? No, absent DM fiat.

The general rule is that spells do only what they say they do, and anything more is up to the DM. The description of gust, which you've quoted, says it can be used to push others, but says nothing about moving the caster.

To put this in perspective, consider that you are effectively looking to use gust to target yourself on the receiving end of the push effect. Technically, targeting yourself might be fair game; the spell says you can target "[o]ne Medium or smaller creature that you choose," and you could perhaps choose yourself. The trouble is, the target is "pushed up to 5 feet away from you," and you cannot be pushed away from yourself. The DM could choose to ignore that bit of illogic, but it would be fiat.

By Rules as Fun? Absolutely yes.

Gust is an underwhelming cantrip. Most of its effects are duplicative of other (better) cantrips -- except for the push effect, which can be achieved just as easily by any character using the rules for shoving a creature. See PHB p. 195. Permitting a PC to cast gust in order to achieve 5 feet of lateral movement while levitating rewards creative play, and is highly unlikely to break the game. (Consider, for example, how difficult it would be to abuse this setup in combat. The PC is expending an entire action for 5 feet of movement.)