Yes you get to learn all those spells you mentioned, but magic always comes at a price. In this case while you "know" a bunch of spells you can still only cast a specific number of them each day. So at level 2 you can cast your at-wills 1 encounter, 1 utility and 1 daily spell (not including cantrips and rituals). You just get to choose which one you will, for lack of a better term, "charge up" for the day. Like Peteris said, at each extended rest you must choose which of your spells you will prepare for the following in game day. This keeps you at the same power level as other same level characters (except you get +1 at will that you can use at any time).
So, Bob the level 2 wizard opens his spell book in the morning after getting a good night sleep and turns to his "Utility Spells" page. He sees Expeditious Retreat and Shield, not wanting to get too close to enemies, he chooses to memorize the evasive spell Expeditious Retreat absorbing the spells magic for use later that day. He then flips through his spell book and does the same thing for the daily spells he has in his book. He then sets off on his travels with his companions being able to cast 3 at wills, 1 encounter, 1 utility and 1 daily.
The benefit here is if you know you'll be facing a certain creature and you have a spell that will deal lots of damage to it, you can prepare specifically for that engagement.
The idea with cantrips is that those are spells that the wizard is most familiar with. He used/practiced them so many times that he can use them at will, effortlessly. It doesn't make much sense that he could forget one of those and perfect another in a short span of time. For a permanent effect, I would agree that he could change it during Downtime.
If your player really regrets one of his choices, though, and you're not planning to give them downtime any time soon, you can think of a special one-time mechanic just to accommodate this change. For example, he could get possessed by a ghost of an ancient wizard during an encounter and his mind could get permanently altered as a side effect. You can always add other fun side effects as a "price" for such catering to the player's wishes, like strange visions or making his magic more unstable (always hurting him on a critical miss).
If your player just want the ability to occasionally use one of the cantrips he does not have, let him. Just make him use a spell slot for it. Any cantrip can be reasonably used as a first level spell in its weakest form. If he wants to use a stronger version of the cantrip, make him use more slots. You can look into how much damage it would do compared to regular spells to decide on the appropriate number of slots. If it's not a damage cantrip, one slot per improvement should do it.
Personal example of cantrip roleplay:
When I made my cleric, I thought carefully which cantrips would make sense for my background, so I chose Mending as one of my cantrips instead of something that would usually prove more useful (like Light). I've also avoided taking Guidance at first level, as I plan to build up to it through our campaign by continually assisting and advising my fellow adventurers. If a change of a cantrip would ever be more than a change in mechanics of the PC and actually make sense and contribute to the roleplay, then I would petition for it strongly. Otherwise, I'd say it's dabbling dangerously in the realm of min-maxing.
While I agree that being able to change one cantrip per level is mechanically sound, I'm still unsure if it fits from a roleplaying perspective. Maybe one every two or three levels. But I am leading only a low level group, where the time between levels is relatively short. If your players are around level 10, it might be enough time to retrain a cantrip. Especially if they're traveling in between distant locations, with not much happening on the road. The player should definitely roleplay the activity, though, making sure to note he's training the cantrip of his choosing.
On that note, some sort of training regime could be an additional mechanic. You could specify the number of training points required to change a cantrip. The player could then roll the die of your choice every time he trains (at maximum once per long rest) to accumulate training points. Or, he could spend a spell slot to do it, enabling him to spend all of his unused spell slots before his long rest. That would speed up the training the more unused slots he has, but leave him open to the risk of being "too tired to cast proper spells" in case of a surprise ambush during the long rest.
The "using unspent spell slots" scenario fits roleplaying well, too. For instance, if a player would want to learn Ray of Frost (1d8 damage), he could say that instead of casting one 1d8-damage ray he's casting 1d8 one-damage rays. After a sufficient number of rays cast, you'll say that the player has perfected the spell and can use it at will from now on. This would, of course, only be possible with cantrips, as the other spells are "too complicated" or "require too much energy" to be cast without spending a slot.
Best Answer
There is no general way to gain Cantrips as they are Wizard Utility powers with no level. Without a level they can't be used to substitute for another gained power.
There are a couple other ways besides being a gnome though