The idea with cantrips is that those are spells that the wizard is most familiar with. He used/practiced them so many times that he can use them at will, effortlessly. It doesn't make much sense that he could forget one of those and perfect another in a short span of time. For a permanent effect, I would agree that he could change it during Downtime.

If your player really regrets one of his choices, though, and you're not planning to give them downtime any time soon, you can think of a special one-time mechanic just to accommodate this change. For example, he could get possessed by a ghost of an ancient wizard during an encounter and his mind could get permanently altered as a side effect. You can always add other fun side effects as a "price" for such catering to the player's wishes, like strange visions or making his magic more unstable (always hurting him on a critical miss).

If your player just want the ability to occasionally use one of the cantrips he does not have, let him. Just make him use a spell slot for it. Any cantrip can be reasonably used as a first level spell in its weakest form. If he wants to use a stronger version of the cantrip, make him use more slots. You can look into how much damage it would do compared to regular spells to decide on the appropriate number of slots. If it's not a damage cantrip, one slot per improvement should do it.

Personal example of cantrip roleplay:

When I made my cleric, I thought carefully which cantrips would make sense for my background, so I chose Mending as one of my cantrips instead of something that would usually prove more useful (like Light). I've also avoided taking Guidance at first level, as I plan to build up to it through our campaign by continually assisting and advising my fellow adventurers. If a change of a cantrip would ever be more than a change in mechanics of the PC and actually make sense and contribute to the roleplay, then I would petition for it strongly. Otherwise, I'd say it's dabbling dangerously in the realm of min-maxing.

While I agree that being able to change one cantrip per level is mechanically sound, I'm still unsure if it fits from a roleplaying perspective. Maybe one every two or three levels. But I am leading only a low level group, where the time between levels is relatively short. If your players are around level 10, it might be enough time to retrain a cantrip. Especially if they're traveling in between distant locations, with not much happening on the road. The player should definitely roleplay the activity, though, making sure to note he's training the cantrip of his choosing.

On that note, some sort of training regime could be an additional mechanic. You could specify the number of training points required to change a cantrip. The player could then roll the die of your choice every time he trains (at maximum once per long rest) to accumulate training points. Or, he could spend a spell slot to do it, enabling him to spend all of his unused spell slots before his long rest. That would speed up the training the more unused slots he has, but leave him open to the risk of being "too tired to cast proper spells" in case of a surprise ambush during the long rest.

The "using unspent spell slots" scenario fits roleplaying well, too. For instance, if a player would want to learn Ray of Frost (1d8 damage), he could say that instead of casting one 1d8-damage ray he's casting 1d8 one-damage rays. After a sufficient number of rays cast, you'll say that the player has perfected the spell and can use it at will from now on. This would, of course, only be possible with cantrips, as the other spells are "too complicated" or "require too much energy" to be cast without spending a slot.

You can only cast them once per long rest

As you quoted above, you can only cast each spell once per long rest. A cantrip is still a spell (just at 0 level, which would be the lowest level you cast it as) and therefore falls under the same requirement.

Counter-examples

Dragonmark differs in its language and requirements from things like the Magic Initiate Feat or The cantrips gained from a certain race (like High-elf.) Those simply add that cantrip to your current list and do not have a limitation on uses/day.