Verbal components are audible, but the message is not.

The message itself is separate from the verbal component

The message spell has a verbal component. However, the verbal component of the spell is not the same as the message itself, merely a necessary part of casting the the spell in the first place; the actual message is part of the spell effect. (Likewise, the somatic component of message is separate from the caster pointing at the targeted creature as part of the spell's effect.)

This is supported by a related official ruling in the Sage Advice Compendium regarding the spells suggestion and command, both of which also have verbal components:

Is the sentence of suggestion in the suggestion spell the verbal component, or is the verbal component separate?

Verbal components are mystic words (PH, 203), not normal speech. The spell’s suggestion is an intelligible utterance that is separate from the verbal component. The command spell is the simplest example of this principle. The utterance of the verbal component is separate from, and precedes, any verbal utterance that would bring about the spell’s effect.

All three of these spells have a verbal component, and separately involve speaking a word or sentence as part of the spell effect. They all work the same way in this regard; the verbal component that's part of the casting of the spell is distinct from the utterance spoken as part of the spell's effect.

Verbal components are audible, but it's up to the DM how audible

The description of verbal components in the rules states, in part (PHB, p. 203):

Most spells require the chanting of mystic words. The words themselves aren't the source of the spell's power; rather, the particular combination of sounds, with specific pitch and resonance, sets the threads of magic in motion.

The rules don't specify how loud a verbal component has to be, so it is left to DM discretion. This is also reiterated by rules designer Jeremy Crawford in an unofficial tweet from October 2015:

when I cast a spell does the verbal component has to be clearly audible or can I "whisper" the spell?

The verbal component of a spell must be audible to work. How loud is audible? That's up to the DM.

The caster ostensibly doesn't have to scream it at the top of their lungs, but in general I think verbal components of spells would need to be spoken in (approximately) a regular speaking voice, or at a comparable volume.

DMs may choose to allow spellcasters to attempt to hide spell components (e.g. by whispering/muttering the verbal component) - potentially requiring an ability check to do so - but this runs the risk of devaluing the sorcerer's Subtle Spell Metamagic option (which removes verbal and somatic components from the spell entirely). Even beyond that, granting all casters a way to hide spell components for free generally makes all spellcasters more powerful regardless.

Only the target hears the message; only the caster hears the reply

As for the volume of the message, the description of the message cantrip states (emphasis mine):

You point your finger toward a creature within range and whisper a message. The target (and only the target) hears the message and can reply in a whisper that only you can hear.

The spell description makes it clear how loud the message itself is, and who can hear it. The caster whispers a message that only the target can hear, and the target can reply in a whisper that only the caster can hear. Thus, even if someone can tell that you're casting a spell, only the creature you point at can hear the actual message.

No, Vicious Mockery is not a charm

While vicious mockery is a spell from the school of Enchantment, none of its capabilities include charm effects or the charmed condition.

The spell itself is not an opinion-effecting spell - it is a damage and debuff spell. You can see this in the tags on dndbeyond and note that it does not carry the social tag.

Mechanically, the spell only does what it says, which is:

You unleash a string of insults laced with subtle enchantments at a creature you can see within range. If the target can hear you (though it need not understand you), it must succeed on a Wisdom saving throw or take 1d4 psychic damage and have disadvantage on the next attack roll it makes before the end of its next turn.

These effects are not subtle. You have cast a spell, dealt psychic damage, and done something to make it more difficult for them to succeed on their next attack.

A possible way forward

This doesn't mean that a DM could let you use it in this way, but just that the general mechanics for it don't include it. If you did use this, and the target didn't fully engage in combat back, you may be able to pair it with a more social skill of intimidation, but that is going to be entirely up to the DM and wouldn't be that difference than threatening someone in general after hitting them with a melee or ranged attack.

Narrating effects

It's also important to consider the narration process. This is a highly personal piece, but I generally approach by narrating to support the mechanical effects. Narrating as a means of explaining may introduce more confusion than necessary and create misunderstandings of what a spell can do. Some caster classes, like the Bard, are limited in what spells they know. Stretching spell capabilities beyond that minimizes that limitation - but if that matters is highly table dependent and would be a creative use of a spell.