Vow of Enmity can be used if all of these are true

1. You are able to use a bonus action.
2. You are able to "utter" things.
3. There is a creature within 10 feet of you.
4. You can see that creature.
5. You have a use of your Channel Divinity available.

That's it. There is no requirement that the creature even be hostile - you could utter a vow of enmity against a housecat that's calmly sitting at your feet.

D&D 5e is balanced around abilities doing what they say they do

A common mistake that new DMs make is applying "logic" to mechanics. I call it a mistake, not because it's forbidden by the rules - the DM has the power to describe the world as they see fit - instead, I see it as a fundamental misunderstanding of what 5e is as a system. D&D 5e is primarily mechanics-focused, particularly when it comes to combat - hence the scare-quotes around "logic".

If you start applying "logic" to the mechanics of 5e, it's really easy to break things. The classic example is DMs requiring rogues to actually sneak around to use their Sneak Attack feature (rather than just have advantage or an adjacent ally). This "makes sense", but it also completely guts a rogue's combat ability. On turns without Sneak Attack, a 20th-level rogue only deals slightly more damage than a 1st-level wizard.

Bringing it back around to your Vow of Enmity - Oath of Vengeance is balanced around having advantage against one creature per short rest. In exchange for this powerful ability, their 7th level Relentless Avenger feature is the weakest of the three PHB paladin subclasses. On the flip side, Ancients and Devotion paladins get less impressive 3rd-level options, but powerful 7th level options.

The answers to this question may also be of interest to you: What is the source of the "spells do only what they say they do" rules interpretation principle?

Class features may have non-mechanical names

When I say "abilities do what they say they do", I'm really talking about the text of the ability, not its name. There are many abilities with names that don't match their mechanics¹:

• The aforementioned Sneak Attack works fine while dual-wielding a flame tongue shortsword and a sun blade and screaming at the top of your lungs.
• Hunter ranger's Colossus Slayer works on creatures of any size.
• Chill touch is a ranged spell that deals necrotic damage.
• Detect evil and good is a spell that detects certain creature types, not alignment or morality.

The Sage Advice Compendium PDF covers this (p. 16)²:

I find it confusing that the mage armor spell is named that when it doesn't count as armor. Some spells and class features have figurative, not literal, names. The text of the spell or class feature explains what it does.

How does that mechanic work narratively?

The path to narrative justification should start with the assumption that the mechanic works as-written¹. Once you and your DM are on the same page mechanically, you can examine the narrative why and how. For Vow of Enmity, that may be as simple as "that hobgoblin stands in our way - threatening our mission - so I vow to eliminate it".

Reskinning is another option³. You and your DM could reword Vow of Enmity, without changing any of the mechanics:

Precise Strikes. As a bonus action, you can utter a prayer for victory against a creature you can see within 10 feet of you, using your Channel Divinity. You gain advantage on attack rolls against the creature for 1 minute or until it drops to 0 hit points or falls unconscious.

In the end, you and your DM can agree to anything. Change any feature you want - D&D has a long history of homebrew. Before you go changing things, though, I'd recommend meeting D&D 5e on its own terms - as a reasonably well-balanced, mechanics-focused system.

^{1 Comment from V2Blast
2 Comment from Ruse
3 Comment from Please stop being evil}