There is no absolute way to tell, no spell that says "he's level 10/10 HD." You have to go with context clues and observation instead. That leads us to a two-part answer.

DM Description and Observation

Observe more closely and the DM should be more forthcoming with details. In many cases, higher level NPCs/monsters look different - think World of Warcraft, you can tell a high level person because of their crazy pimp looking gear and purple energy coming off them and stuff. Maybe he has ioun stones whizzing about his head, etc. Judicious use of detect magic, detect evil, etc. can tell you strength of auras. A DM should also think through setting up context clues per How can DMs effectively telegraph specific dangers in D&D? - like maybe someone powerful comes to mess with him and gets disintegrated while he's chatting amiably with the PCs.

Many high level characters are obvious - a gnome in a robe of the archmagi riding a dinosaur with all kinds of magical protection glimmering around him is obviously bad news. If all your gear is platinum and adamantite it means you're bad ass enough to keep a hold of it from random bandits and/or adventurers. Your DM should be conveying this detail, as long as you pause to take a look.

In some cases (monsters, undead) skill use may let you get a better read on what it is you're facing. In 3.X this was sadly routine (I will roll and you will tell me exactly what this is), in 5e this isn't as guaranteed a method of knowing exactly what's up and there's more DM discretion, but at least asking for a roll and getting hints might help.

It definitely sounds like your DM needs to be thinking through this more and giving more context. "Hey this dead guy looks a lot like that statue of Archmage Xulibraz back at the Acadamae..."

Play Smart

In some cases, a high level person isn't going to look/detect different from a low level person. This should be more rare unless your DM is a dink, though there are exceptions. A high level monk in rags looks like anyone else and a king might have super pimp gear despite being level 1 just because of daddy's money. Now, if you see the monk fight it should be like watching Jet Li and you should get the clear impression he could snap your neck and isn't some random yokel. But this is where the onus shifts to you. If you decide to just roll up on someone without knowing much about them, there's a good chance you're going to get owned one day. You have to be smart.

Observe folks. Ideally, use time prior to the actual encounter. If you know about them way ahead of time then there's divination spells or sage research or "asking around on the street." See if you can observe them fight someone or cast a spell or use a skill to where you would see "holy crap they are about 500% better at that than I am." Have a backup plan. If you just kick down doors and attack people, 1 in 10 times you're going to bite off more than you can chew and you'll need a fallback. That floating thing could be a beholder or a gas spore. That undead could be a zombie or a lich. Knowing that there is that range of foes, it's up to you to decide whether you want to be all reckless and maybe die one day or whether you want to be super cautious with everything or where you want to fall in between - just like real life.

Conclusion

You have just learned a valuable lesson, and learned it without dying! Congratulations. The world is dangerous and it's not always neatly labeled or level appropriate. Let this inspire you to think about how to suss out your foes both from afar and when in direct contact with them. It'll keep you alive more than a fistful of plusses.

The rules as written for this, as taken from the DM DnD Basic Rules version 0.1 say:

Typically, XP is awarded for defeating the monster, although the DM may also award XP for neutralizing the threat posed by the monster in some other manner.

It doesn't specify how much of the XP you should award, so it is reasonable to interpret it as meaning you may award any amount of XP you like, ranging from none at all to more than if they had fought them and won.

You could base your decision on the difficulty in avoiding them, or change the amount based on the behaviour you want to encourage (e.g. full XP if you like them finding ways around combat).