RAW - No, Otto's Irresistible Dance is not a Charm spell

It does not say that the target will be "charmed", and the condition it imposes is significantly different than the description of the "Charmed" condition.

OID:

A dancing creature must use all its movement to dance without leaving its space and has disadvantage on Dexterity saving throws and attack rolls. While the target is affected by this spell, other creatures have advantage on attack rolls against it.

Charmed:

A charmed creature can't attack the charmer or target the charmer with harmful abilities or magical effects. The charmer has advantage on any ability check to interact socially with the creature.

A dancing creature can attack the caster (with disadvantage -- and that disadvantage applies to any attack, not just against the caster) and can cast AoE spells that include the caster and anyone else they choose in the target area with no penalty. They just can't move freely. A charmed character can move freely but can't attack or target the caster.

Furthermore, if it were a Charm spell, it would not be necessary to specify that "[c]reatures that can't be charmed are immune to this spell." That appears to be listed as a special exception.

RAI - No, Otto's Irresistible Dance does not impose the Charmed condition

Jeremy Crawford has tweeted saying:

Being charmed means being subjected to the charmed condition

Since OID does not impose the Charmed condition, the Fey Ancestry resistance to "being charmed" does not apply.

What makes sense?

Webster's dictionary says:

charm: to affect by or as if by magic : compel

Magically forcing someone to dance seems to fall completely within that definition.

I would give those with Fey Ancestry advantage on saves against all enchantment spells that compel the user to do something that is not their own free choice. This includes many spells that do not specifically say they are "charms" or that they impose the "Charmed" condition, such as command, compelled duel, and yes, Otto's irresistible dance. I believe those all fall within the common sense and dictionary definitions of "charmed" in this context, and there's nothing in the description of Fey Ancestry that says it only applies to things that grant the "Charmed" condition.

It's every creature...

Area of Effect spells and abilities, unless specified otherwise, hit everyone in their area. The same wording "each creature in a [certain] foot radius" is present in Fireball. By the logic of your fellow player, you should also be able to decide which creatures are affected by fireball. But that is not the case.

For reference, a spell that does let you choose creatures inside of an area is Slow, which has a 20 foot cube and allows you to choose 6 creatures. So, because the phrase "creatures you choose" is not present in the description of Fey Presence, it affects everyone in the specified area.

The use of the word can is not relevant here, since it is stating that you can use an action to use this ability, not that you can or cannot affect certain creatures.

But...

you can choose whether they are charmed or frightened by you, and a player charmed by another player in combat may not behave any differently than before, although they might not particularly care to be charmed by you. It only lasts until the end of your next turn.

Caveat

There is no hard and fast rule in the PHB that says all AoE spells behave thusly. However, the description of each spell will tell you whether or not you can pick and choose targets. If you can't target creatures individually in the area, it always says "each creature", as opposed to "each creature you choose" or "up to X creatures you choose". You need to look at each spell individually to see how it behaves.