It is hostile, but that will not affect charm monster until you lose control
My other answer makes the assumption that hostile means the same thing as fighting against you, but this isn't necessarily true. Taking from this answer to a different question about what hostile means, the DMG (pg. 244) describes the following:
- A friendly creature wants to help the adventurers and wishes for them to succeed.
- An indifferent creature might help or hinder the party, depending on what the creature sees as most beneficial.
- A hostile creature opposes the adventurers and their goals but doesn't necessarily attack them on sight.
Since a hostile creature "doesn't necessarily attack them on sight", this means that a demon doesn't have to be fighting the party to be considered hostile, and clearly demons are hostile to everything with regards to opposing everyone's goals.
Despite being hostile, the demon isn't fighting against you initially because the summon greater demon spells allows you to maintain control over the demon until it breaks free, after which point it attacks any non-demon, including you, the caster.
Whilst it isn't free of your control, it will not fight you unless you fight it first, so it will not give the demon advantage against charm monster for that reason, since that spell specifically says "fighting".
Charm monster (Xanathar's Guide to Everything, pg. 151) says:
A creature you can see must make a Wisdom save, and has advantage if you or your companions are fighting it. If it fails, it's charmed by you until the spell ends or until you or your companions harm it. The charmed creature is friendly to you. When the spell ends, the creature knows it was charmed by you.
Also note that, as described in this question, some demons have Magic Resistance, meaning those demons will have advantage against charm monster anyway, regardless of whether you're fighting it or not, so although it's hostile, and you may or may not be fighting it, it'll still have advantage on the saving throw.
Also note that having used its true name as part of the summoning spell only affects the summoning spell, so it'll always have advantage against charm monster, being a different spell.
Best Answer
You could get 5 additional rounds.
tl;dr The unit of concentration is a round. Breaking concentration in the final round gets an uncontrolled demon for up to 6 rounds including the one in which concentration was broken.
Breaking concentration immediately results in an uncontrolled demon.
The spell description details what happens when the demon is no longer controlled:
The character concentrating chooses to break concentration on their turn during the last round of the spell's normal duration. If the demon's initiative is after the character, its is going to attack. If the demon's initiative was prior, it's already used it's turn. Either way the current round it is now uncontrolled.
Additional rounds.
The special case of ceasing concentration does add additional rounds:
Rolling a 6 when you cease concentration means that it does not disappear for six rounds. Since the round you cease concentration is the first round it is uncontrolled, this could net 5 additional rounds it remains beyond the duration of the spell.