Getting the Demon Spirit Out
The rules don't really specify what happens when a demonic spirit is exorcised, so we're going to be using the assumption that the spirit is still around once it leaves the body. If you rule differently, you won't need the latter half of this answer.
Dispel Evil and Good explicitly mentions ending possession of a creature. This will probably be the easiest way to get the demon out.
Break Enchantment: As your action, you touch a creature you can reach that is charmed, frightened, or possessed by a celestial, an elemental, a fey, a fiend, or an undead. The creature you touch is no longer charmed, frightened, or possessed by such creatures. (PHB, 233-234)
What is told about demonic possession from the Monster Manual is that demons can possess creatures when their essence emerges from the container that holds it, which implies the demon was bound to an object. Essence here implies more of a spirit or soul, rather than tangible body. You can read about binding demons and possession in the Monster Manual, page 51, under the headers Bound Demons and Demonic Possession. This also suggests that demons can't normally possess other creatures if they have their own tangible body. There's nothing to suggest their spirit can move from body to body willingly, but there's nothing suggesting they can't possess another creature after their spirit has forcefully been removed from the previous host either.
What this likely means is that exercising the demon will probably result in it attempting to possess another creature.
You have two options:
Sending the Demon Spirit Back to the Abyss
With this option, you're looking to send the demon spirit away. If the DM allows it, the spirit should be able to be targeted by spells. There's no real rules for a spirit, so you'll need to decide if this is possible. If it is you can use a simple spell like Banishment to send the spirit back. Another, more complicated way is to cast True Resurrection on the spirit (you'll need to know his true name) and then fight its new body to send it back. You may also use that as a bargaining chip, to perhaps make a deal with the demon, if that is even possible.
You might also be able to remove the demon by finding an amulet it has put part of its essence into. Destroying the amulet will trap the demon in the Abyss for a year and a day (MM, pg 51, Protected Essence). This would probably end possession and send it back all in one go, depending on how you rule it.
Binding the Spirit to an Object
You also have the option to bind the Demon to an object, which is likely how it came to possess something in the first place.
The Monster Manual suggests that The Book of Vile Darkness, the Black Scrolls of Ahm, and the Demonomicon of Iggwilv are the foremost authorities on this subject. The Book of Vile Darkness is detailed in the Dungeon Master's Guide on page 222, in the Artifacts section. Suffice it to say that binding the demonic essence or sending it back to the Abyss is something you're going to have to use your DM powers for. Keep in mind you'll most likely need an evil character to use the book. The Book of Exalted Deeds is basically the polar opposite of this and might also help, altough it doesn't detail any control over fiends and the like, but it does help you fight them and can even destroy an evil creature that tries to read it (if a demon, this would likely send it back to the Abyss).
Either Way...
It would be smart to cast Protection from Evil and Good on your party and any participants in the exorcism, as it protects you from possession. It can also be used to exorcise the demon from the host, but only if there is a relevant save (there isn't one listed for demonic possession).
I'd also suggest a good reason for the possession. The MM suggests that demons that escape their bindings seek revenge on the one who bound them, which could be the noble or someone close to him, depending on how powerful the demon is. Greater demons are usually smarter than lesser demons - I mean that to say that a greater demon would likely plot and scheme through possession, where a lesser demon would likely just rage around and create chaos.
But then I went to read the rules because our DM had some difficulty figuring out when it took its turn in combat and stuff
RAW - On the Ghost's turn. Doing the turn on the possessed character's turn could grant the ghost an unintended extra turn in the action economy. Moreover, there is no reason to think that the possession would change the initiative order.
Since a ghost doesn't gain access to the creature's class features does that mean also no spellcasting? To me it seems like spellcasting is a class feature, just like danger sense is a class feature to a Barbarian, am I correct?
Any spell granted by class is a class feature. Spells that are granted by race are racial abilities. So using a spell slot is class feature, and the Ghost, RAW, can't do it.
That said, it is the DM's game, and he makes the rules for the table -- he can houserule otherwise.
Best Answer
The DMG lists the various ways to get a ghost out of a person. It's pretty much exhaustive. So, here we go. The possession lasts until:
The body drops to zero hit points Any party can handle beating someone up. Healing potion to bring the victim back is optional.
The ghost ends it as a bonus action This requires voluntary action on the part of the ghost. Options include negotiating with it to leave, or tricking it into leaving (possibly by offering a better host).
The ghost is turned or forced out Explicitly called out here are Dispel Good And Evil as well as Turn Undead. The best option is, of course, to have a character who can cast/use that in the party. One alternative to that is recruiting/renting such a person. The other seems to be finding a magic item that can cast the spell.