It behaves like any other creature would.
The PB states that any creature that moves out of your reach provokes an opportunity attack (195):
You can make an opportunity attack when a hostile creature that you can see moves out of your reach.
If the specter moves through a creature and out of their reach, then it will provoke an opportunity attack. If it passes through a creature but stays within its reach, it does not provoke an attack. This is, of course, the same for any other creature.
If a creature can avoid opportunity attacks, it's explicitly listed in the listing. For example, the Giant Owl (MM 327):
Flyby. The owl doesn't provoke opportunity attacks when it flies out of an enemy's reach.
Specters do not have such an ability, so they are subject to opportunity attacks if they move out of a creature's reach.
Compare to a ghost
It's worth noting that a ghost has a similar incorporeal movement ability and an Etherealness ability, which truly allows it to float through anything freely (MM 147):
Etherealness. The ghost enters the Ethereal Plane from the Material Plane, or vice versa. It is visible on the Material Plane while it is in the Border Ethereal, and vice versa, yet it can't affect or be affected by anything on the other plane.
This ability would allow the ghost to ignore any opportunity attacks (though it would still technically trigger them).
Yes, it counts as magical.
You already quote arguably the most specific reference - the definition of the force damage type:
Force is pure magical energy focused into a damaging form.
All force damage is magical. Unless contradicted more specifically in a feature although I doubt this will occur. It is far more likely that they would define an additional damage type or errata the existing definition.
It is also the only damage type that is defined explicitly as magical. One example would be that a dragon's breath is fire damage but not magical; however, the type of damage from the spell fireball is fire and is magical.
Sage Advice further clarifies how to determine what in the game is considered magical under the question "Is the breath weapon of a dragon magical?"
Is it a magic item?
Is it a spell? Or does it let you create the effects of a spell that’s mentioned in its description?
Is it a spell attack?
Is it fueled by the use of spell slots?
Does its description say it’s magical?
The last one being the most pertinent in the reference to the original question.
Best Answer
Yes for Objects, No for Creatures
Note that the Specter's Incorporeal Movement feature does not specify that they take damage if they end their turn inside a creature, only an object.
As for dealing with Objects, yes...a Specter may choose to take damage in order to stay inside of an object and yes, that will give them cover based on how much of them fits inside of it. If they are parked stuck through a table...that's probably half cover. If they are completely inside a wall, that's Total Cover.
As for dealing with Creatures...
The Specter's Incorporeal Movement feature has nothing to say about ending its turn inside a creature. The standard rules for this (on PHB 191) say...
As always, specific rules beat general rules, but in this case...there are no specific rules overriding that final statement.
Thus, in the case of dealing with another Creature...a Specter remains bound by the same rules as everyone else because its Feature doesn't say otherwise. The same way that a Halfling can move through the space of a creature that is only one size larger than it, a Specter can move through another creature's space regardless of size. But neither of them can willingly end their turn in another creature's space.