No, you can't take the -5/+10 while throwing a dagger.

Thrown: If a weapon has the thrown property, you can throw the weapon to make a ranged attack.

Throwing a dagger is a ranged weapon attack, so the first two benefits of the Sharpshooter feat will apply. However, the third benefit doesn't require a ranged weapon attack:

• Before you make an attack with a ranged weapon that you are proficient with, you can choose to take a -5 penalty to the attack roll. If the attack hits, you add +10 to the attack’s damage.

It specifically requires that you attack with a ranged weapon. The thrown property doesn't change the fact that a dagger is a melee weapon. Throwing a dart, for example, would benefit from the -5/+10, because it's a ranged weapon. The thrown property isn't what's important here - what's important is whether the weapon is a ranged weapon or not.

If you prefer to hear it from the designers, Jeremy Crafowrd agrees that the first two benefits of Sharpshooter work with daggers, but not the 3rd. He also stated that a dagger is a melee weapon, even when you throw it, as well as including it in the official Sage Advice Compendium:

Does the Archery fighting style work with a melee weapon that you throw? No, the Archery feature benefits ranged weapons. A melee weapon, such as a dagger or handaxe, is still a melee weapon when you make a ranged attack with it.

It effectively removes advantage for the mid-range of a d20.

It's not that you're removing Advantage, but that the advantage is about +5 and is cancelled out by the -5 from GWM/SS.

There are two bases for this:

1. Advantage often acts as a +5

2. +5 is used when calculating passive advantage.

Advantage itself remains

Note that the math is the only thing that applies here with regard to +/-5. You still actually have advantage with all the rights and privileges granted by it (including a better chance to crit.)