The roll and the appropriate modifier are added together to get total damage.
When attacking with a weapon, you add your ability modifier-the same modifier used for the attack roll-to the damage.
Let's say for example, a level 1 rogue with a +1 DEX mod attacks with a dagger(1d4) which does piercing damage. He would add the 1d4 + 1, but he also has Sneak Attack.
If a class feature increases the damage of an attack but doesn't specify a damage type, it is included in the total damage of the attack.
So, total piercing damage would come out to be 1d4 + 1(DEX) + 1d6(Sneak Attack) and if a creature had resistance/vulnerability to piercing damage that total would be halved/doubled respectively.
But if a class feature does specify a different damage type, you split the damage types and apply resistance/vulnerability separately.
Let's say we have a paladin with a +1 STR using a flail(1d8) with Improved Divine Smite(1d8).
Total bludgeoning damage would come out to 1d8 + 1(STR).
Total radiant damage would come out to 1d8.
If the target has resistance/vulnerability to either damage type, the damage of that type is halved/doubled independently of the other.
PHB p.197
Resistance and then vulnerability are applied after all other modifiers to damage. For example, a creature has resistance to bludgeoning damage and is hit by an attack that deals 25 bludgeoning damage. The creature is also within a magical aura that reduces all damage by 5. The 25 damage is first reduced by 5 and then halved, so the creature takes 10 damage.
You take off the damage reduction and then halve the damage. Option 1 in your question.
Best Answer
You always round down unless told otherwise.
In the very beginning of the Player's Handbook it lays down some ground rules for everything that follows. Along with the "how dice are used" and "specific overrides general" stuff, it includes a subsection entitled "Round Down" (p. 7):
So half of 11 damage would be 5, and half of 1 damage would be none.