When you Wild Shape/Polymorph you "assume the beast’s hit points" thus essentially creating a new, separate pool of HP from your own, original form similar to how Temporary Hit Points work, as Alexis Wilke has stated.
Damage taken in animal form doesn't affect your original form's HP unless you're dropped to 0 HP in animal form and there's excess damage. Nowhere is it suggested that max-HP reduce would work any differently. Because Wild Shape/Polymorph gives you a new pool of HP (as supported by Jeremy Crawford in the link below), only that pool is affected by the reduction.
So, using your example, if a PC has 30 HP in their original form and transforms into a beast that has 50 HP, the PC effectively has 50 HP. If the PC has their max HP reduced by 30 while transformed then they don't die as a result of having 0 HP because they're using the beast's HP and they still have 20 HP left in that pool.
As for whether the max HP reduction carries over to your original form when you revert, according to Jeremy Crawford, lead rules designer, the answer is no:
Jonathan Longstaff
@pukunui81
@JeremyECrawford What happens when a wildshaped druid that has had its HP max reduced reverts back to normal? Does the reduction carry over?
Jeremy Crawford
@JeremyECrawford
Wild Shape—a reduction to hp maximum doesn't carry over from your beast form to your true form or vice versa.
By RAW, the Druid dies.
Wild Shape protects you from a few specific things:
When you revert to your normal form, you revert to the number of hit points you had before you transformed (and then take overflow damage).
If you revert and have hitpoints remaining in caster form, you don't fall unconscious as a result of being reduced to 0 hit points.
Power Word Kill bypasses both of these restrictions by killing the Druid rather than dealing damage or reducing its hit points to zero. Returning the Druid to life (or preventing its death) is not one of the things that Wild Shape does.
This is basically the same argument as the one for massive damage, except that there's definitely no sequencing of events. The Druid dies, and Wild Shape doesn't protect you from dying. Game over.
House ruling options are slim here. Killing a druid for using a utility form is harsh. On the other hand, relegating a 9th level spell to "knock a druid out of wildshape once" is also pretty rough.
Does the druids combined HP matter or just the wild-shapes HP?
Just the Wild Shape's. Wild Shape says:
When you transform, you assume the beast's hit point and Hit Dice.
Like most of your other stats, your old hit point total "goes away" while you're in beast form. Your hit points are the beast's hit points.
With that said, adding beast hit points to caster hit points, or using the Druid's base hit points for spells such as Power Word Kill may be a useful house rule.
This ruling is confirmed in the Sage Advice Compendium, the source of official rulings from WotC:
What happens if I’m polymorphed or Wild Shaped into a creature with fewer than 100 hit points and then I’m targeted by power word kill?
You die.
Jeremy Crawford
This has been confirmed by Jeremy Crawford:
If a druid wildshapes into a wolf and is then targeted with power word kill does the druid revert dead or alive?
If you have 100 hp or fewer, power word kill causes you to die. Notice that it doesn't say you drop to 0 hp.
So what is PHB pg. 66 "You automatically revert if.....drop to 0hp, *or Die* ." telling us? Form dies, Druid reverts, yes?
Beast form ends if the druid dies; things like power word kill can end you without reducing hit points.
Best Answer
The druid dies, then reverts to their normal form
Cutting off the head is not an effect that reduces hit points. Hit points are an abstract quantity (PHB p 196):
When decapitated, you do not die due to loss of hit points, you die due to losing your head.
The reasons you revert automatically are:
Since the beast is not losing hit points or falling unconscious, it would not revert unless it died.
Either the beast can live without a head, in which case the vorpal blade does not kill it, or it cannot, in which case it will die and then revert.
Reverting to druid form here is not due to losing hp, it is because you already died.
Currently none of the creatures in the rules that are typed beast can survive without a head. This means the druid will die, and then revert to their normal form.
The situation is the same as for Power Word Kill, on which the Sage Advice Compendium ruled: