The goal of this question is to compare both Ranger level 11 features Whirlwind Attack & Volley and examine their comparative strengths and weaknesses for Rangers.
Whirlwind Attack
You can use your action to make a
melee attack against any number of creatures within 5
feet of you, with a separate attack roll for each target.
Volley
You can use your action to make a ranged
attack against any number of creatures within 10 feet
of a point you can see within your weapon’s range. You
must have ammunition for each target, as normal, and
you make a separate attack roll for each target.
At level 11 you would have two attacks available during a normal action so you would need to have at least three enemies within 5 feet of you to make Whirlwind Attack viable from a pure damage output standpoint. The way I see it, you need to have good positioning, a number of enemies clustered, and be willing to sacrifice the utility of focused fire. I have trouble seeing this as being more useful than Volley.
Is there something that makes this choice mechanically competitive besides being in a campaign where you are frequently engaging hoards of swarming melee combatants?
As a rough surface calculation I am looking at volley as ranged AoE which can effectively hit 16 squares, while Whirlwind is a melee attack that can hit 8. The only direct advantage's to melee attacks that a melee focused ranger could have is a +2 to damage with every attack, or an additional melee attack as a bonus action. So if Whirlwind attacks and hits all 8 squares it can do 16 extra damage on a melee focused build. Or if we went with two weapon fighting one extra attack that would probably max out at slightly less than 16.
At a bare minimum if I could add one single target with volley to an ideal whirlwind I would getting almost the same exact damage spread.
If I were already surrounded by 8 opponents obviously WW is better, but chances are I don't actually want that to happen in the first place.
Best Answer
I run a group with several melee combatants, including a melee ranger. We have not gotten to L11, but I can tell you from my experience that an ability that would allow you to attack more than 2 enemies is incredibly useful.
Here's the thing, as you level up, you're supposed to not only face greater threat level foes, but you're going to be facing more low level foes at once. Adding powers to a power list that allow a ranger to effectively operate as crowd control, especially when they are dealing more damage and can knock a low level foe out completely, is a great power.
Think about this for a moment, let's take the lowly goblin a CR 1/4 foe. Using a party of 4 characters, at L1, 4 goblins are a deadly encounter (50xp *2). At L5 it takes 27 of them to be a deadly foe, and by L11 you need 72 of them to be a deadly encounter.
While no, you're not likely to face off 72 goblins at L11, you may face off against 10 CR 2 opponents in a deadly encounter. This means there is likely to be a situation where you will have 3+ foes around you if you are engaging in melee combat.
From experience, I can tell you that this is a frequent occurrence for melee characters, they are often swarmed by 3, 4 or even more low level foes and potentially need to be able to deal damage to all of them, especially if they are much lower level and can be killed in one shot.
As far as whether it's a mechanically sound choice over Volley, that likely depends on both your character and your campaign. If your character is never going to use a ranged AOE (his ranged attack isn't good or he is a front line fighter), that's not going to be a good choice for him. However, if your character is a ranged ranger, Volley is a no brainer over this.