[RPG] What are the odds of rolling high all the time

dicestatistics

So, I have a suspicion that a player in an online game is… being creative in the way they call their dice rolls. I play with them in several games, and in 2 games I have convinced the DM to enforce online rolling, but in the 3rd game that is unlikely to happen.

Today I noted all their D20 dice rolls:

18, 13, 20, 8, 15, 11, 19, 7, 2, 15, 20, 8, 17, 18, 17, 11, 16

So of 17 rolls:

  1. 10 were 15 or above

  2. Only 4 were below 10 (to make it even worse all the low rolls just so happened to be when the player knew a low roll would do, such as hitting a zombie with a +11 bonus, and the higher rolls on checks they cared about where the DC was uncertain, a common pattern)

What are the odds of this happening?

Note: I am not interested in ways to attempt to stop what I think is cheating. At some point I will just talk to the DM in question, but this particular distribution seems even worse than I expected.

Best Answer

This may not be as unlikely as you think

Preamble: Probability is Hard

So distributions like this are tricky, because a lot of people assume, wrongly, that they only need to calculate the odds of getting a distribution this good (or better), and then presume, based on the improbability of those odds, that it's evidence of tampering/cheating. But that's not quite right, and to demonstrate, I'm going to borrow an example Matt Parker used when he was assessing the odds that a speedrunner cheated an RNG mechanic in a video game.

In this example, we consider an experiment where someone flips a [presumed to be fair] coin 100 times. Then, after the experiment is concluded, a third party looks at the results, and notes that at one point during the experiment, there was a run of 12 flips which resulted in 10 Tails results and 2 Heads results. They then note that the odds of getting at least this many Tails results in a run of 12 flips is only about 1.9%, and conclude that this is improbable with a fair coin; therefore, they conclude, the person flipping a coin must have cheated, either by using an unfair coin, or by using some kind of technique to bias the results.

However, as Matt goes on to point out, you can't simply consider the odds that a run of 12 flips results in 10 (or more) Tails; you have to also consider the odds that, over the course of the entire experiment, you could get a run of 12 flips with an outcome this extreme. And as it turns out, those odds are actually about 88%. In other words, it's actually very likely, given enough trials of flipping coins, to get an individual run that's relatively improbable on its own.

So, in your case, the question we need to solve is not "how unlikely is it to get a run of d20 rolls this lucky in the course of a single night?", but rather "over the course of several sessions of a game, how unlikely is it for someone to get a run of d20 rolls that was at least this lucky?"

Let's do some math

So in your case, you've tracked 17 rolls from this player over the course of a single night, where the results were unusually high. Below are the odds of the two facts you've chosen to note:

  • At least 10 of 17 rolls were 15 or higher: This has a probability of about 1.27%
  • At most 4 of 17 rolls were below 10: This has a probability of about 5.96%

One more fact I'm going to track:

  • Of 17 d20 rolls, the average result is 13.824 or higher: This has a probability of about 0.88%

So a brief sanity check we can perform on these odds, before we go further, is to note that none of these outcomes are terribly unlikely. The second condition happens more frequently than the odds of someone happening to roll a natural 20 on a d20 on any given roll, and we don't generally assume that any person who happens to roll a natural 20 is cheating based on that one roll. And there are a lot of improbable events that happen all the time that we generally would not think of as being evidence of cheating despite their absurdly low probability. As an example, I'll submit this combat log from a session my group had about a month ago, where on a 4d8 roll, our cleric rolled four ones, and then promptly rolled a natural 1 on her subsequent attack roll:

Image of Combat Log showing 4 1's rolled on 4d8, and then a natural 1 on an attack roll

And, just so it doesn't go unstated: I can personally verify, as the DM and maintainer of the VTT these results were obtained upon, that these were fair results, despite the fact that the odds of this happening were about 0.00122% (or, 1 in 81,920).

But, I should also not leave unstated: that was a cherrypicked result from a long series of rolls over the course of a campaign that has run almost every week over two years. You could dig into any campaign and find individual runs that were at least as unlikely, perhaps even moreso.

Now, in your case, we don't have every single d20 roll to analyze; only the 17 from the session you chose to record. So we do have to make some educated guesses about, for example, how many sessions you've been in this campaign/game with this player, and how many d20 rolls they made in those other sessions. I'm going to assume that each session you've participated in has had a similar number of rolls (so, 17). We then have to ask, given the probabilities for each of the facts we're considering, how likely they are to have occurred at least once over X sessions.

Sessions Odds of >=10 greater than 15 Odds of <=4 less than 10 Odds of average >=13.824
1 1.27% 5.96% 0.88%
2 2.52% 11.56% 1.75%
3 3.76% 16.84% 2.62%
4 4.98% 21.79% 3.47%
5 6.19% 26.45% 4.32%
6 7.38% 30.84% 5.17%
7 8.56% 34.96% 6.00%
8 9.72% 38.84% 6.83%
9 10.87% 42.48% 7.65%
10 12.00% 45.91% 8.46%
11 13.12% 49.13% 9.27%
12 14.22% 52.16% 10.06%
13 15.31% 55.02% 10.86%
14 16.38% 57.70% 11.64%
15 17.45% 60.22% 12.42%
16 18.49% 62.59% 13.19%
17 19.53% 64.82% 13.95%
18 20.55% 66.92% 14.71%
19 21.56% 68.89% 15.46%
20 22.56% 70.74% 16.20%

Now, it's important to note that not all of these numbers are simultaneously relevant. It's much more likely that one column is the most relevant, depending on how exactly you think this player is cheating (i.e. are they fudging die rolls higher? Are they making up numbers and just happening to choose them to be high? Are they making up numbers, but only when failure would be really bad for them?).

What's relevant for our purposes is that if you've played only 6 sessions with this player, then regardless of which properties we think are relevant, the player has at least a 5% chance of achieving those results at least once off a fair set of dice. If you've played more sessions with them, those odds get a lot better. A 5% chance is low; but again, 5% chances happen all the time.

And this assumes that we only care about the average result, i.e. we assume their method of cheating is that the player has been systematically nudging their dice results upwards. If we think their method of cheating has been to (secretly) reroll low results, then the results they achieved actually have a 30% chance of happening legitimately.

Conclusion: The results you sampled do not prove cheating

To be clear, they don't prove innocence either. A 5% chance is pretty low, and if the player is cheating, these results would be consistent with what you'd expect given a player who either systematically fudges die results, or who just makes up numbers and biases them to be high enough to succeed.

But what I would say is that, if you plan to accuse a player of cheating in this game, I think this data would not be good evidence to support it. The odds that they're playing fairly are just too high. At best, it suggests a need to monitor their results and see if they continue to get lucky or if this was just a hot streak, and I think that to say the player is definitively cheating, they'd have to have odds lower than what's being shown here. If you collect more of their rolls, you'd be able to run a similar analysis with both sessions' data and narrow things down a bit, which might get closer to proving that they are or aren't cheating.

Related Topic