It's not just in extreme cases that this doesn't add up, though. Let's say that you hit a target on a 16+. If you get a +1, you now hit them on a 17+, which is a 6.25% increase in your chance of hitting, and a 25% reduction in your chance of missing.
People are using "increased by a percent" sloppily — or, I guess, if you sigh and admit that language represents how people use it, not logic, people are "using 'percent increase' in a non-technical sense".
More formally, what people mean when they say "increased by 5%" is increased by 5 percentage points. Percentage points are units used to describe the arithmetic difference between two percentages of the same thing. This resolves the ambiguity we run into otherwise:
If there are 4 million voters in Senator Grayton's state, his initial polling at 20% put him at 800,000 supporters. If this "plunged by 19%", that would be a decrease of 152,000, down to 648,000. Of course, given the described circumstances, support presumably actually plunged by 19 percentage points, from 20% to 1%, leaving 40,000 oddly dedicated supporters.
So, it sounds kind of pedantic, but it's one of those areas where when we are talking about precise things like game rules, it'd probably be better for everyone to be precise. On the other hand, the more loose language is widespread, and I think it's a losing battle to try to correct it everywhere. Instead, just be aware of the ambiguity — it's usually clear what is actually meant.
Anyway, after all of that, the "+1 = 5 percentage points" statement also assumes that the to-hit chance is somewhere near the middle of the range, not only-hit-on-20 or only-miss-on-1. Especially with 5E, that's a fair assumption for most comparisons of choices, due to the bounded accuracy design principle.
Thinking about +1s on a d20 in terms of +5 percentage points is useful because it's easy to go from there to expected damage (or similar). That is, if a successful attack does 10 points of damage, expected damage with a 50% chance to hit is 5; or with a 55% chance, 5.5. That change in damage (in this case .5) is the same even if the initial to-hit is 40% or 60%. And the relative change is the same even if the attack damage is 5 or 50. Of course, advantage, disadvantage, critical hits, and auto-fails complicate this, but in the middle of the range, thinking in terms of the percentage-point change is simple.
Using research done by platypusbill on ENWorld, namely their spreadsheet, I calculated the average saving throw modifier for all creatures below Challenge Rating 1:
- Strength: +0.503
- Dexterity: +1.288
- Constitution: +0.806
- Intelligence: -2.266
- Wisdom: +0.065
- Charisma: -1.619
As a spellcaster, most spells ask for a Dexterity, Constitution, or Wisdom saving throws, so if you had to choose between the three most common saving throws, your best realistic bet for a creature to fail a saving throw would be Wisdom. If you're considering all saving throws despite their rarity, then the most likely to fail is Intelligence.
Best Answer
Finding out the specific probability between critical miss, miss, hit, and critical hit gets a lot more complicated, but a "hit and miss" probability can be shown with a simple
output d20 + 2
in AnyDiceUsing the table looking at "At least", you can see your percent chance to hit any AC, knowing that you always have a 5% chance to critically fail and a 5% chance to critically hit. In this case, you have a 40% chance to meet a 15 AC
Here is a small AnyDice program that you can use to calculate hit, miss, critical hit, and critical miss, if you want to be more specific with your rolls. This is easily modified to allow crits on 19s and to eliminate critical misses as well. Be aware that it does not output your rolls, only numbers to represent the result. You also need to input an AC to roll against, since this program doesn't output a probability curve.
0 = critical miss.
1 = miss.
2 = hit.
3 = critical hit.
The question Incorporating expanded crit-range into anydice? would also be helpful in learning how Anydice can be used and what it can be used for.