The guidelines for making your own monsters are based around the Monster Statistics by Challenge Rating table on p274. Inspired by this question regarding Animated Armor (MM 19), which is CR1, let's see where the Armor would be if we made it from scratch…
Defensive CR: The Armor has 33 HP, but step 9 (Damage Vulnerabilities, Resistances and Immunities, DMG 277) has us adjust effective HP based on resistance.
So we start with 33 HP, giving us CR 1/8, which tells us that we're using the first line of Effective Hit Points Based on Resistances and Immunities, for an x2 multiplier for each Resistance or Immunity.
The Armor has two Immunities, so has 33x2x2 or 132 "effective" hit points. That puts us at CR 5. We then look at its AC of 18, which is 3 higher than the normal 15 for CR 5. That adjusts us up by one, to CR 6.
Offensive CR: It does 10 (2×5) points of damage, for a CR of 1. Its Attack Bonus is +4, which is 1 higher than the normal +3, but doesn't adjust CR up.
Final CR is the average of the two, or (6+1)/2. That gives us 3.5, which we round against the players, for a final final CR 3.
How is Animated Armor a CR1 challenge? Its only notable disadvantage (from a combat perspective) is its Antimagic Susceptibility, but that has no effect on CR, per DMG 280. What's knocking it down from CR 3 to CR 1? Or am I horribly misunderstanding how the creation rules work?
Best Answer
Short answer: No.
Longer answer:
Paul Hughes, of blogofholding, has written a series of blog posts investigating this particular question. His approach was one of doing a statistical analysis of all the monsters in the MM (and some other books). He arrived at several significant findings:
Paul also references an interview by Mike Shea (aka Sly Flourish) of D&D lead rules designer Jeremy Crawford about monster design:
So, what are the more accurate rules?
Paul has summarised his findings in a reverse-engineered set of 5e monster creation rules that more closely mirrors the results apparent in the MM et al. He has both a one-page set, and a two-sides-of-a-business-card summary.