Smoke is normal in an electric oven, but flames are definitely not.
In order to start a fire, you either need a spark, or you need to heat something beyond its autoignition temperature (AKA kindling point). You might have had a short - or you might actually be using a gas oven with spark ignition - but I'm guessing your issue was the latter.
Cooking oil or grease being heated beyond its autoignition point is one of the most common causes of kitchen fires (grease fires). Supposedly, some oils have autoignition points as low as 550° F (or 288° C), though I'm not sure which oils those are. Olive oil would be my guess as the lowest, but pepperoni grease could very well have ignited at self-cleaning temperatures (which, as you noticed, go up to nearly 1000° F).
Fortunately for you, all modern ovens have a mechanical interlock which prevents them from being opened during a self-cleaning cycle. If you'd opened it, you would have made the problem a lot worse by (a) supplying the fire with abundant oxygen, and (b) drawing all the hot air and flames out of the oven and into your kitchen, quite possibly setting your whole home on fire. Heat wants to move to where it's cold; that's why you keep your doors and windows closed in the winter.
There are a multitude of oven cleaners available for self-cleaning ovens - you are supposed to use these before you run a self-cleaning cycle. Yes, I know it's odd, but "self-cleaning" doesn't really actually mean that it cleans itself, it just gives you a little extra help. You need to try to clear out all the grease and big chunks of food first using one of these cleaners, then run the self-cleaning cycle to deal with anything you might have missed.
Coated (e. g. enamel, PTFE, ceramic)
I can't answer in general, but that one's easy. Sudden thermal shock causes strain in a material by unequal expansion, either in the same material by high thermal gradients, or in interfaces between materials with different coefficients of thermal expansion. The strain in this case (two different materials) can be very high. If the material in question is not elastic (e.g. enamel + ceramic; I would think PTFE is different, but I'm not sure), then the bonds between the coating and the metal would be severely strained and it would likely crack and chip.
I can tell you from personal experience that I have actually used this to my advantage:
In the spring, I produce a small quantity of maple syrup by boiling sap in an uncoated stainless steel pan. On rare occasions, accompanied by the release of many expletives, I have let the syrup boil down too far, at which point it burns and seems to coat the bottom of the pan with a thin but hard and very resilient layer of carbon black. The trick to removing this stuff is to get some kind of stress crack started, e.g. by scrubbing w/ steel wool or a copper pad, and then what I do is I put the pan on the stove for a while to let it heat up hot (but not red hot), and then bring it over to the sink and spray cold water on the inside pan bottom where the carbon black has stuck to. After a few times, the carbon black will start to flake off and then it becomes easier to remove by a combination of abrasion and thermal shock. (The two pans I've done this on have been fine; both are stainless steel with a thick (>8mm) bottom, and I've put them through at least 30 or 40 thermal cycles of this type.)
edit re: general topic:
Wikipedia says this:
The robustness of a material to thermal shock is characterized with the thermal shock parameter:
where
- k is thermal conductivity,
- σT is maximal tension the material can resist,
- α is the thermal expansion coefficient
- E is the Young's modulus, and
- ν is the Poisson ratio.
Higher thermal conductivity means it's more difficult to get a large thermal gradient across the material (less prone to shock); higher thermal expansion means more strain (more prone to shock), and higher Young's modulus means more stress for a given strain (more prone to shock).
So theoretically you could compare the different materials. (exercise for the reader ;) Most likely copper would be more resilient than the other metals, because of its higher thermal conductivity and higher ductility.
Thermal conductivity k: Copper = 401, Aluminum alloys = 120-180, stainless steel = 12-45 (units = W/m*K)
σT: no idea:
Coefficient of thermal expansion α: Copper = 17, Aluminum = 23, iron = 11.1, stainless steel = 17.3 (units = 10−6/°C)
Young's modulus E: Copper = 117, Aluminum = 69, iron/steel = around 200 (units = GPa)
Poisson's ratio ν: Copper/stainless steel/aluminum are all around 0.3-0.33, cast iron = 0.21-0.26
So stainless steel is worse than aluminum or copper (much lower thermal conductivity, higher Young's modulus).
Best Answer
They do go above 550F, it's a question of design and the type of wiring used in an oven. THWN is commonly used to wire up home ovens, which has a maximum surrounding ambient temperature rating of 105 degrees celsius.
So, oven makers need to design ovens where the ambient temperature of things outside of the insulated oven cavity don't reach a temperature above 220 degrees fahrenheit (that's where the wiring lives), and not exceed the insulation around the door of the oven, and not burn anyone coming in contact with the sides.
So it's both safety and technical cost reasons. Hotter home ovens are possible, and available, but they're made a bit differently and the cost to produce them is a bit higher. Since most people cooking at home don't need something that goes north of 500 degrees, there isn't a big market for them - and the market that does exist is mostly in the commercial space.
A residential implementation would have some pretty specific insulation requirements, be slightly larger than most home builders anticipated (making them harder to install) and naturally quite a bit more expensive and scarce.