Nowadays, foodborne botulism seems to be quite rare, even from potentially high-risk things like home canned goods. (The CDC Botulism Surveillance reports indicate on the scale of tens of confirmed cases per year in the US.)
But I can only assume that the strict guidelines we have today were motivated by a significantly larger issue in the past. What's the history here? When did those guidelines come about, and how big a deal was botulism before then?
This isn't a pure historical curiosity question, of course. Sometimes the low prevalence of botulism is used as evidence of low risk and thus low importance of following food safety rules, but it's unclear what the actual level of risk would be without most people doing things "right" already.
Best Answer
Summary:
It's impossible to give a good statistical answer to this question, since historically botulism was associated with only certain foods, and diagnosis was mostly based on symptoms occurring after consumption of those foods. Thus, old statistics include a small subset of actual cases. Actual medical testing for botulism in an ambiguous case was not common until at least the mid-20th century (after most modern food-safety protocols to prevent it came into effect).
In any case, the reason why protocols came into effect was likely due to the seriousness of the illness. Between 1900 and 1950, the U.S. fatality rate from confirmed cases of botulism was greater than 60%. So, even if the incidence of this type of food poisoning was very small, there was great interest in prevention. The two main botulism prevention techniques (nitrate/nitrite curing of meats and pressure-canning) were understood by the 1920s and widely implemented commercially.
More details (including more than you ever wanted to know about the history of sausage illnesses) below.
Part 1: Why We Cure Meat
I should start out by noting that this is a very hard question to answer. The specific botulism bacteria was not identified until 1895, so we cannot know the exact cause of illnesses before then. Historically, botulism illness was also associated with certain foods, so cases caused by other foods were not considered to be "botulism." Foodborne illness in general was probably quite common in the past, but it's hard to come up with exact numbers for the same reason that we tend to have arguments and questions periodically here about people saying, "I've been doing this all my life in this way, and the food never caused me a problem!" Even if a particular illness is frequently associated with food, it's hard to identify the cause without a lab test. And if illness is delayed by hours or even a day or more, people may not make any association with food that was previously eaten.
Botulism is one of the few foodborne illnesses that was identified earlier (back in the 1700s) only because its death rate was so high.
Historically, it was likely a significant cause of illness, particularly in processed meats. The word botulism comes from Latin botulus, meaning "sausage," because it was originally associated with sausage-making. Botulism was clearly identified in the first couple decades of the 1800s. Due to the efforts of Justinus Kerner, the Kingdom of Württemberg (now a part of Germany) passed laws requiring the reporting of all incidents of sausage poisoning. Within a 35-year span, there were 234 reported cases, with 110 fatalities. Keep in mind these are deaths from one type of food (and most of them specifically from blood sausage) in a population of only a few hundred thousand. (For comparison, there were a bit over 1000 cases of foodborne botulism from all foods in the entire U.S. from 1950-2000, in a population nearly a thousand times bigger.) And these estimates from Württemberg are still low, since more and more cases were identified over that period as people became more familiar with the symptoms and likely cause.
Of course, we cannot confirm that all cases from Württemberg were actually botulism, since no bacteria test could be done; some were undoubtedly early cases of trichinosis or other diseases. But Kerner identified many of the symptoms, as well as realizing the necessity of the absence of air. (Sausages which were not stuffed fully didn't seem to cause the illness.) So it's likely many if not most were actual cases of botulism. Even after the cause was discovered and more sanitary methods encouraged, sausage botulism was still a major problem in Germany until the early 20th century (with about 800 cases and 200 fatalities occurring between 1886 and 1913).
Once the bacteria had been identified, new protocols could be put in place, and these numbers declined dramatically. The use of nitrates and nitrates as a curing agent for meats (which dates back centuries) became standard practice in the 1920s, since nitrate/nitrite was shown to prevent botulism bacteria growth. Although they are criticized by some today, nitrates or nitrites are absolutely essential to guaranteeing the safety of meat stored for long periods. (It should be noted that most "uncured" meats advertised today actually contain adequate amounts of added nitrate/nitrite to prevent botulism growth; these chemicals are just added in "natural" sources like celery powder/juice. Thus, they're still "cured," just with a different source for nitrate/nitrite.) Uncured meats continue to be a problem, for example, in Alaska, where native food processing practices result in a botulism rate many times the rest of the U.S. (The CDC reports that Alaska accounts for around 15% of botulism cases, despite having only about 0.2% of the U.S. population.)
In an era where more people are becoming interested in homemade charcuterie, it's important to keep in mind the etymological root of the word botulism and why it's absolutely necessary to cure meat if not consistently refrigerated. Ground or chopped meats are a particular hazard.
Part 2: The Canned Vegetable Conundrum
We don't tend to hear about botulism in relation to meat since curing solved that problem. (And, despite recent trends, home curing of meat is uncommon, so related incidents are relatively rare.) Instead, botulism is probably more associated with home canning these days, often of vegetables or fruits.
Keep in mind that botulism bacteria require anaerobic conditions to grow, along with moisture, the lack of an acidic environment, and storage above refrigeration temperatures. Traditional preservation methods before the mid-1800s tended to involve fermentation (which tends to produce an acidic environment), large quantities of sugar, and/or drying, all usually open to air, all of which will prevent botulism. It was only with the advent of canning low-acid foods that botulism could become a new foodborne hazard.
The canning of low-acid food was basically solved by trial-and-error. While canning was first invented in the early 1800s, it wasn't until the 1860s that the cause of preservation (i.e., destruction and exclusion of bacteria) was understood. And soon after, pressure canning methods were invented for commercial canning, which sped up the sterilization process and could also kill botulism bacteria. However, this mechanism was not understood and universally implemented until the 1920s.
And here we find another problem in answering the question. Botulism, until around 1900, was thought to be a sausage-related illness only. So, we have no way of documenting the statistics of botulism caused by other foods before then. Starting around 1900, it was recognized to be a problem in preserved meats in general, but it was not immediately associated with other foods, like canned low-acid fruits or vegetables. Therefore, if there were cases of botulism from canned foods, they perhaps were not recognized as such until the link was clearly shown in the 1920s. (Before this, canning recommendations claimed that adequate processing was achieved by extremely long processing or by periodic reboiling and processing over a period of days.) While sporadic cases of botulism in canned goods were identified starting in 1904 with a case related to German canned white beans, the evidence was mostly anecdotal and cases were likely only occasionally identified and reported.
That all changed after an incident in 1919, when botulism from canned California olives resulted in 19 deaths, followed by around 20 deaths in 1920-21 from canned spinach. Within a couple years, the risk of botulism from canning was thoroughly investigated and recognized. At first, only home-canning was implicated, but once people started looking, many cases associated with commercial canning were also found.
The CDC notes 477 outbreaks of botulism with 1281 cases of botulism in 1899-1949 in the U.S. Many of these cases involved canning before the necessity of pressure canning standards were understood in the 1920s. But it was probably significantly underreported. There were only a few decades between the time when home canning became widespread and when pressure-canning was recognized as essential, and for most of this period botulism caused by plant-based foods was unlikely to be associated with the traditional "sausage illness." The main evidence I was able to find that anyone recognized low-acid foods as being a problem in canning is the divergent recommendations in early canning recipes. Low-acid fruits and vegetables (along with meat) required significantly longer processing times or periodic reboiling (Tyndallization) even in the late 1800s, long before the botulism danger for plant-based foods was understood. Clearly, they recognized the greater spoilage patterns (and perhaps illnesses) and tried to prevent them. (For some examples: This 1890 canning book recommends 3 hours for processing corn and similar vegetables, after giving processing times of only a few minutes for many fruits. Another 1892 book has similar advice and notes if you see a bulging lid or froth, you may boil and use fruit immediately -- probably assuming fermentation due to inadequate seal or processing -- but "vegetables must, of course, be thrown away" if they display similar symptoms. Other sources recommend extra care, e.g., in sterilization of equipment, when canning vegetables.)
From that perspective, the question is really unanswerable. Obviously people knew something was wrong with low-acid canned foods before anyone would have even suspected a connection with the "sausage illness." And they were already taking rather extreme measures to try to prevent it (like very long processing times, which were probably at least somewhat effective in decreasing the incidence of botulism, even if not preventing it entirely). Also, if you start poking around in sources from the late 1800s, you'll see a lot of references to illnesses caused by commercial canned goods. Mostly the illnesses are blamed on acute metal poisoning (which was possible back then), but there apparently were many incidents of people contracting serious illnesses in a short time and even dying, including symptoms like paralysis. In 1887, the Journal of the American Medical Association was even led to report on such "acute poisoning from canned goods" symptoms (many of which sound to me like botulism symptoms according to what we know today).
For decades up until 1920, canning businesses circulated articles assuring consumers that their products were safe, but all the while other articles were appearing about mysterious illnesses. Immediately after testing in earnest was started for botulism in canned goods, the toxin was found -- both in homemade and commercial goods. And within a few years, canning processing was altered to prevent the problem.
We just don't know how many prior illnesses may have actually been botulism.
Part 3: Recent Trends and New Botulism Types
The CDC statistics for 1950-1996 list 444 outbreaks of botulism with 1087 cases, which (apparently) changed little in annual incidence from the first half of the 20th century. But in this period, it's likely the data is more complete. Moreover, the vast majority of recent cases are due to errors in home preservation procedures, such as canning, whereas outbreaks in the early 1900s were also likely to be associated with commercial processing.
These statistics are also misleading in other ways. For example, before 1960, only type A and B botulism toxins were tracked. Since then type E botulism was recognized as a hazard and connected to consumption of marine products (fish and marine mammals), which added 67 cases out of the 444 reported in 1950-1996.
This again shows the problem of evaluating historical statistics. Type E botulism may have been a significant problem earlier, but it wasn't identified and tracked specifically until 1963 when a major outbreak occurred involving various kinds of seafood (especially smoked fish, where one incident resulted in 17 illnesses and 5 deaths). This follows the general pattern of botulism over the centuries -- first it was related to blood sausage, then sausage in general, then preserved meats, then low-acid canned foods (first meats, then fruits and vegetables), etc. As the incidence from one cause went down due to new standards for prevention, another type or cause would be identified, which would lead to an increase in reported cases when doctors made the association with new foods and ran more tests. And botulism concerns have since expanded further, with the identification of infant botulism in the 1970s (now by far the leading cause of botulism in U.S. statistics, accounting for roughly 100 cases annually). Thankfully, incidence of botulism in general in many countries in recent decades is quite low.
We'll probably never really know how many people may have died from botulism during the "experimental" era of food canning before 1920, but we do know that large numbers of people likely died from botulism in meat before modern curing was understood. And the thing with botulism is -- even when treated with modern medicine, there's still a high fatality rate (5% or so). If not recognized and treated immediately, the fatality rate may be 50% or more. To me, that's justification enough for being extra cautious no matter how rare it is.