According to this extensive bean cooking guide from University of Alaska Fairbanks, the secret is to simmer the beans gently. Unfortunately, they have little more to say on the matter.
Kenji Lopez-Alt, in his column on Food Myths at Serious Eats says (emphasis added):
Most of us have been told at some point in our culinary careers that
salting beans will cause them to toughen. It's incredible that this
little bit of culinary mis-wisdom still lingers, for it couldn't be
further from the truth. A simple side-by-side test can prove to you
conclusively that salting beans (both the water used to soak them in
and the water used to cook them) actually tenderizes the skins.
It's got to do with magnesium and calcium, two ions found in the bean
skins that help keep the structure of the beans' skin intact. When you
soak the beans in salt water, sodium ions end up replacing some of the
magnesium and calcium, effectively softening the skins. Your beans
come out creamier, better seasoned, and have a much smaller likelihood
of exploding while cooking.
He provides this image of the contrast:
I believe the take away from this is:
- Pre-soak your beans overnight, as your grandmother did using salted water
- Cook them at bare simmer, stirring them minimally, to prevent physical damage to the skins.
The second can easily be achieved by placing the bean pot in a moderate (350 F, 180 C) oven rather than doing it stove top.
Summary: Don't toss the soak water.
First, to address the food safety issue, phytohaemagglutinin is gradually destroyed by temperatures above 175F or so. The FDA has referenced studies (see pp. 254-256) which show that 10 minutes of boiling will completely destroy any of that toxin in beans, though they recommend 30 minutes at boiling temperatures to be on the safe side. Yes, some studies have recommended discarding the soaking water to decrease amounts of toxins before cooking, but there's no need to as long as you are certain to have your beans at or near boiling for at least part of the cooking time.
Also, just to clarify your statement about temperatures near 170F increasing the toxin -- it's true that undercooked beans which never reach higher temperatures will release more of the toxin than raw beans. However, I see nothing so far in the literature that suggests a problem if the beans spend some time at that temperature as long as they spend at least 10 minutes (and preferably 30 minutes) near boiling during the simmering. (As long as your beans are actively simmering for a good portion of the cook time, the toxin should degrade: the real problem comes with slow cookers which never reach a boil or even a decent simmer and might maintain a constant temperature that never rises above 180F to destroy the toxin.)
Finally, I should also note that you've asked about black beans, and the concern with phytohaemagglutinin is greater for red kidney beans and a few other related beans where the concentration is particularly high. Thus there are sometimes recommendations for discarding soaking water for those specific bean types. The concentration is significantly lower in black beans, and while there many be some other lectins in black beans to worry about, as long as they are cooked thoroughly, including boiling for at least 10 minutes, there is no reason to worry.
Regarding your other questions about nutrients, flatulence-causing agents, etc. and the soak water, I quote Harold McGee (who is also referenced in one of your helpful links) from On Food and Cooking (pp. 486-487):
A commonly used method for reducing the gassiness of beans is to boil
them briefly in excess water, let them stand for an hour, then discard
the soaking water and start the cooking with fresh water. This does
leach out most of the water-soluble oligosaccharides--but it also
leaches out significant quantities of water-soluble vitamins,
minerals, simple sugars, and seed-coat pigments: that is, nutrients,
flavor, color, and antioxidants. That's a high price to pay. An
alternative is simple prolonged cooking, which helps by eventually
breaking down much of the oligosaccharides and cell-wall cements into
digestible single sugars.
In line with McGee's recommendations, I generally don't bother soaking at all anymore, since the longer I cook the beans, the more I break down the gas-causing elements. (Also, except in unusual cases, I find soaking -- even long-term -- rarely saves me more than 30-45 minutes of cooking, and for beans I'm going to simmer for a few hours anyway, that doesn't seem to help much.) Regardless, I never discard soak water, since I prefer to preserve the flavor and nutrients.
Some people still find that soaking for a short time will improve final texture or keep beans from bursting or breaking apart or whatever. If you find that it makes a difference for you, soak. But aside from an odd batch here or there, I found that the best textured beans I've made generally have not been soaked at all.
One other minor point is that, while I do sometimes use the oven to cook beans, there's no reason not to simmer them on the stovetop as long as you use a heavy pot. If you're really concerned about toxins, bring them to a boil early (while there's still a lot of liquid), but after 10-30 minutes, you can turn them down to a simmer. Unless you tend to cook beans to the point that they burst and cause the liquid to thicken significantly, you shouldn't need to stir them often (or at all) to avoid sticking.
You can add tomatoes or other mildly acidic ingredients once the beans are near-done, say for the last 30 minutes or so. You're right that you shouldn't add them at the beginning, but adding them a little before the end can allow flavors to permeate the beans more (if you want that). Also, adding small amounts of acid toward the end will actually strengthen the beans somewhat and preserve their structure even if you overcook them a bit.
Otherwise, I agree with basically all of your ideas, which seem well-researched. You'll find similar advice given in many other questions here.
Best Answer
Soaking beans will not soften them. If done for a very long time (i.e., days), some beans will eventually begin to sprout or ferment, at which point they will become softer. But that is generally not desirable for basic cooking.
Instead, you'll need to cook the beans to get them to soften. Bring to a slow boil and then simmer until the interior is the desired texture. (A common test is to take a few beans out on a spoon and blow on them; if the outside of the bean breaks open a bit when you blow, they are likely soft and cooked through.)
By the way, the main purpose of soaking is to hydrate the seed coat (the outer covering of the beans). Once that happens, the moisture can more easily penetrate the interior when cooked, allowing the beans to soften. If you don't soak, you'll simply waste the first part of your cooking hydrating the seed coat. Depending on the type of beans, their age, and other factors, that could increase your cooking time by anywhere from 15 minutes to an hour or so. (For more details, see my previous answer to another question here.)
Also, do NOT try using an acid. Acids can actually toughen the seed coat, making the beans take longer to cook and soften. This is one of the reasons chili recipes (for example) often recommend cooking the beans first, before adding to acidic ingredients like tomatoes.
(Note that cooking times for beans and lentils can vary significantly, depending on type of bean and age. For a mixed bean soup, I'd expect about 1-2 hours of simmering for pre-soaked beans, and perhaps as many as 3 hours. Older beans can take longer, and I've occasionally had old batches of dried beans which never quite softened to a consistent texture.)
EDIT: Just to add onto Joe's great advice from the Dry Bean Council, Harold McGee has this to say about beans which never soften (from On Food and Cooking):
McGee also mentions that occasionally you may encounter batches of "hard-seed" beans, which hardened during particularly hot and humid growing conditions. In such cases, the beans will be tough no matter how early they are used; they may take much longer to cook than usual (and sometimes may never soften).