I would recommend using Whiskey Stones. They are used in whiskey instead of ice cubes. So they should be heavy, won't rust, and are supposed to be immersed in liquid that you'll consume. I think that probably meets all your criteria.
Whiskey Stones
Another alternative is to use a rack. This comes with the Sous Vide Supreme and I find it quite useful for keeping meat submerged. It does however only accommodate certain sizes of meat. It works quite well. You can buy it separately online for $12.
There is absolutely real truth to improper sous vide cooking and botulism. Clostridium botulinum is an anaerobic organism - it grows when there isn't oxygen - like in sous vide vacuums and canned goods.
The risk is that sous vide cooks both without oxygen and at temperatures so close to the perfect repoduction rate for the organism. If you cook it a little lower than recommended, you could be creating a perfect place to reproduce. Clostridium botulinum dies around 126 F (52.222 C) - so most sous vide won't go lower than 130 F (54.444 C).
The opponents state that the temperatures in general are far too low and if we were cooking for a few seconds, it would be. Luckily, pasteurization is a function of temperature and time. This is part of the sous vide magic. Bacterial death is a result of heat and time - if you have a high heat you may only need it for seconds. If you have lower, but sufficient heat, then as long as you cook it long enough (see recommended reading below) - then you can still pasteurize the food. Sous vide often cooks foods for hours and hours - either for taste and/or pasteurization sake.
A great resource for information here is Douglas Baldwin .
Additional, real, danger comes from if you store your finished product in the vacuum bag at improper temperatures (not freezing). Botulism spores need to reach 250 F (121.111 C) to die (this is why, in canning, some food needs to be pressure canned). You won't hit that in sous vide cooking. If you cook the food, cool it, and then store it in a non freezing temp - there's a real risk that the spores could eventually become active and reproduce. If you're going to keep sous vide food after its been cooked, generally freeze it and then reheat (quickly, in sous vide terms) in an eating temp sous vide bath to consume.
Best Answer
Wrap the bone ends with a layer of foil, plastic wrap...or even folded paper towel. Then seal.