Basically anything that's liquid. Fruit is very common, but we've also done herbs (mint and sage specifically). I've also had vegetable spheres like pureed peas and beets as separate spheres. At restaurants, I've had beer sphere's served with pretzel (at Cyrus) and I've seen cointreau spheres used in a drink.
They pop like caviar, and that's what releases the flavor, so they either need to be quite large or have a strong flavor. Otherwise the taste is lost and you're left with just the (fun) "texture". It also often helps to use some food coloring if your liquid isn't a strong color. With small spheres, light colors will disappear and you're left with relatively transparent spheres. I'd also recomend trying reverse spherification (i.e. mix calcium chloride with your liquid and then drop in to an alginate bath), as this doesn't use your flavored liquid to make the shell, leaving you with more "pop".
So look for a strong flavor and a nice pairing. Make it easy to get the spheres in to your mouth. It's easier if the spheres are on another piece of food, as cutlery will often break the spheres. The best thing about spheres is presentation and the caviar pop, so use that experience and look to your advantage and feature it in whatever you're serving. Here's my Sushi Dessert revolving around spherification.
There certainly are solutions that will not work. The reaction will only occur within certain pH ranges. You will sometimes see the addition of sodium citrate in a recipe, this is to correct the pH in to acceptable ranges for the gel to form. For sodium alginate, the acceptable pH range is 2.8 - 10. However, if the pH is < 4, that can inhibit the process (requiring the citrate). I can't find a definitive answer in a quick search for vinegar pH, but it looks like it may be too low. I'd guess that was your issue (since you had success with the milk).
I have also heard of issues with getting alcohol to gel at certain concentrations, but have no direct experience with making alcoholic gels (beyond a little flavoring).
Absolutely the best resource for this stuff at the moment is the Khymos Recipe Collection. Check out the appendix for detailed properties of the different molecular substances. However, Modernist Cuisine may soon become the 'bible' when it's finally released (and some people shell out $500 for it).
In my experience, if you want the caviar "pop" with spherification, you need to do it immediately before service. Leaving the caviar in a water bath can leach color and flavor, and as you say, the spheres continue to gel. My best results have always involved doing it at the last minute. Fortunately, it's not hard to do, and it's a great parlor trick, so you can incorporate the creation in to the service.
I also think that reverse spherification (putting a calcium solution into an alginate bath) holds the liquid center better as the gel forms outward rather than inward. This does leave you with a flavorless membrane thought, as the bath is technically gelling around the solution. So for preparations with a thick membrane, this may not be ideal.
Best Answer
All you need to do is put the liquid on a spoon and slowly lower it into the bath while tipping so it falls off. It may take a few tries but it isn't difficult to master. Naturally, larger spheres are a bit more delicate so will require gentle handling to remove from bath, rinse, and plate.