I would venture a guess that @Tester101's suspicion about your floaties coming from your glass (unrinsed soap, dust, etc.) may be heading down the right path. I would try rinsing a glass, scrubbing it with a new sponge or clean washcloth, wiping it dry with a clean towel (not paper towel, as this may leave pieces of paper behind), and then refilling it. @NiallC's comment about asking for a water report from your municipal water supply is a pretty good alternative, but doesn't factor in anything that may come from the pipes between them and you. Most health departments will do water testing, so you could always fill a bottle and take it in.
Beyond that, if you really have to know what's in the water, your only real solution to absolutely know with 100% certainty (technically, unless you watch the health department do the tests, and can verify their results, you can't KNOW that they're right...but I'd probably just accept it) is to get testing kits for this, that, and the other thing (that was just the first page offering a wide range of kits that I found), that operate similar to what you might use for pool water. Depending on the sensitivity levels, you may pick up on things like trace amounts of chlorine...but of course this is going to get back into the more expensive end of things, as those kits aren't cheap (Grainger's seem to run around $40-$50 and up...it's about $20 for their cheapest, which is testing for copper, on that first page of results).
You could pick up a pool test kit, which will cover a couple of parameters (chlorine, calcium, hardness, etc.), but it's not going to cover everything and may not operate in the range you need to identify your floaties...it's designed primarily to help you maintain clear-looking pool water that you're not drinking, not crystal clear tap water.
So really:
Ask for report from municipal water supply
Take in sample to health department
If you're still paranoid and want to know what it is, buy test kits
If you just want to get rid of the floaties, filter, filter, filter, and make sure your dishes aren't the source
Varnish is usually a mixture of a drying oil, some resin such as pine tar, and a solvent such as turpentine.
The drying oil most likely won't kill you. The most commonly used oil, linseed oil, is actually just non-food-grade flax oil by another name.
The Greeks put pine tar in their wine, and it's a tasty treat.
The solvent is a little more troublesome. The pinene in turpentine for example, has a nasty looking chemical structure. However, Solvents, by their liquid nature are pretty easily washed off, epecially when you put plenty of soap in your water.
I'd give the dishes a good washing, and use them without a second thought.
I'd also try not to leave a paintbrush in the kitchen sink again.
Best Answer
You don't specify your location. However, a child's play mat, a yoga mat, a good quality carpet underlay, a doormat, or a scrap of high density foam from a dumpster all can serve the need you describe.