This is typical. It's called "grout haze", and you might have to wipe it off a couple of times before it won't reappear.
You just need a clean, damp rag, and some scrubbing. If you didn't take it off immediately, it can set up, so you might need to check your local hardware store for 'grout haze remover' if water and elbow grease isn't doing it for you.
Well, if it had been on the tile, it'd have been typical, but as Jeff pointed out, you said it was on the grout itself. I've never done colored grouts, so I have no idea if it's typical or not ... and you wouldn't want to scrub at the grout to remove the haze, particulary when the grout's fresh (which is when you want to remove it from the tile).
Looking around on the internet, I found mention that leaving water-based grout sealer to evaporate could do it. They recommended re-sealing, and wiping off any excess, but others mentioned it took a lot of elbow grease and solvents. On another site, it was mentioned it could be too much water in the grout, and suggested vinegar or a stronger acid to clean, then resealing; a few people mentioned getting grout dye to cover it up, but you have to strip any grout sealer first.
In some older homes built between 1900-1930's it was common to set the ceramic tile in a bed of cement. The cement was poured over the subfloor. While this was a great stable base for the tile it did raise the floor. You may also have a case where the floor was damaged and during the repair or remodel they added additional layers of plywood or tilebacker to form a base for the new tile.
Best Answer
That does not look like residue. 1.) if you get it wet, does it fade or change in color? 2.) if you scrape at one spot wiht your fingernail, does it come up?
Looking at the photos you've posted, I think that is both mold (in a few areas - showing that the area does indeed stay wet often as you imply), and damaged flooring.
I suspect the material of that flooring is porous and the water has actually damaged the "cheap" surface.
Also, it looks like you may have used a scrub tool on it, which has scored the surface further (the scratches and lines), and the condition will continue to get worse.
My recommendation at this point is to keep it clean and dry, solve the issue that introduces the water/dampness. you will have to replace the damaged areas to return it to like new condition.
If that is real wood, and it has a finished surface, you can probably use a buffer on it, but it does not look like real wood.