A question - were there elevated spore levels in the rest of the house? It sounds like you disturbed something removing the old drywall. If that's the case you might be OK.
You need to identify the source. Once you've done that you can usually deal with the problem yourself.
The hard part is going to be identifying the source. Mould is usually caused by condensation on the walls, which in turn is caused by poor ventilation and insulation. So there's warm, damp air in the room which when it comes in contact with the cold wall forms condensation.
If there was no visible mould before removing the drywall, then you should be all right - but I'm not an expert so please don't quote me on that.
When you replace the drywall make sure you install adequate insulation and ventilation in the room and then perhaps test again for spores before decorating. If the levels are still high then you'll need to do more investigation.
If it's ground water, the issue isn't water vapor, but rather just plain water. Is the water table close or higher than your basement slab? If so, not a whole lot you can do to completely stop moisture issues, as that basement was just built in a bad spot.
That said, the main solution would be a sump pump and drainage tile system. Any water coming up through the earth dumps into the drainage tile, into the pump well, and gets pumped out. Again, though, if the water table is already higher than the basement slab, that'll be a never ending battle as well.
That said, perhaps the issue isn't as much about ground water as you think. Does it get humid in your region? If so, then 'damp' basements are par for the course. Basement walls are usually always going to be cooler than the air, and, a such, will be were condensation forms in a humid environment.
To prevent that, you need to a) dehumidify and/or b) insulate the walls.
A dehumidifier constantly running tends to be a normal part of any basement in a humid zone, so that's a good start. Insulating with walls with XPS or EPS foam will help too, it'll keep the moist air further away from the cold wall.
I can't say if an air exchanger would help or not. If it's including an A/C system, it could help (as the A/C is a dehumidifier) but otherwise I have a hunch you'd just be pumping more humid air into the space.
Products like drylock don't do a whole lot. They're not strong enough to prevent hydrostatic water pressure (high water table) and do nothing to prevent condensation. They're a bit of a gimmick.
Best Answer
If you've had problems with water damage in your basement before, there is a really good chance that you have mold. The downside is, it can be hard to detect mold on your own since it can grown within the walls or under carpet. You can buy a mold test kit from somewhere like Home Depot to check on your own. I would probably start there if I were you and if you see you have a problem, then I would probably call a professional. Especially if you've noticed a decrease in air quality in your basement.