We've had crickets getting into our basement, I believe through a gap somewhere in the basement egress window edges. Those windows have timber-lined window wells, and they like to sneak around between the timbers. I just threw down a bunch of Ortho Home Defense MAX all around the house last night, paying special attention in the window wells, and it was like a war zone down there this morning...dead and dying crickets abound.
If I remember, I'll come back in a few days with a report on the population trend inside the house.
It MAY have been a carpenter bee. They look like big bumble bees, and are good to pollinate plants. However they will do a great deal of damage with the galleries they build, gradually eating away a lot of wood. Don't let them get a foothold or you will truly see a great deal of damage done.
A carpenter bee will drill a hole that is perfectly circular. They like cedar, and go for places like fascia boards and soffits. Then they turn, and will drill galleries just under the surface of the wood.
A symptom of carpenter bees is that you have woodpeckers hammering on your house. They will find the galleries, and can hear the echo as they tap against a hollow spot.
If it looked like a wasp, then it still has a nest there. A wasp will not just wander into a hole. It has a reason to go in there.
No matter what it is, I would suggest dusting the hole with an insecticide. (Delta Dust seems to work well for me. I use a bulb to inject the dust into a hole. Carpenter bees are not aggressive, but then stand well back, in case these are more aggressive wasps or hornets.)
If you find carpenter bees tumbling from the nest, wait until they are all dead. The last nest I got rid of had 15 (LARGE) bees in it. I had to repeatedly dust it until they all were dead. Then you need to clean out the galleries (I use a Rotozip tool for this) and fill them with caulk or Bondo, as otherwise the bees will just re-inhabit next year. And if you just fill the entrance hole, a hungry woodpecker may still find that nest.
Once you manage to eradicate the nest, yearly spraying will prevent them from building new nests. They will just find other places to live, NOT in your home.
Best Answer
Seeing cracks on the walls or ceiling is fairly obvious, but the common hidden gaps that let insects in are usually hidden from sight. They are usually around light fixture electrical boxes and under moldings around windows or exterior doors.
The best way to stop these unwanted pests is to remove device covers and light canopies and caulk around the electrical boxes. A thin carefully applied bead of painter's (latex) caulk (do not use silicone caulk because it can't be painted) can seal up gaps around window and door moldings.
Not only will you close up places for bugs to get in, but you will help stop outside air from entering the room, thus lowering with your heating and AC costs.