Consider the question differently: Are all of my citizens working fully improved squares?
A quick rule of thumb from my experience is to have one work per city, always working for that city (unless something like urgent need for a resource arises). Then watch the city and consider: is the worker keeping up with city growth? If he's not, build another in that city, and it'll help slow the growth and gain you a worker.
You will also sooner or later want 2 or 3 workers connecting cities with roads, building forts, etc... things not linked to one city. You can often gain these when cities stop growing, cause that city's worker can actually finish enough squares! Then he can move on to other work. If not, you can always just build a worker for that specifically.
All of this is if you want the BEST cities. Keeping every citizen working a fully improved square is the ultimate goal, but if you spend your resources on that, you're not doing something else. There are many ways to play Civ4, and having the best cities is only one of them. Prioritize accordingly.
My estimate from the info you gave: 7 or 8 workers is the most you'll need. 1 per city, and 2-3 on national road detail/helping with a fast growing city. You probably don't need the 8th if one of your other cities has stopped growing and the worker finished.
Finally, remember that one of the best ways to build workers is to take them from your enemies!
From my experience, 3 military is good enough for average defense, it allows you to hold out till you can reinforce.
But for your style of play, perhaps 5-10 would be better, you don't want to many as it causes slow down in tech etc..., but then you don't want to get crushed. You should have more on the outer cities and reinforce from your inner ones, I try to always have 3 minimum in all cities, but doesn't always work, then any bigger cities up it to 4-5.
If you notice your enemies normally attack you with massive stacks, the only way to counter those are with a large stack yourself. Have 1 large stack that walks between cities and does the attacking, then leave 3 units in all your cities for normal defense, remember if you suddenly get attacked at a choke point (hopefully you have one) you can always reinforce from your other cities, knocking them down to 1-2 stuff like that.
Best Answer
Building Worker first is far better than most of newbie strategies (which usually consist of not building Workers at all ;) )
But it's not that good either. If only because it takes time for newly founded city to build a worker, and all that time it does not grow. Better build workers in some mature cities, where it takes just a few turns to finish them. Especially in those which can not grow anyway at the moment (i.e. due to happiness cap). Obviously Worker First is the right game start, because you have the only city at that moment.
conqueredliberated cities, previously built by AI.The goal is to never work unimproved tiles. Ideally no city should ever do that.
Hope that helps