The supply limit based on your empire size is not related to the upkeep cost. Instead, if you exceed the supply limit, all your cities take a 10% production penalty. In most games, you will run out of money before you get to that point.
The answer is: it depends. :)
The best strategy is to have a strategy, i.e. decide how you want to win the game in advance. Are you going for a domination victory? Scientific victory? Cultural victory?
Pair this with your chosen civilization and the surroundings of your city(ies) to decide what to build and research.
Are you going for a cultural victory and are playing Ramesses? You should try to build as many wonders as possible and any improvements that'll help you do so.
If you're going for a domination victory with the Chinese though, you should go for military buildings, anything that'll enhance production of great generals and happiness-boosting buildings (since your sprawling empire needs to be kept happy).
For any civilization though you need to balance these decisions with the given situation. Is your city in the mountains but has very little access to food? Then you can safely skip the production-boosting buildings but should concentrate on granaries and water mills. Or are you situated on a tiny island? Then harbors and marine buildings should probably be your preference.
You need to play to your civilization's strengths and balance out your situation's weaknesses. Playing "evenly all over the map" usually works in easier difficulty settings, but you'll need to specialize and focus on higher difficulties. If you set yourself a goal, the question of what to build or research next usually answers itself.
Try to shoot for a cultural victory for instance, and begin by choosing an appropriate civilization. This should be a good experience to show you how playing a focused game can turn out. Playing "all random" is usually more challenging, since it may take quite a few turns until you have formulated a strategy (or you may never get around to formulating one at all).
Anecdote: I just won a cultural victory with the Egyptians, and it was an enormously fun game. I had all of two cities with no military power to speak of (two Infantry units rather late in the game). My empire was tweaked to perfection though and was oozing culture (obviously) and money, which I used to keep good relations to my neighbor Wu (China), who apparently went for a domination victory. I was declared war upon twice by other civilizations (once by two civilizations teaming up against me), and both times used my good relations to have my enemies wiped out by China for me. I had to bribe my way through the last 20 or so turns, since Wu and I were the only civilizations left (~40+ cities vs. 2), and I was the only thing standing in the way of Chinese world domination.
Lots of interesting gameplay to be had in this game. :)
Best Answer
The combat strength of a city is determined by population and terrain, and is then effected by any defensive buildings built inside of it and social policies selected by its ruling civilization. Garrisons inside of cities also boost its combat strength. Hit points are the same way, but they're not effected by garrisons.
Source