Anybody who has tried defeating the deity AI in a domination game is aware of the challenge they are undertaking. Personally, I have found it to be extremely frustrating, and there are times when I screamed at my monitor because of how insane the AI is.
For example:
a) On turn 11, I meet the Iroquois who greeted me with: "Peace be with you…" 5 turns later an Iroquois army emerges out of the bushes and surrounds my capital and KILLS me in 3 turns!
b) I discover that Suleiman has founded a city right next to my capital! I quickly purchase a tile that contains one of my starting luxury resources. He is obviously upset and even after apologizing he declares war against me! And overruns me with Janissaries!
c) In another game, I befriended Pachacuti who emerged as the strongest player in the game (luckily for me). After many turns he says: "I have had enough of you…" and DENOUNCES me! He then sends a great artist to one of my cities and STEALS my coal and iron!
d) I hate Montezuma. In one game that I played, he founded a trail of 12 (TWELVE!!!) cities, 2 tiles apart from one another, all the way to my capital and then overran me! His empire was akin to a phallus that was being driven into my capital.
e) In a game that I was playing, Washington's empire was so vast and his army so huge that his units began spawning in unclaimed tiles in my territory because his army already occupied every tile in his own lands (I knew this because I sent a caravel there and was disgusted by what I saw)! After two mobile SAMs and a giant deathrobot had spawned in those tiles he ATTACKED ME!
f) BUT THE WORST, the very worst thing about the deity AI is their greediness. Given the chance, they will STEAL the land closest to you. It is not uncommon, in a late game, to find that several of your cities are completely enveloped by their borders. And if you dare buy the land near them they will KILL you!
So my question is: how do you stop the AI from aggressively expanding towards you so that you can peacefully grow your cities? And also, why does the AI insist on building their cities two tiles apart from one another? Granted, it makes it more difficult to capture these cities but it also means that these cities need to share many tiles with adjacent cities, stunting their growth and greatly reducing their productive output. I have tried rushing the AI but when I arrive at one of their weakest cities they immediately buy walls, a castle and a catapult to defend it making it impossible to capture with my modest-sized army.
Update: I can't very well mark my own answer as correct since that would be conceited! So congratulations agent86 and to everybody who participated in this fruitful discussion.
Best Answer
At the higher difficulties, I've found that the only way to win is to play defensively. Siege units, at least one in each city, probably flanked with another siege unit (or two on a border with an aggressive enemy) and some front line infantry is really the only protection.
My second city is almost always founded in an area with a lot of production bonuses (hills, iron, stone/marble, horses/cows/sheep, etc) and focuses completely on producing military for the entire game. If you grow large enough, founding an additional city of this type can be useful, but if you've got around half a dozen cities or less, one is generally sufficient. Have this city crank out units and only build military buildings there (ideally by rushing them, if you've got the cash). Most of the rest of the empire is money- and happiness- focused, although they might churn out disposable units if war is looking inevitable or going poorly.
When it comes to science, pick techs that get you closer to:
The AI will become increasingly aggressive towards you if they feel your military power is less than theirs. However, they're terrible military strategists. They will send their entire army right to your cities, with no regards to how that leaves their defenses or how positioning effects their chances at survival. At the higher levels of difficulty, this can mean an almost constant trickle of units.
However, they tend to fail to prepare for the bonuses you gain for fighting on your own turf - take these whenever possible, and then turtle heavily when they declare war. After a while, you'll have an army of extremely powerful units (ie, 2-shot per turn siege units with extra combat range, etc) and great generals, which you can use to push the tide back. Plan on cranking out front line infantry units, as they're somewhat disposable. Put a large priority on preserving your high-level siege units, though.
Likewise, if you see a weak neighbor, go after their stuff. The AI gets big bonuses to production, so their cities are going to be better equipped than ones you could found yourself. Declare war, turtle for a few turns while they bumble about ineffectually at your borders, and then steamroll.
Just remember not to completely abandon your own defenses...