The most critical part of your choice is "Does your graphics card support directx-10 (or 11)?"
If your graphics card does support higher versions, its usually best to run with the most recent version, to get whatever bugfixes, feature improvements (advanced lighting I believe) and performance gains.
Naturally, if your card does not support the higher versions, you should NOT try running the game in that mode, and stick to Directx9 instead. As StrixVaria says, running the game in that mode should simply not work, (that is what not supported means after all) but the best you can hope for if you do try is broken graphics or the game crashing.
Expenses
The top two things that cost you money in Civ 5 are:
To reduce the first one, be strategic about what you build where. Cities should be focused on a task. Pick terrain, tile improvements, and buildings that compliment this focus. For instance, if you are building a production-focused city, build it near hills and forests, and focus on military and production enhancing buildings. Don't build science or cashflow generating buildings, since you won't be producing much of those in this city.
Likewise, if you are building a science focused city, don't build military or other production enhancing buildings there. The city's focus should be on science or economic enhancing buildings instead.
The only exception is happiness - in order to maintain a positive balance on the happiness scale, you'll likely have to build some happiness generation buildings in most or all of your cities. This tends to be one of the largest trade offs economically in most of my games - do I make the people happier, or do I bank some extra gold?
You can also guide each city's governor to focus on a particular area, and this can help your citizens understand what they need to do in order to support your vision for the city. If you leave the governor alone, the city may ignore some of your tile improvements in order to work tiles that create something you're not interested in, but which makes the city's output more balanced.
Also, don't let your workers run free - focus your tile improvements on whatever works best for the city the tile is in the radius of. Automated workers don't always pick the best terrain improvement for you. They may destroy forest or plant farms around a city that is going to be production focused, for instance.
Every unit you produce costs you money every turn as well. Try to focus your attention towards a small, but mobile army, and don't fight on multiple fronts if you can avoid it. Mixed unit tactics involving ranged artillery units and close-up front line fighters work best. The AI is terrible at war, and they will generally start on the offensive, even though you've likely got your army massed and ready to funnel them into a meat grinder.
If you find yourself with a strong surplus of units, gift some to an allied city-state. You'll gain influence with them, and if you declare war, those units are likely to still benefit you indirectly.
Income
You gain money via citizens assigned to money-generating tiles, trade agreements with other Civs, and by trade routes between cities. Any duplicate of a luxury resource does you no good - it is there to trade.
If you end up with 2 of a luxury, or more of a strategic resource than you need, find someone rich and sell it to them for a tidy profit. Suppling your possible enemies with strategic resources might sound like a bad idea, but since you control the flow of that resource, if they start doing something you don't like, you can cut them off and drastically reduce the effectiveness of any resource-requiring units they produced in the meantime.
There's also a glitch that I don't believe has been patched whereby if you destroy a luxury good improvement (thereby reducing the amount of that luxury good available) you can end a trade agreement early at no penalty, and then rebuild the improvement and sell the good back to that same Civ for another sum of money. This could be considered cheating, and I imagine someday it (will be/has been) patched however.
Roads between cities are generally profitable, but you want to try to minimize the number of redundant road links you have, as having 2 roads between 2 cities counts the same as having one, and costs you extra.
Best Answer
From Wikipedia:
So the Gold edition contains all of the GOTY content, plus the Korea and Ancient Wonders DLC packs and the first expansion Gods & Kings. G&K is the most significant difference between the two editions because it's a full expansion that adds a lot of new mechanics and content to the base game: religions, espionage, new techs, improved combat and diplomacy, plus several more civs and city-states.
Most of the DLC packs on the other hand are extra maps and civs that just give you new options to choose from, but as someone pointed out they are often required to join online games. They are not strictly necessary, especially for single player, but they do offer more variety.
I disagree with the other answer's conclusion; G&K is a massive upgrade that is well worth buying Gold over GOTY.
To address your edit, Gold edition does not include future DLC: there is now a second (and very significant) expansion Brave New World which is not included in either of these editions. To get the full package you would need to have Gold + BNW, or at least Civ5 + G&K + BNW.
Update: The Complete edition has since been released, containing "both expansions and all the DLC packs."