I'm a Diamond Terran frequently matched up against Masters level players. If a TvP ends before 15 minutes, I'm usually the one winning it, but in lategame I rarely do as well.
I usually focus on Marauder+Ghost+Viking as I've seen top players do so very effectively, but it rarely works for me due to Zealot heavy compositions. Mixing in Marines or Hellions essentially gives me units that die to 2 Colossi shots, so that just doesn't work. My stutter step micro is very very good and I tend to have strong EMPs, but against Zealots with +3 armor (rushing this is the norm) they just don't die before I run out of room to micro, and meanwhile Colossi are ripping through my infantry and my Vikings are getting picked off by Stalkers. If they add Archons, it's even worse.
It feels as if Protoss can just a-move their force and I have to micro Vikings to stay out of range of Stalkers while stutter stepping while landing EMP's, and even if I do so perfectly I still don't come out ahead unless I have a significantly stronger force to begin with or am microing a drop at the same time (which Protoss can defend with instant warp-ins).
The only thing I haven't attempted is mixing in Tanks, but the level of micro necessary to include them in an already hugely micro-intensive force is not even remotely comparable to what a Protoss has to do with their ball. Most don't even include Psi Storm or Sentries and can still power through me despite weaker upgrades and less supply.
I can attach a replay or two later today, if that would help.
Best Answer
I'm a protoss player so I'm on the opposite end of your situation.
I believe the way for Terran to win is to never directly encountering the army.
Use drops to keep his army out of position and win the attrition war. What I mean by this is that when they pull their army back to base, use your main force to knock out expansions. If you can keep denying expansions, they can't replenish their force as quickly as you can, or they can't afford it. If they don't pull their army back, they're warping in units at home, which means their front line army is that much weaker.
Maurader/ghost/viking is a decent composition, but you have to have marines in there as well. Stimmed marines are the highest DPS unit in the game (not counting storms/fungal over a group of units). When the fights start, make sure your vikings are target firing the colossi, a few shift right clicks in the fight will reduce their numbers much better than just A-moving them in. By keeping colossi numbers down, your infantry army is that much stronger.
Your ground force should be something like 60% marines, 35% mauraders, and a few ghosts. That said, your army composition should be reflective of what he's making more of (This is assuming an infantry force). If he has more zealots get more marines, more stalkers? More mauraders. I also assume that you're making medivacs to help your healing.
You also mention that it's the norm to get +3 armor really fast, and it's right, I do as well, marines do 5 damage (no upgrades) zealots have 2 armor, take off 2 more from guardian shield and you're tickling the zealots with no upgrades. Upgrades make them that much stronger. The answer? Keep up on your attack upgrades too. They essentially cancel out if you are just as high with your attack upgrades. And since you need to make the armory to keep upgrading, might as well do air attack as well (so your vikings can knock down colossi faster).
It's also key to keep their colossi numbers down. Losing 2-3 vikings to take out 1 colossi is huge and well worth the trade. This can be combined with drop play. If you see that they are back at base dealing with your drop with warp-ins and haven't moved their army, stim in and attack his main force before he reacts, by targetting the colossi and taking them out of the battle, your infantry force will be that much stronger.