The rules for combat in D&D do start to break down in various places when you start dealing with very small or very large creatures and as you've observed, one of these places is flanking.
I'd guess the reason flanking works this way is simplicity. It's a lot easier to say you need to have reach to flank and be on opposite sides than to have special cases around small and large creatures (which would have to take relative size, not absolute size in to account and D&D avoids that wherever possible - just look at how the AC and attack bonuses for size normalise things to be medium). Most of the time the existing rules work just fine.
This is partially because PCs are normally medium or small, monsters are normally a bit bigger than them but not gargantuan and making things more complicated for things that don't come up very much doesn't really add to the game. Additionally, for very large creatures they've already got a sizeable penalty to AC from being so large and normally they have a pretty huge number of hit points. They normally are tough because they can take the hits, rather than because they are hard to hit and because of this don't care much about their AC. A flanking bonus isn't worth anything if you're already hitting reliably without it (unless you're dealing precision damage, but then isn't darting under the gargantuan dragon and stabbing it in the weak point where it's missing a scale just what a rogue should be able to do?).
Then again, if you're running a campaign where this oversimplification will cause problems, possibly because you've got a tiny or diminutive PC or because you're going to be fighting lots of them, you might want to make some changes to bring back some realism.
One easy tweak to make is to say you can flank a creature if you threaten them. If you're two diminutive creatures flanking a tiny creature and you're all in the same square the flanking works fine. This does make it hard to work out who's where in the one square as you still need to be on opposite sides of the creature so maybe scale up the size of everything in a small scale fight. For example, if you've got 3 tiny adventurers fighting 7 diminutive creatures just make the tiny creatures medium for the fight and the diminutive creatures small. The grid rules work fine, you assume people who are fighting are in the same 5ft square so can hit each other (0ft reach) and anyone with a reach weapon still gets the benefit.
If you want to make things a bit more complicated you could say you can only flank someone if you can reach the centre of the creatures space. A medium creature can flank a large creature with a normal weapon but they need a reach weapon like a spear to effectively flank a huge creature. This means you won't flank many larger creatures because you'd need two characters to both have the reach weapons to get a flank. To fix this you could say two characters can flank a creature if their combined reach crosses the creature.
Now two medium creatures with normal weapons can flank a large creature, one of them needs a reach weapon to flank a huge creature and they both need reach weapons to flank a gargantuan creature.
This really penalises rogues and other classes that rely on precision damage as they don't tend to have reach weapons and if they can't flank they won't be able to pull off a sneak attack. There are already plenty of things in the game that stop rogues having fun (plants, elementals, undead, constructs, and oozes to name a few) so they really don't need any more. As @Carcer points out below, Pathfinder did away with a lot of the immunities to precision damage that various creature types had in 3rd ed so rogues have less things interrupting their fun than they used to. It's worth remembering that Pathfinder made this change for a reason though, and introducing these flanking changes will still unfairly penalise precision damage classes while most likely not changing that much for others.
You could fix this by allowing rogues and other precision damage classes to use the old flanking rules to determine if they can get sneak attack damage but still requiring sufficient combined reach to get the +2 bonus.
It does all get rather more complicated though and at the end of the day I'd still say the existing rules, flawed though they are, are probably the easiest and most balanced way to handle things. Just don't play a tiny rogue.
There's no reliable and generic method for converting specific 4e distances to real-life units. If the fluff for the monster mentions it, go with that. Otherwise, based on a lot of correlating of fluff with mechanics in the Monster Manuals and adventures, I'd estimate a given creature's longest dimension as being within 2 or 3 feet of: the number of squares it occupies in that direction multiplied by 5 (result is in feet, see below for D&D's metric conversions). If the dimension is height, probably add 2 or 3 feet. If it's length/breadth, probably subtract.
That would put a Huge pack lord at maybe 13 feet long (or 18 feet tall, if you think its tentacles at rest reach up further than its horizontal body length--which seems likely given the picture, but I prefer to imagine the tentacles are folded up when not in use), give or take. Its other dimensions can be inferred based on the proportions shown in its art (4e is great about giving us art of most monsters).
A single square on the tactical map is roughly 5 feet (or one meter) to a side.
A 1-inch square on the battle grid represents a 5-foot square in the game world. So a dungeon room that is 40 feet by 50 feet would be 8 squares by 10 squares, which is a huge room but a good size for a busy combat encounter. [PHB1 266, sidebar Visualizing the Action]
(Depending on the country of publication, D&D has a history of converting its standard five-foot grid unit to one meter or 1.5 meters. The latter is more accurate.)
Although there's no official conversion to creature length/height like 3.5 has, the 5x5(x5)-foot square should give you a rough sense of a creature's dimensions...
...but 4e doesn't really care, and it shows.
There are two important things to remember when converting squares to "real-life" measurements. Both arise from the fact that 4e's primary purpose is to be an awesome tactical combat simulator--and it's willing to sacrifice realism and simulation for that goal whenever necessary.
First: obviously a human isn't five feet wide. That width represents the space a person controls in combat, rather than his own physical dimensions. A Huge-size pack lord isn't cube-shaped so much as it's free to wheel and turn within the squares it controls.
Second: most humans are more than five feet tall, but the 4e grid square which a human occupies is a 5-foot cube. That's just... ignored.
So it's nigh impossible to actually say what a displacer beast's dimensions are in metric or imperial units because the squares it occupies are only rough approximations of the space it controls in combat.
And things get even weirder when you measure stuff that occupies multiple squares at once.
Tactical movement on 4e's grid uses taxicab geometry, or Chebyshev distance, so the effect is... often nonintuitive. I'm pretty sure 4e floors are constantly shifting hyperplanes.
I've got no idea how to do a literal unit conversion from Chebyshev distances to "real-life" units for a displacer beast who occupies 9 squares at once, and I don't think it's desirable. In the end, the implications of 4e's tactical movement rules on the physics of the world are clearly unintended and entirely ignored by its creators, and I fully endorse taking the cue from them.
Best Answer
The modifiers are static, and make attacks relative to each other appropriately adjusted for the situation. The equation bakes the advantage / disadvantage of size modifiers into base stats, and makes no impact on two creatures of the same size fighting each other.
Using the idea of equivalent equation, the size modifiers automatically take relative sizes into account.
For example - two tiny creatures duking it out are effectively at +0 size modifier against each other. +2 Hit, +2 AC VS +2 Hit +2 AC = total of zero modifier relative to each other. So they are on equal footing size wise.
A Colossal creature Vs a tiny creature
+2 Hit, +2 AC VS -8 Hit, -8 AC = +10 modifier in favor of the tiny creature. No need to re-apply the bonuses a second time.
A colossal creature VS a Colossal Creature
-8 hit, -8 AC VS -8 hit, -8 AC = total of zero modifier in relationship to each other.
A Gargantuan Creature fighting a tiny creature would be at a total + 6 modifier, and a Gargantuan creature fighting a medium creature would deal with a +4 modifier.
So it scales and auto scales. The math is already worked into the base stats of the creature to prevent excessive calculations during combat.