I remember this question popping up a lot on the forums (And eventually getting an official answer) when I was still playing D20Modern.

The answer is that they do not stack - in any fashion. Not even to the degree DuckTapeal says in his post. Every time you roll to attack somebody, you must decide to either Brawl or use Combat Martial Arts.

The word from the developers on why two fighters couldn't win against each other when only Brawling (For combats where characters have sufficiently high HP and CON values), the answer was that in most brawls, you'd expect a combatant to eventually pick up a bottle or other means of dealing real damage.

SOURCE that brawl and CMA don't stack: https://www.wizards.com/default.asp?x=d20modern/bp/20030320a

SOURCE that Advanced CMA and Knockout Punch don't stack: https://www.wizards.com/default.asp?x=d20modern/bp/20051122a

SOURCE that Street Fighting does stack with CMA: https://www.wizards.com/default.asp?x=d20modern/bp/20060411a

RAW

No damage dice => No dice to double

Sucks, but them's the rules. So a critical hit with an unarmed strike does nothing more than 1 damage, excluding modifiers. (I'll be excluding those for the rest of the answer) The tables in the player's handbook doesn't list a "1d1", just 1 damage. "1" is not a dice roll, it's a number.

Of course, the designers do seem intent on making unarmed strikes do 1 damage. I doubt it's a typo, especially considering the basic rules have been updated (it's on v0.2). They even supplied a feat (tavern brawler in PHB) and a class (monk) which increases the unarmed strike damage from 1+mod to some actual dice (like a 1d4). If they never intended the damage of unarmed strikes to be so lame, why would they provide that feat and that class?

As a side note: a critical hit from any weapon causes two failed death saving throws... including unarmed strikes. This is possibly the only time they are on par with more traditional weapons.

Possible RAI and RAF; The Spirit of Critical Hits

Critical hits seem to be meant to do more damage than a normal hit. This is reflected in the doubling of an attacks damage dice. It makes you generally do more damage than normal, although there is a slim chance that you don't do as much on that strike. You can see the odds for some weapon dice values here.

If you're looking to extend this logic to critical hits, I suggest instantiating a house rule for upping it to a 1d4 or supply 2 damage on a critical hit. This could be a fun rule, and it may alleviate the disappointment your players feel while getting that critical hit while hitting something with their... fists? ("Unarmed Strike" is really vague!)

However, it does seem that the designers really did want unarmed strikes to not hit hard unless you've had some experience. Once again, this is because of the Tavern Brawler feat in PHB and the monk class. Is this because they want people to use weapons unless they're a monk or a tavern brawler? Were they afraid of things like the grapplemancer builds coming into 5e? Hard to say. You could ask wizards of the coast by emailing sageadvice@wizards.com.

A Note

5e greatly acknowledges the importance of house rules in D&D culture, so I would not stick with the RAW reading of this. In that article linked above, written by Jeremy Crawford, one of the two lead designers for 5e, says:

We expect DMs to depart from the rules when running a particular campaign or when seeking the greatest happiness for a certain group of players.

The RAW just is not fun and takes the joy out of a critical hit. Come up with a rule that seems right (doubling to 2 damage, or going to 1d4 damage, or something else) for your group. In the words of Jeremy Crawford:

Fifth edition now belongs to the thousands of groups playing it.

Modify 5e to have fun in your group!