[RPG] multi-class between a class and an archetype of that class

Thou shalt not sacrifice caster levels.

As the golden rule of 3.5 multiclssing, it's hard to beat the above. It's quite possible to apply requirements gathering methods to 3.5 as well as 4e, and estimate approximate to-hit and damage at each level.

Plan out your character to 20.

To answer:

s there an effective way to identify class combinations which seem appealing up front, but which will fall behind as the characters level up (without actually taking the character through its paces in a game)?

is simple: plan out your character to level 20. Identify, at each level, what benefit the character is deriving from your choices. Test against the requirements given in your requirements step. This way, when your character shows up in game, you have an idea of your intent and the capabilities of the character.

To answer what classes synergize well:

Non primary-casting classes tend to synergize well.

Given that most aspects of a level up are cumulative with prior choices (BAB, feats, HP) you want to avoid class features which depend on your level in the class. Therefore, avoid class features which have level as a variable within the feature.

Rages from barbarian are fine. You get more as you level up:

a barbarian can rage for a number of rounds per day equal to 4 + her Constitution modifier. At each level after 1st, she can rage for 2 additional rounds.

but you don't have the benefits of the rage reduced (proportionally to your compatriots) as you see in spells.

For the casting classes it's certainly possible to combine them in "gishy" characters (spell-slinging fighters) but they require a specific focus into the combination, preferably enabled by appropriate choice of prestige classes.

So, if you fail to raise rogue, you don't get as much sneak attack bonus damage, but if you're increasing fighter as the other class, you get a higher BAB to compensate.

Whereas a rogue would effectively be writing off "improved uncanny dodge" due to multiclassing.

Therefore, a good combination can be fighter/rogue, because each class brings a separate thing to the table, without having a significant opportunity cost for choosing the other. Other good combinations are those that lead up to exceptional PrCs, which combine features from both classes in useful synergy.

To the first part of your question, you don't gain the bonus feat progression.

This ability does not automatically grant feats normally granted to fighters and monks based on class level, namely Stunning Fist.

As a 5th level Brawler, you'd add your Brawler level (5) to your effective monk level (0) and effective fighter level (0) for the purposes of feat prerequisites and other interactions.

For the second part,

While a character can multiclass with [a hybrid class's] parent classes, this usually results in redundant abilities. Such abilities don't stack unless specified.

I'm not entirely certain how this interacts with damage dice, but a ruling in line with intent is that you use the larger of the two damage dice. A fair ruling could allow the progressions to stack anyway.

AC bonus is questionable, due to the abilities sharing the same name. I personally would rule that they do stack, due to different typing.