No, and Yes.
First of we need to consider this post by Jeremy Crawford:
Dual Wielder is intended to work with actual weapons. Using the feat with improvised weapons is up to the DM.
See also: Shield and Improvised Weapon
It is not RAI to use a shield or other improvised weapon with the Dual Wielder feat. The rest that follows assumes you talked your DM into allowing that.
One Bonus Action
can I attack 4x in my Attack action (axe, shield, axe, shield)?
You can attack twice in your attack action, after level 5.
You, however, only get one bonus action to make one more attack if you attack with a light weapon you could attack (axe, axe, Shield), if your DM allows you to attack with the shield.
Relevant text from PHB 195:
you can use a bonus action to attack with a different light melee weapon that you're holding in the other hand.
So, you get a total of 3 attacks.
Rage Bonus Does Apply
Rage is a modifier added to the damage, so the following rule (also PHB195 would apply):
You don't add your ability modifier to the damage of the bonus
attack, unless that modifier is negative.
Rage, however, is not an ability modifier, and still would apply. You wouldn't, however, add your strength modifier without the feat -- but since we're talking about with the feat your good on this front.
+3 to AC — Um... Maybe?
I'd argue it is either a shield or a weapon depending on how you are using it. I, personally, don't think a shield grants any more armor because it is being used a weapon. That said, using the shield is already against RAI, but might be allowed, and if the DM allows that, you could push for the +3 -- but I think it is a stretch.
By strict rules as written, you are correct. The feat is written poorly and has been discussed ad nauseum on the official Paizo forums, with many folks arguing both sides. There are two factors to consider that allow you to clarify the intent of the feat.
First, removing all penalties to hit provides an overly large benefit in comparison to other feats. Most GMs would agree that this is clearly out of line and would not read the feat this way or allow players to use it in this fashion.
Second, (and perhaps more importantly) there is a feat table in the Pathfinder RPG Core Rulebook where this feat was originally printed. I've provided a tiny excerpt of the table below, where the Shield Master feat is listed (pg 116 in the 5th printing). You'll note that the condensed benefit for the feat is listed as "No two-weapon penalties when attacking with a shield". While this benefit description is by no means the be-all-end-all rules source for the feat, it does clarify the intent of the author.
For the two reasons above, it is likely that this feat was only intended to remove the two-weapon penalties when attacking with a shield along with another weapon (options 2 and 5 from your list above).

EDIT: Thanks to SevenSidedDie for bringing to attention the fact that we also now have an official FAQ Answer on this topic which reads:
Shield Master allows a character to ignore the Two-Weapon Fighting penalties on attack rolls with a shield while wielding another weapon, but not any other penalties.
Best Answer
A general preface: each edition of D&D is its own game, and rules should be evaluated as such. Just because things happened in old rule sets does not mean that it will in 5e. Though it can be helpful to look at old editions for inspiration, you should be careful of the rules environs of the edition you are trying to modify when making house rules.
First, Shield Bash is already a named mechanic in 5E: it's the additional power associated with Shield Master that allows you to attempt to push someone as a bonus action when you make an attack. Also, you can do this same thing as a Full Action (or a piece of an attack action) using the contest rules and the "Shove" improvised action (Players Basic 74). That said, that's not what you're asking about here. What you want to know is how to do damage with your shield.
If you want to hit someone with your shield, it is treated as an improvised weapon attack.
The question then becomes does it resemble any existing weapon we have game statistics for? as per the "Improvised Weapons" section on pg 47 of Player's Basic. Looking at the list, I don't see an obvious resemblance (though feel free to make your own observations/additions). Thus it acts as a 1d4 weapon and proficiency is only granted if you have proficiency in improvised weapons.
There is also no provision for removing the defense bonus. In fact Shield Master improves the shield with attacks without removing the defense bonus so there seems no need to import that rule from a prior edition. (Additionally, it seems to be punitive beyond the spirit of 5e's rules.)
So, ultimately, shield bashing is a flavorful power that is probably not a good mechanical choice. If you want to do it, talk to your DM, but without house rules, it's not a great use of an action. If you want the mechanics, and don't mind not doing damage on the push, Shield Master's push is probably the best you're going to do.