First, let's clear up a misconception.
Armor Penetration gives you a constant %damage bonus, regardless of how much armor the enemy has see here.
It doesn't matter if your enemy has 300 armor or 15 -- penetrating 10 of that will have the same effect on total DPS increase.
That said, let's take a look at why Armor Penetration is preferred, even with its linearity.
Attack Speed is easy to acquire via items.
Armor Penetration is not.
You can more than match the effects of attack speed runes with a single dagger - 420 gold.
The cheapest item with Armor Penetration is the Brutalizer at 1337 total gold. By this metric alone, Armor penetration runes are worth approximately three times that of attack speed runes.
It is easier to use items to stack Attack Speed over Armor Penetration -- but to get the highest DPS, you want to stack both.
Given that the amount of Armor Penetration available via items is strictly limited, if one's goal is to achieve maximal values for both Armor Penetration and Attack Speed, runes for Armor Penetration are much better than that of Attack Speed.
Your entire attack animation consists of three different kinds of frames: your initial swing, the hit, and the backswing. Animation cancelling consists of cancelling the backswing portion of the animation, usually by moving, to get in to a better position to continue attacking.
Keep in mind though, even though you cancel the backswing, there's still a built in-cooldown before you can attack again that's identical whether or not you choose to cancel. The main advantage, as well as the point trying to be made in the Irelia guide, is that by cancelling it allows you to reposition more aggressively(move forward) so that if a champion was retreating that can result in additional attacks.
Certain champs can cancel in to skills to effectively "double" their auto-attacks though since it resets their auto-attack cooldown, such as Sivirs bouncing blades, or Jax's Empower. Effective use of this is very important to maximize damage.
Best Answer
Turrets start with 40 armor which equals about 28.6% damage reduction.
Caitlyn at level 1 has 53.7 AD and .568 attack speed.
53.7 * .714 = 38.3 damage per auto attack. 38.3 * .568 = 21.8 DPS to a tower.
Add in a long sword you get:
(53.7 + 10) * .714 = 45.5 damage per auto attack. 45.5 * .568 = 25.8 DPS an 18.3% increase costing 350 gold (1% per 19 gold)
Adding in a dagger instead:
53.7 * .714 = 38.3 damage per auto attack. 38.3 * (.568 * 1.12) = 24.4 DPS a 12% increase costing 300 gold (1% per 25 gold)
Long sword is clearly the better option if you are limiting yourself to 1 item even though it is a bit more expensive.