Learn English – Where did the idiom “anally retentive” originate

I cannot speak to its origin, but I can speak to the alleged redundancy. There is a reason it works as it does:

Every last one took their seat

The last one took their seat

Everyone took their seat

All three of these sentences mean slightly different things. Referring to only the last one you could be talking about one person. Referring to everyone is too generic and, in usage, it doesn't necessarily mean every single one:

The class will start when everyone is seated

Even after saying such, it is entirely plausible for the class to start before one or two stragglers get seated. By saying every last one, it is clear that every person must be seated.

This also holds for the usage akin to, "The strawberries were tasty; I ate every last one." This implies a much more thorough version of all of them. Plausibly, someone else could have eaten a few or you threw one of the nasty ones away. Using every last one works to reenforce the tastiness of the strawberries and the fact that I singlehandedly ate them all.

There are other phrases that work the same way:

Every single one

Each and every one

These have their own connotations — and avoid the confusion of every last — but they do not fully replace the connotations of every last one. Putting last in the phrase is very powerful.

(The rest of this answer is not terribly important.)

The phrase also has the advantage that it is three short words with a 2-1-1 syllable count. You can verbally morph this structure to emphasize the fact that they are all gone. Every single one is 2-2-1 which is workable, but doesn't quite have the same fast-slow-slow feel to it. More options are good things when it comes to timing and delivery.

This is commonly written by using periods after every word:

I ate every. Last. One.

I personally dislike this style but noticed that advertising from the last decade or two seems fond of it.

From The Encyclopedia of Word and Phrase Origins, by Robert Hendrickson:

The original version of this expression, 'on the water wagon' or 'water cart,' which isn't heard anymore, best explains the phrase. During the late 19th century, water carts drawn by horses wet down dusty roads in the summer. At the height of the Prohibition crusade in the 1890s men who vowed to stop drinking would say that they were thirsty indeed but would rather climb aboard the water cart to get a drink than break their pledges. From this sentiment came the expression 'I'm on the water cart,' I'm trying to stop drinking, which is first recorded in, of all places, Alice Caldwell Rice's Mrs. Wiggs of the Caggage Patch [1901], where the consumptive Mr. Dick says it to old Mrs. Wiggs. The more alliterative 'wagon' soon replaced cart in the expression and it was eventually shortened to 'on the wagon.' 'Fall off the (water) wagon' made its entry into the language almost immediately after its abstinent sister.