Etymonline offers no insight. The British National Corpus has three cites from 1989, 1991, and 1992. The Corpus of Historical American English has two cites, from 1981 and 1986. Wiktionary doesn't say anything about etymology, but marks the phrase as UK, Australian, and has a much older cite from Rose Of Spadgers by C. J. Dennis, 1924. The most extensive discussion I have found so far is over at The Phrase Finder:
There have been a few attempts to explain the origin of this odd phrase. [...] The more prosaic suggestion — that it alludes to the practise of throwing stones at crows — is much more likely.
I've found mid-20th century references from England that describe it as an Americanism and American newspaper articles that call it 'an old English phrase'. The dates of those are more or less right but not the locations — the phrase appears to have originated in Australia. Most of the early citations in print come from down under. It has a sort of Australian twang to it and is in common with several other similar phrases, all with the same meaning: starve the bardies [bardies are grubs], stiffen the crows, spare the crow.
Partridge also lists "starve the bardies or lizards or mopokes or wombats", marking them all as Australian expletives, and noting that "Wombats may also be speeded".
From Google Books, here's a couple of thousand for too slow by half and a few hundred too quick by half. Even a couple of dozen too fat by half, and I'm sure there are plenty more adjectives you can be too much of by half.
The earliest usage I can find is Too Civil by Half A Farce in Two Acts by John Dent (1783), which the reviewer judges "Too dull by half!" Obviously it was a current expression even then. Maybe someone else will find an earlier instance, but I doubt the expression has a literary origin.
Richard Lovell Edgeworth Popular tales, 1837 has "Why, my horse Dobbin has more sense by half!", showing that it's not always too [adjective] by half.
I think OP's "only half clever" explanation is unlikely, not least because it wouldn't make any sense at all in Edgeworth's usage above. The actual fraction probably never had any significance - adding "by half" just happens to be the idiom used to add emphasis in these constructions.
If someone is too clever by half it often means they are irritatingly devious and manipulating, rather than actually very clever (the implication being that the speaker, and probably many others, see through the trickery).
Here's another (slightly different) definition from The Cambridge Advanced Learner's Dictionary...
too clever by half - too confident of your own intelligence in a way that annoys other people.
Best Answer
Max O'Rell, is widely credited with being the first English writer to frame the rule of "half plus seven." Here is his wording of the rule, in O'Rell, Her Royal Highness, Woman: And His Majesty—Cupid (1901) :
The first thing to notice here is that O'Rell puts this rule forward as applying to an ideal difference in age between newlyweds—not a maximum age difference. The second and perhaps more glaring feature of the rule is that it applies to older men/younger women relationships, and not to the reverse. Evidently O'Rell would find the reverse application to be far from ideal.
In a Google Books search, matches for the rule of half plus seven go from zero in the period up to the year 1900 to five in 1901, two more in 1902, and two more in 1903. So the popularization of the rule evidently begins in 1901. However, one of the matches from 1901 suggests an earlier origin. The letters to the editor of Literature magazine (June 8, 1901) has two interesting posts on the question. First from the enterprising (and self-promoting) Mr. O'Rell, in a letter dated June 3, 1901:
Immediately beneath that letter, the editor printed this one, written from Dublin on June 3:
A search for the phrase "half the age of her husband with seven years added" confirms that Frederick Locker-Lampson, Patchwork (1879) does indeed contain the paragraph that A. L. cites.
Whether Mr. O'Rell independently devised the same rule (owing to its innate ruleworthiness, presumably) or whether he took it from Mr. Locker-Lampson without acknowledgment is a matter of conjecture. I can't think that in today's world either gentleman is likely to be admired for his insight so much as to be derided for his bald-faced presumption. But in any case, the rule of half plus seven most certainly appeared in print in 1879.
For a vivid illustration of the thinking of the times, consider Lady Poore, "Does Disparity in Marriage Tell Against Happiness?" in The Lady's Realm (November/December 1901):
Clearly, Lady Poore views O'Rell's rule as a restraint against even wider disparities in age between the (older) man and the (younger) woman. I suspect that the age-disparity-at-marriage question arose far more often at the turn of the twentieth century in instances of widowers seeking a too-young bride than in instances of middle-aged divorced men seeking a younger companion.