As you note indirectly (in that your quote refers to it), the form smoothe is recognised by the OED, and it's the other Oxford dictionaries, including the free-access public online version.
We can find it going back some date, but since a greater variation was tolerated in the beginnings of Modern English, than now, that means little of that period. It's also pointless to speak of who would or wouldn't use a spelling (as we might perhaps a grammatical construction), when we move into the middle of the Modern period, because such would be decided upon by editors or printers, as well as writers, especially in later editions. It's interesting to note the following changes with the same book, Samuel Richardson's Pamela, or Virtue Rewarded, first published in 1740. (Italics his emphasis, bold mine, in each).
Taking one section of a length sentence from the Third Edition of 1741:
…Let no thwarting Accident, no croſs Fortune (for we muſt not expect to be exempt from ſuch, happy as we are in each other !), deprive this ſweet Face of this its principal Grace : And when any thing unpleaſing happens, in a quarter of an Hour, at fartheſt, begin to miſtruſt yourſelf, and to apply to your Glaſs ; and if you ſee a Gloom ariſing, or ariſen, baniſh it inſtantly ; ſmooth your dear Countenance ; reſume your former Compoſure ; and then, my Deareſt, whoſe Heart muſt alwayſ be ſeen in her Face, and cannot be a Hypocrite, will find this a Means to ſmooth her Paſſions alſo :…
Here smooth appears twice, in the spelling most conventional today.
By the 11th Edition of 1776, this spelling has been altered:
…Let no thwarting Accident, no croſs Fortune (for we muſt not expect to be exempt from ſuch, happy as we are in each other !), deprive this ſweet Face of this its principal Grace : And when any thing unpleaſing happens, in a quarter of an Hour, at fartheſt, begin to miſtruſt yourſelf, and to apply to your Glaſs ; and if you ſee a Gloom ariſing, or ariſen, baniſh it inſtantly ; ſmoothe your dear Countenance ; reſume your former Compoſure ; and then, my Deareſt, whoſe Heart muſt alwayſ be ſeen in her Face, and cannot be a Hypocrite, will find this a Means to ſmoothe her Paſſions alſo :…
Here, the spelling has been "fixed" to smoothe.
In a later two-volume version of 1811, we have much more changes:
…let no thwarting accident, no cross Fortune (for we must not expect to be exempt from such, happy as we are in each other !), deprive this sweet face of this its principal grace: and when any thing unpleasing happens, in a quarter of an Hour, at farthest, begin to mistrust yourself, and to apply to your glass; and if you see a Gloom arising, or arisen, banish it instantly; smooth your dear countenance; resume your former composure; and then, my dearest, whose heart must always be seen in her face, and cannot be a hypocrite, will find this a means to smooth her passions also:…
Here the orthography and capitalisation are as we would have them today, and punctuation is close (there's still more space before the colons and semicolons than a printer would put them today, and much more before the exclamation mark). Smooth is yet again, in its most common spelling.
The move from smoothe to smooth tells us little, for we know that is the most common spelling today so would expect to see such a change alongside those to punctuation, but that from smooth to smoothe between 1741 and 1776 points to a controversy; someone decided that smoothe was to be preferred over smooth enough to change the previously printed text. While the change was not as radical as that which changed the capitalisation for the 1811 version, it was either in the same spirit, or was stronger still and held the original to have been in error.
Or in other words, there's nothing new here. We could go back earlier, but it's rather pointless. While spelling prior to the 18th century wasn't quite the free-for-all it's sometimes described at, it was considerably looser than it was since then, and also the ngrams data we're using is too sparse to tell us anything useful. For the record, the graph for 1500-1800 for relative preference looks like:
These two spikes are massive compared to what we'll find later, and against a general trend of nobody ever using smoothe. This isn't showing a trend, or lack of trend, this is showing a lack of good data. (Any earlier again, and we're looking at Middle English, and while the ending of smoothe/smothe/smethe may have had an effect on the existence of the form smoothe, it's clearly meaningless in terms of the sort of comparison we're doing here).
For the period since though:
Incidentally, bar some differing spikes near the beginning, these are much the same for British and American sources. As we can see, smooth was always the more popular by a massive margin, but smoothe was never entirely extinct.
Now, let's compare with the third-person singular:
Now, the relative popularity of smoothe vs smooth is included here, but never rises high enough to feature visually on the graph. While smoothes was always considerably less popular than smooths until the 1990s, it was still massively more popular than smoothe was.
Possibly this could be explained by smoothe only being used of the verb form (or at least, it is only accepted in that spelling in the verb form by some dictionaries that accept it at all), while smooth is accepted in both.
The most interesting point of comparison prior to the 1990s, is that while both smoothe and smoothes gain in relative popularity in the first half of the 19th century, after that smoothe begins to die down, while smoothes holds a relatively stead position.
It' would not be surprising that those who use smoothe would also use smoothes and those who use smooths would also use smooth, but what we have from the mid 19th century until the 1990s is a sizeable number who use smooth but also use smoothes.
We can get a rough measure of that preference from this (again though, this will over-estimate due to those who use smoothe as a verb using smooth as an adjective):
And it's a curious matter that we had so many, even prior to the massive spike in the 1990s. It's not hard to see why some would favour smoothe when we consider soothe (all the more since soothe/sooth is a spelling mistake people are warned of, as they result in different words). It's more surprising to see people essentially treat smooth as having an -es ending, that other -th words do not have. Granted, there are some "gotchas" when it comes to -es endings (particularly when it comes to -y and -o words), but it would certainly be interesting to find a justification made for the spelling choices at the time.
When it comes to the 1990s, the change is truly massive. While ngrams can show some strange peaks and troughs around this period, presumably due to matters of the available data, this seems to go way beyond that.
I'm inclined to believe that it must be strongly linked to the spell-check matter that you brought up. In a way there's nothing new in this either; reliance upon computer spell-check algorithms are new obviously, but changes in technology and conventions around how writing is produced for consumption affecting our spelling, are as old as Modern English. In other words, I think much of the blame lies with Microsoft Word.
There is a dip for the last year of the dataset. This could be nothing but an artefact of the data collection (that is, a side-effect of ngram's imperfections, and nothing else), but it could also be the fact that as you say neither are accepted by the current version, sending people to other sources to check them, while some other spell-checks differ in opinion (I don't have a copy of Word to hand as I'm not on a set-up used for the sort of programming tasks where I'd want to test how Word treated something, only for writing for which I use LibreOffice. It, and Firefox as I type in this box, both allow smooths but question smoothes).
The period before then I find more puzzling. Dr. Johnson records smooth, and is perhaps the first to condemn the use of smoothen ("A bad word among mechanicks for ſmooth"). He makes no mention of smoothe and smoothes and in his examples he includes ſmooths and the obsolete ſmootheth which could be modernised as smoothes, but just as likely as smooths.
Webster seems to be inspired by Johnson's remarks when he notes smoothen "for smooth, is used by mechanics; though not, I believe, in the U. States.". Again, he lists smooth without any comment on an unusual third-person singular, or any variant. By the 1892 edition, his successors have dropped smoothen but added nothing of note to smooth.
That a dictionary of record like the OED notes smoothe, and by extension smoothes is to be expected; both forms are quite clearly in considerable and respectable, albeit minority, use for some time. The OED would be out of line not to note it. There seems to be nothing to learn from the versions I have to hand on the curious use of different variants for the conjugation.
The Heritage dictionary is the most curious, in noting (according to your question) only smooth and smoothes but not smoothe or smooths, in their respective positions. I can't think of any analogous case of the top of my head, so we can imagine the usage panel were consulted at some point, and presumably returned the opinion that smooth and smoothes were simply the correct forms. It could be no more than the matter of asking too small a pool of opinions (just about anyone will find something strange in any usage panel's decisions).
Amusingly, I went to write smooth in the third person singular in a comment, while in the midst of writing this. I could not for the life of me decide how I normally spell it! I wouldn't feel particularly strongly that it must be one or the other, but I was stuck for some time; "I know there are two ways it is spelt, but can't remember how I spell it!". Maybe the usage panel felt not too differently to that!
Best Answer
A rule of thumb guide for consonant doubling before suffixes is this:
If the last syllable of the root is:
we usually double the final consonant before the suffix. There are some letters that we don't generally double before suffixes. The most important are 'w' and 'y'. The letter 'l' is not one of these letters:
Above we see four words whose roots have stressed final syllables. As they all have a consonant/vowel/consonant ending, the final consonant is doubled. This also occurs with the verb expel in both British and American English.
Note this does not happen with most words if the final syllable is not stressed. In other words if condition (1) is not satisfied:
In the examples above there is no doubling of the final consonant.
However, traditionally, in British English, but not American, we have a rule for doubling double root final 'l'. If condition (2) above applies, whether the syllable is stressed or not, we double final 'l':
[It should be noted, however, that some American style spelling is now an acceptable alternative for some British words. So you can also find 'pummeling' for example in texts using British English.]