Is it -er, -ar, or -or?

The first thing to understand about historical variation in modern beggar versus older begger is that the history of writing ‑ar, ‑er, ‑or for English word-endings is rather complex and not a little muddled. It is unwise to look for perfect predictability here.

In answer to your question about why other words didn’t do that, it turns out that some did. Like beggar, liar too started off with ‑er, as did pedlar, and even today both peddler and pedlar continue to occur even though the ‑er form is the elder. Going the other way, Scots once had socerar for sorcerer, and you can still find people who write sorceror.

It should be no great surprise that there would have been broad historical variation in spelling these sorts of words, considering how all three of ‑er, ‑ar, ‑or as unstressed final syllables are and were pronounced identically by most speakers. Before spelling was regularized, whether a written word ended in -er, ‑ar, or ‑or was up to each individual writer’s preferences, and those in the north of the Isle of Britain often used ‑ar here where ‑er was more prevalent in the south. A lot of the northern ‑ar words later got reworked into ‑er words, but not all of them. And some, like begger, went the other way.

How that sorted itself out under regularized spelling, let alone readjusted spelling by those trying to toe a more Latinate line, did not always follow the same path for each word. The French also had some hand in this, since today’s friar is spelled that way because despite having been a frater in Latin, we got the word from frere in Old French (Modern French frère).

Although the normal agent-noun suffix in English is ‑er, English also has a number of agent-nouns that derive from Latin now have ‑ar there instead, such as bursar, scholar, vicar; vulgar and even cellar are also from Latin. But those only settled out that way due to spelling reform; most were originally ‑er words in English because of having come to us through the French, who had changed Latin -arius words into -(i)er words. Some of those instead ended up looking like solitaire in English.

Thanks in part to the invention of Old French which had only ‑(i)er there, many of these words were once spelled with ‑er before getting put “back into” ‑ar form under 17^th-century spelling reforms. And some — like pedlar, liar, and beggar — seem to have been dragged along for the ride more by analogy rather by etymology.

While these ‑ar words that we got from Latin (with or without French intervention) are in theory distinct from native English words ending in ‑er and from Latin ‑or words (mostly agent-nouns like author, cantor, doctor, censor, cursor, elector, inventor, lictor), this distinction was not always well-preserved: notice how both sorcerer and sorceror occur, as do both imposter and impostor, with sorcerer and impostor now the more accepted or common renditions of those pairs. Plus even though ‑er was usually a native-English ending, Latin also contributed some ‑er words of its own to English, like neuter, integer, dexter, sinister, super.

Because of how 17^th century orthographers wanted to make words look more like Latin when writing them, eventually some of our words that were normally ‑er even up north got reworked into ‑ar words instead, consciously or unconsciously tying them to ‑ar Latin forms whether real or imagined.

Examples already mentioned include liar and pedlar, but there are many more. A lot of words had their standardized spellings changed into unhistorical forms during this time, famously including ones like island and debt. In its article on English Spelling Reform, Wikipedia states:

From the 16th century onward, English writers who were scholars of Greek and Latin literature tried to link English words to their Graeco-Latin counterparts. They did this by adding silent letters to make the real or imagined links more obvious. Thus det became debt (to link it to Latin debitum), dout became doubt (to link it to Latin dubitare), sissors became scissors and sithe became scythe (as they were wrongly thought to come from Latin scindere), iland became island (as it was wrongly thought to come from Latin insula), ake became ache (as it was wrongly thought to come from Greek akhos), and so forth.

As the last sentence cited above shows, mistakes were made during this time. And while it is arguable, beggar seems to have been one of these. The OED is not completely certain of this, as they say “probably imitating”. In particular, they say of beggar:

The spelling in -ar has been occasional from 14th c., but the usual form in 15–17th c., as an ordinary agent-noun from beg, was begger: see ‑ar³.

Which states:

‑ar, suff.³, casual variant of ‑er, ‑or, suffix of agent, and ‑er suffix of comparative. Very common in north. dial., as syngar singer, forebear predecessor, soutar sutor; hear higher. And in modern Eng. in beggar, liar, pedlar. Probably imitating the refashioned scholar, vicar, pillar for earlier scoler, viker, piler: see ‑ar¹, and ‑ar², above.

The ‑ar¹ case contains such words as altar, collar, pillar, solar, lunar, regular, similar, and so includes words that came to us both directly from Latin and via Norman French, and in English sometimes showed up as ‑(i)aire as in ordinaire and which are related to the ‑ar² case.

The ‑ar² case is words we refashioned from Old French ‑ier, but which ultimately have the same origin as ‑ar¹ words. These include words like bursar, mortar, vicar. Many of these used to be ‑er words in English, but got redone in a “more Latin way” to turn them into their current ‑ar forms.

The histories behind ‑er and ‑or words in English are both of them even more complex than those of ‑ar are.