Prepositions – Using ‘In’ or ‘At’ Regarding Front Side

What a neat, thoughtful question. In my writings, I've only ever used therefore, thereby, and therein (with the expression "Therein lies the problem.") I think it's the same for many other average Joes :) I wouldn't consider someone who employs the other ones when talking "insane," just quite quaint (in a good way).

Google Ngram colourfully conveys that, while most its siblings have been steadily obsolescing for the past three centuries, thereby stablized to a comfy plateau in the mid-19th century and only in the last decade does it show a minimal amount of decline.
Thereof used to be quite a favour among writers but is now even less common than thereto, which itself is pretty rare but had seen a resurgence in use in the '50s.

It's only logical to compare this cohort to their vis-à-vis: The "here-" adverbs. Even the most popular hereafter has always been much less prevalent than even thereof. (Note the extra zero to right of the decimal point on the y-axis.) The Here Family is even more formal and archaic, as a whole, than the Theres.

So you were right-on in your assumptions. Last productive since centuries ago, (t)here + prep. are now very limited constructions. They're once-thriving dynasties that mostly came into existence in between the 9th and 11th centuries, just before Old English began to evolve into Chaucer's Middle English.

The last progenies to the families were therefrom (1250s), hereat (1550s) and herefrom (1590s), which all practically died out a couple of centuries after invented, as people started to slice these concise words into separate and more flexible units or simply opted for other prepositions altogether, as indicated by the Oxford Engl. Dict. Moribound, they survive on linguistic respirators such as officialese and legalisms.

When I see questions like this, I like to point out that this on-or-at dilemma plays out in other ways, not just with the word side. Both prepositions have a host of meanings, some overlapping, some not. Context is everything!

Sometimes swapping the preposition will cause a sharp change of meaning:

The children were at the table.
The children were on the table.

Please put my slippers on the foot of the bed.
Please put my slippers at the foot of the bed.

(In those examples, on means "atop", while at means "nearby".)

Other times, they can't be swapped; one preposition fits very naturally, while the other creates a seemingly nonsensical expression:

The mouse is on the mousepad.
?-The mouse is at the mousepad.

Stick the stamp on the envelope.
?-Stick the stamp at the envelope.

Dave and Janet are at the movies tonight.
?-Dave and Janet are on the movies tonight.

Dave and Janet should be here at ten o'clock.
?-Dave and Janet should be here on ten o'clock.

Then there are idiomatic uses, where one preposition is more fitting than the other:

During the last minute of the game, the fans were on edge. When the buzzer sounded, they finally felt at ease. (Or, if the other team won, the fans might have felt at a loss).

But sometimes, the difference between the two is minimal, because one of the several meanings of at (and on) is "in a particular place" (which is why you were having problems with the word "side"):

Draw a small 'X' on the center of the circle.
Draw a small 'X' at the center of the circle.

The fish seem to be biting on that side of the pond.
The fish seem to be biting at that side of the pond.

Which is better in those situations (such as Carter's excellent side of the road examples)? In those cases, it may just boil down to a matter of personal preference. Then again, there could be a clear-cut favorite, depending on the context:

During that difficult time, she was standing at the side of her brother.
During that difficult time, she was standing on the side of her brother.

Here, either on or at could be used, if the sentence is merely saying that she was physically located near her brother's left arm. If the sentence is meant to suggest emotional support, though, then that would be better conveyed with the idiom at his side. However, if there was a feud in progress, and the brother and sister were allies, then on his side would be the best way to express that.

Lastly, you would always use "on the side" is when alluding to a side dish:

At lunch, I ordered a turkey sandwich, with slaw on the side.