Learn English – Capitalising “river” or such geographical entities when preceded by “the”

Capitalisation to this extent wasn't around in Old English, and I didn't remember any in Chaucer's Canterbury Tales, but it seemed exist in some Shakespeare folios and not others, so it certainly hasn't been around since the beginning of written English.

I found this in an actual printed book, The Cambridge Encyclopedia of the English Language (David Crystal), p67, where the internet let me down. It's in the section about emerging orthography in the 16th Century:

Hart recommended his readers to use a capital letter at the beginning of every sentence, proper name, and important common noun. By the 17th century, the practice had extended to titles (Sir, Lady), forms of address (Father, Mistris), and personified nouns (Nature). Emphasized words and phrases would also attract a capital. By the beginning of the 18th century, the influence of Continental books had caused this practice to be extended still further (e.g. to the names of the branches of knowledge), and it was not long before some writers began using a capital for any noun that they felt to be important. Books appeared in which all or most nouns were given an initial capital (as is done systematically in modern German) - perhaps for aesthetic reasons, or perhaps because printers were uncertain about which nouns to capitalize, and so capitalized them all.

The fashion was at its height in the later 17th century, and continued into the 18th. The manuscripts of Butler, Traherne, Swift, and Pope are full of initial capitals. However, the later 18th-century grammarians were not amused by this apparent lack of discipline in the written language. In their view, the proliferation of capitals was unnecessary, and causing the loss of a useful potential distinction. Their rules brought a dramatic reduction in the types of noun permitted to take a capital letter.

It seems odd that Hart's recommendations on capitalisation should have taken root where his suggestions for phonetic spelling have fallen on deaf ears...

There are sort of two categories here- your first three examples are pronouns, while the latter two are epithets. Traditionally, epithets are capitalized for any individual, be they god, godess, Greek hero, or office worker. Epithets are grand titles that go beyond mere descriptive phrases, standing in for the name of the individual. For instance, an epic might start using "Father of Waters" any place it would name Poseidon.

Capitalizing pronouns is a bit different- from the best of my observations, pronouns are only capitalized when referring to the Abrahamic deity, but it is simply a convention used to respect and set apart God, Jesus or the Holy Spirit from any other male that might be in the setting. It is certainly grammatically acceptable to leave any such pronouns in lowercase- indeed some translations of the Bible do this. It should be noted, however, that if you do so, you may confuse and quite honestly upset members of those faiths.