Learn English – In old books, why is the first letter of the word after the exclamation mark not capitalised

I think if you ask the experts who would claim that they know what the “correct” way to punctuate something is, they would tell you that a sentence may only have one terminal punctuation mark—that is to say, neither “?!” nor “!?” is correct.

So, no matter what order you use, you’ll never please those people. The Corpus of Contemporary American English has 3742 examples of “?!” and 1197 examples of “!?”. Clearly both orderings enjoy substantial usage, although it does appear that “?!” enjoys a majority of usage, probably because most sentences that get the double punctuation treatment are syntactically questions that have an exclamation point added for emphasis.

Edit: I checked in the British National Corpus, and it has 224 instances of “?!” and 121 instances of “!?”.

Punctuation apparently got short names and nicknames in the printing industry when humans set the type, literally by putting small pieces of metal into frames. They needed to say "pass me a ..." and multi-syllable names like "exclamation point" didn't work as well for them as short and easily distinguishable ones.

Something similar happened in the early days of computers, when we had to type a lot of things and often tell each other what to type. Again, short names that didn't sound anything like each other were preferred to long ones. The Unix guys, who like to use words related to magic and wizardry whenever they could, re-animated many of these really old fashioned printer's type words and invented a few of their own. (Here's a list but there are many more you can find - just search for a few of the nicknames and you'll find pages with lists of them.)

As to bang, if you read the comics you'll see that certain words are always written with an exclamation mark, and Bang! is one of them. It gives more emphasis to the word, which is representing a sudden loud sound like the firing of a gun. (Things can fall over with a bang, but it's not the canonical example of the sound.) Printers also called it screamer, I guess because it could change an ordinary sentence like get out to something that was screamed like get out! Like all names, it doesn't need to "make sense" - why is a rose a rose and not a tulip? - it just needs to be memorable, which it is.