Do linguists recognize a distinction between the "rules" of a dialect on the one hand and its individual pronunciation "quirks" on the other? Is there a term for this phenomenon? Is it considered merely a variation on regional preferences for certain words over their synonyms (e.g., rubbish vs. garbage), or is there something else at play here?
I would say that linguists in general tend to consider the entire lexicon of each language/dialect/sociolect/regiolect/idiolect as a more or less complete system unto itself. This system may share or not share more or fewer characteristics with other like systems, but they are ultimately separately existing systems.
These systems are all basically phonetically based, though. Writing complicates matters, but it doesn’t change the basics of the systems: they are at their inmost based on spoken sounds.
As an American, you can learn to pick up RP quite easily to the extent that RP mirrors American phonemically. If the underlying phonemes of a word are the same, a simple translation or transposition from one phoneme-based system to another is all that is needed. But in any case where the underlying phonemic structure differs, there is no other way than just to learn by rote. Lieutenant is one such word: it is simply ‘sound-spelt’ differently in the two dialects. The fact that the codified and standardised conventions of our writing system has them spelt the same way makes it less obvious to draw this conclusion, but American and British ‘lieutenant’ are really two different words, just like ‘truck’ and ‘lorry’ are. Of course, it is easier to remember in a practical situation that a word for something in a foreign system is almost, but not quite, the same as in your own system, so there is a cognitive difference. But as far as direct transposition from system to system goes, there is no difference: something is either mappable or it isn’t.
I don’t think there is really a term for this, except perhaps in the guise of pointing out that words that aren’t mappable constitute isoglosses between the systems (which isn’t really what you’re looking for). With all related languages, there are words that map perfectly between systems, and there are words that don’t—the latter category being made up of words that, once you know them, can be retrofitted into almost mapping, and words that do not match at all. But for all languages, this is a continuum.
Between American and British English, the vast majority of the language maps, and a minority of words don’t. Between languages like Spanish and Portuguese or Ukrainian and Russian, a good percentage of the words map, but many don’t. Between Danish and German, a fair few roots are mappable, but the majority of the morphology—and hence of the complete surface forms of words—are not. Between Icelandic and Dutch, virtually nothing is mappable, though many words still fall in the ‘lieutenant’ category of words that are close enough to be recognised as being related to each other. And between Greek and Swahili, virtually everything is neither mappable nor recognisable.
I've read through all of the words beginning with a through c in WS2's very useful list of -tion words, and so far I've found that the vast majority of the words in the -tion family carry a sh sound at the beginning of the final syllable.
The main exceptions to that pattern are some words ending in -stion (bastion, combustion, congestion, counterquestion, countersuggestion, etc.) or in -ntion (attention, contention, convention, circumvention, etc.), which instead carry (in typical U.S. English) a ch sound at the beginning of the final syllable.
There is also a red herring in the form of cation, which is of course not a -tion ending at all, but a cat[a]- prefix attached to the root word ion.
Most significantly to the point of the OP's original question, none of the a through c words in WS2's list carries the sound zh at the beginning of the final syllable.
[One hour later...] I finished reading through all of the -tion entries at MoreWords.com, and the only one that—in my generic U.S. English pronunciation—has a zh sound at the beginning of its final syllable is equation. I should have taken Peter Shor's word (in a comment above) for it.
I did come across the variant spelling kation to go with cation, and I encountered one interesting exception to the -ntion exception that I noted earlier: To my ear, at least, dissention carries a sh sound (and not a ch sound) at the beginning of its final syllable; but dissention is a bit weird anyway because it is a variant of the more common spelling dissension.
Anyway, I am fairly confident that the answer to the question "How many -tion words are there whose last syllable sounds like the last syllable in vision?" is one.
Best Answer
First of all, as Peter Shor says in the comments:
These differences may sometimes be used to represent a difference in pronunciation, but more often they are simply transcriptional variants.
Moving on to the words listed: it mainly depends on the history of each word. This is not really very helpful to know, as it suggests that you have to look up the pronunciation of each one individually. That is in fact the safest option. However, sometimes there are word parts that you can identify that have a shared pronunciation between many words.
One of these parts appears to be the -on ending used mainly in words derived from Greek: although it has no single source historically, these words tend to be pronounced similarly with the sound /ɒn/. This ending is discussed more in the answer to this question: Why do photons and protons exhibit such anomalous behavior? It's not clear, but I'll speculate that one reason why these words are pronounced with an unreduced vowel is because they are more "learned" terms, so people base their pronunciation more on the spelling. Another relevant factor is that these are all relatively recently coined words; Marcus_33's answer to that question says:
The words cotton and mutton come from French coton and mouton respectively. But they entered English a long time ago (during the Middle English period), as indicated by their altered spelling compared to the French, and the placement of stress (on the first syllable). Since the second syllables are unstressed, the vowel became "reduced" here to a schwa sound /ə/. The schwa, when followed by the sound /m/, /n/, /l/, or /r/, may also be transcribed with the subsequent consonant as a syllabic consonant. This is what /n̩/ means.
For the word wanton, the parts appear to come from Old English, and the word has existed at least since Middle English, so it also has undergone many changes in pronunciation over time, among them reduction of the second vowel.
So the advice I would give: if you can tell that the word came from Greek, or if it is the name of a physical particle, it is probably pronounced with /ɒn/. Otherwise, it is probably pronounced with /n̩/~/ən/ (What I mean with the ~ is that you can use either pronunciation interchangeably). And if you don't know for sure, look it up in a good pronunciation dictionary! (Because there are also words that don't follow the rule I gave, like crouton (from French), which the OED lists as /ˈkruːtɒ̃/, but for which I've always used the pronunciation /ˈkruːtɒn/).