Learn English – What’s the word for the use of the wrong prefix or suffix

You're quite right.

-en as a suffix is of Saxon/German origin.
Made of. In German it is may be a bit oldish but still possible to encounter "wollen" as made of wool ("aus Wolle gemacht"), and quite common to hear "golden" (von Gold gemacht). That's for adjectives.
Plural. For noon, you would have *-en" as a Saxon plural oxen (German Ochsen) or children (Kindern, old-german and dutch Kinderen), bretheren (German: "Brudern", old G. "Bruderen").
In dialectal english you can still find "Hosen" (German "Hosen" => kind of trousers), "Shoon" (G: Shuen), "Housen" (Häusen), "Treen".
Diminutive. Vixen also (German Füchsin : little female of the Fuchs/Fox) is of Saxon origin but this time the "-en" is not the mark of plural.

en- as a prefix this time, is of Latin origin (via French). in- would be directly Latin (sometimes both forms survive : enquiry-inquiry, incase-encase). Strangely enough it also conveys a meaning of "make" as in "turn into" : enslave, enlarge (élargir) enrich (enrichir), enable, endear, endanger.
Some other en- prefixes can be traced back to Greek, enthusiasm, endemic, energy.

However, I don't think there has been a lot of inbreeding between latin "en-" prefix and words of saxon origin. And surely no (legitimate) -en suffix to non saxon words. VAX (meaning Virtual Address eXtension because it was an evolution from 16bits VA space to 32bits VA space) can probably not apply to a Saxon origin ;-).

Many sources: Oxford Dictionary of English Etymology - 1966 Ed.

The Latin suffix -io, stem -ion-, is usually added to the supine stem ( ~= past participle stem) of a Latin verb, in order to form a noun of action (meaning "x-ing"). It is sometimes also added to other stems, but usually not. The verb vocare ("to call"), present stem voca-, supine stem voca-t-, gets vocatio ("calling"); the stem of that noun is vocation-, which is what our suffix was based on.

Because the regular suffix for the supine stem / past participle is -t- in Latin, we often see -tion; but compare also miss-ion, fus-ion, cohes-ion, inflex-ion, etc.. The same suffix can also be found with non-supine stems, though less frequently so, like rebellion, legion, etc.

Now how can we predict what comes before -ion in English? I don't have a neat system based on tight rules, but perhaps I can give a few general directions.

What we could do is try and think of a cognate verb, as you have been rightly doing. Does this verb end in vowel + t or vowel + te? If yes, it is probably based on a (real or reconstructed) Latin supine stem; then -ion can come right after the t, as in inhibit, complete, migrate.

And what if the verb does not end in vowel + t(e)? Then you need to know whether or not its stem is a supine stem in Latin. There is no way to know this except by consulting a Latin dictionary. But there are some unreliable rules.

Irregular supine stems most often end on s or x, so verbs like flex (from verbal stem flect-, "to bend") and fuse (from verbal stem fund-, "to pour") contain the supine stems flex- and fus-, leading to inflexion and fusion. But vex- and pos- are not supine stems, so vexation and position, from supine stems vexa-t- and pos-i-t- (the i is probably a fused theme vowel).

There are many supine stems that end on -pt- or -ct-, like act- and rupt-, leading to action and eruption.

Verbs ending on -nt are nearly always based on the present-participle stem -nt-, and so cannot get -ion right after nt; but there are a few supine stems on -nt- as well, mainly vent- and tent-, leading to -vention and -tention.

But there are also some nominal ("noun") stems that can get -ion, as mentioned before, like mens, "mind", stem ment-, leading to mention; and dens "tooth", stem dent-.

Cohabitation is an apparent exception. But both habē-re ("to have") and habita-re ("to inhabit") exist as verbs. Habitare was actually formed based on the supine stem habit- from habēre; then a new supine stem was formed based on this new verb, habitat-. And so we have pro-hibi-t-ion and co-habi-t-at-ion.

Normally it isn't very functional in Latin to make a new verb based on the supine stem of another, since you already have the original verb; but it is sometimes done, often to add a sense of frequency or intensity, and so two supine forms may come to exist. Usually a is then added after the supine stem to turn it into a new verbal stem, because a indicates a causative verb, i.e. a verb that means not "to do x" but "to cause someone to do x" (with nominal stems, "to turn into x, to affect with x", or simply to turn any nominal stem into a verb). So fugere = "to flee"; fugare = "to make flee, to drive away"; donum = "gift", donare = "to present as a gift" (or simply "to give").

Cf. haerē-re ("to stick") => hae-s-us (past participle, "stuck") => hae-s-ere => hae-s-it-us => hae-s-it-are => hae-s-it-at-us => hae-s-it-at-io =>> hesitation. Notice that the same supine suffix has been added thrice: s, it, at (the t was turned into an s after certain verbal stems; the i and the a are theme vowel and causative vowel, respectively). And so we have co-he-s-ion and he-s-it-at-ion, both from the same verbal stem haer(e)-.

So it is theoretically possible to add -ation after any supine stem; but that is usually not done, because just -ion is shorter. After any stem that isn't a supine stem, just adding -ion is unusual, so we will usually have to turn it into a (real or hypothetical) verb first with -a-, then add the -t-, then -ion — i.e. we have to add -ation. But, as mentioned above, there are many exceptions, where -ion can be added directly to non-supine stems; however, this is a fixed set of words that mostly already existed in that approximate form in Latin.