Learn English – unstressed syllable in the middle of the word = always schwa sound

It is a real pattern. English vowels in fully unstressed internal syllables tend to be reduced to either schwa or /ɪ/, and many American English speakers have the "weak vowel merger" which means /ɪ/ is not distinguished from /ə/.

But it does have a number of exceptions. I doubt I've covered all of them below; you should just consider the following listed categories of exceptions as examples.

Vowels in syllables with secondary stress don't reduce to /ə/

First, I should discuss the concept of "secondary stress". Depending on what you meant by "unstressed", you might have intended to disregard words with secondary stress already, but it's somewhat difficult to prove that a syllable doesn't have secondary stress (which I guess might be one of the reasons to avoid using the concept in theoretical explanations of English pronunciation).

As far as I know, there is consensus that a syllable with secondary stress cannot contain a reduced vowel, any more than a syllable with primary stress can. But there doesn't seem to be consensus about how to determine whether a syllable with a non-reduced vowel has secondary stress or is fully unstressed. E.g. John Wells, in his "strong and weak" phonetic blog post, says:

Some analysts (particularly Americans) argue [...] that the presence of a strong vowel is sufficient evidence that the syllable in question is stressed. In the British tradition we regard them as unstressed.

It seems possible to argue that secondary stress prevents an internal vowel from being reduced in the following words:

• employee, which Araucaria brought up in the comments (it's related to the verb employ, which always has primary stress on the second syllable. Some people have primary stress on the second rather than the third syllable of employee. The OED transcriptions are "Brit /ˌɛmplɔɪˈiː/, /ᵻmˈplɔɪiː/, /ɛmˈplɔɪiː/, U.S. /əmˌplɔ(ɪ)ˈi/, /əmˈplɔ(ɪ)ˌi/, /ˌɛmˌplɔ(ɪ)ˈi/, /ˌɛmˈplɔ(ɪ)ˌi/")

• relaxation (it's related to the verb relax, which has primary stress on the second syllable. The OED transcriptions are "Brit. /ˌriːlakˈseɪʃn/, U.S. /rəˌlækˈseɪʃ(ə)n/, /riˌlækˈseɪʃ(ə)n/")

But if you don't think these words have secondary stress, then they are exceptions to the rule of reducing fully unstressed word-internal vowels.

Vowels (ususally) don't reduce to /ə/ before other vowels

Reduction to schwa doesn't usually apply to vowels before other vowels. "Extraneous" has /i/ in the fully unstressed second-to-last syllable, and "annual" has /u/ in the fully unstressed second-to-last syllable. "Shadowy" is an example (also an example of the next category of exception) with /oʊ/ in the middle, fully unstressed syllable.

Originally word-final vowels don't always reduce to /ə/ in derived words (compounds and some suffixed forms)

Reduction to schwa also does not apply to word-final vowels in standard English (although it does in some regional accents). Words like pretty and grotto end in fully unstressed, non-schwa vowels.

When a word like this is used as the first element of a compound, or has a certain type of suffix attached to it, the fully unstressed non-schwa vowel may be retained and not reduced any further.

• compounds: For example, I have /i/ in "candyman".

• suffixed words: I generally pronounce "pettiness" (and other words ending in -iness) with /i/ in the middle, unstressed syllable, although I can see this being phonetically reduced to [ɪ] or [ə] in "fast speech".

Word-final vowels are not always kept unreduced after suffixation; it depends on the identity of the vowel and the suffix. E.g. reduction of original word-final /i/ to /ɪ~ə/ is necessary in words ending in "-ily" and "-iful" ("happily", "beautiful"); however, reduction of /oʊ/ is not necessary in these contexts ("narrowly", "sorrowful").

Originally unreduced vowels in word-initial unstressed syllables don't usually reduce to /ə/ when a prefix is added

Some people might analyze some of the words in this category as having secondary stress; I don't know.

In originally word-initial closed syllables in words from Latin

In Latinate words, reduction to schwa is fairly uncommon in word-initial unstressed closed syllables. (Compare "bacterium", which usually has unreduced TRAP in the first syllable (/bæk/), to "basalt", which usually has reduced schwa in the first syllable (/bə/). Or "October" with /ɒk/ vs. "occur" with /ə/.)

Syllables that were once word-initial may become internal due to prefixation. E.g. the unstressed first syllable of "bacterial" has an unreduced vowel /æ/, and it remains unreduced in the related prefixed word "antibacterial". So this is one possible source of word-internal unstressed unreduced vowels.

However, in some cases, the presence of a Latinate prefix is associated with reduction of the following vowel, even when that vowel is in a closed syllable. Often, this is associated with stress changes and is more-or-less unproductive as a process to derive new words in English. You gave the example of "infamous", related to "famous"; likewise "monstrosity" and "demonstrate", despite being ultimately based on the same Latin root, show different levels of vowel reduction.

In originally word-initial open syllables

There are also miscellaneous odd words like "tattoo" that have unreduced vowels in word-initial open syllables; this is not affected by prefixation, so there are derived multisyllabic words where the unreduced vowel is in a word-internal syllable (e.g. "untattooed" has /æ/ in the middle, unstressed syllable).