One problem is that the entire concept of "part of speech" is very old. How we use it in English, especially in dictionaries, goes back to the study of Latin and Greek. In this view of English grammar "adverb" is the catch-all category where everything that doesn't fit into one of the other traditional categories ends up. (The others being noun, verb, adjective, pronoun, preposition, conjunction, and interjection.)
Now there is no one, true description of any language (except perhaps constructed languages such as yours). There are merely alternative or competing descriptions which appear over time as more independent analyses of the language are undertaken. Such descriptions or analyses may be called "grammars".
Most (but not all) grammars include a concept of word class under one name or another. So one problem is that "part of speech" has two meanings. One is the specific set of eight categories from the classical languages, the other is as a synonym for word class, which is a lot looser.
So all your example words are adverbs under this older stricter view of parts-of-speech, but their qualities and quirks can be much more thoroughly investigated in newer ways. And various new ways will have various new terms for the classes they put these various words into.
Unless you are inventing a new language specifically to embrace the classical parts of speech you don't have to worry in which they belong, but if you are inventing a new language to learn more about how language works then it will be worth your time reading up on the many newer grammars and language descriptions and analyses.
I had a discussion about this recently. We all agreed that Yes and No were minor sentences when used this way. We also hit Wikipedia trying to decide - plus we're not linguists, so its not like our decision was authoritative. But here's our reasoning:
Definition: "A Minor Sentence is one that does not necessarily have a main verb in it, but which can be understood as a complete unit of meaning." When used as you do in your examples, the minor sentence becomes part of a compound sentence.
We went back and forth on a few of the listed other choices, and decided that adverbs were out since Yes isn't modifying the verb directly, and interjections were only valid if the Yes or No was used as a response such as:
Person A: "I've just secured us two tickets to the basketball game tonight."
Person B: "Yes!" (indicating excitement).
As to why opinions are divided, well... English is a bit of an anarchical language. It seems to be human nature that unless there's an established set or rules or an authority to base your arguments on it's hard to convince people to come to agreement.
Best Answer
They are all nouns. Regardless of the part of speech of the word being referred to, it is a word, and a word is a thing.