Phrase Usage – Me Neither or Me Too?

It is impossible to answer this definitely without knowing a great deal more about the relationship between you and your interlocutor, but this may be what happened here:

You addressed him with a statement which is ordinarily used by children (and quite young children, at that) to adults they admire. In effect, you adopted the role of a child.

He took this to be intended as a jest, and “played up” to your initiative by assuming the role you assigned him: he responded as an adult (and a particularly elderly adult) would to a child, employing an indulgent and perhaps patronizing tone and the stock language of a stock 1930s or 1940s grandfather.

The word whippersnapper is particularly dated; I cannot believe it has been used otherwise than ironically since the early 1960s. I think for most Americans my age it is indelibly associated with Walter Brennan, the character actor who played the cranky but good-hearted Grampa Amos McCoy on the television program The Real McCoys (1957-63). —Google Ngrams (which doesn't handle hyphenated words well, so I've used the unhyphenated spelling)

Yeah, it does have a slight negative connotation. But it's not that voice suggests loudness.

Whenever anyone makes a point of distinguishing between saying something and thinking it, one or the other is being criticized, if only by implication. q.v. "You're just saying that."

The phrase "to voice the opinion" is a deliciously subtle example, because it contains within it the distinction that someone both had an opinion and expressed it. It is otherwise a synonym for "said" or "expressed" or "opined" or "suggested" or "proposed":

User123456 in several comments on meta recently proposed that creation of a tag should always be discussed on meta first.

So the choice to use the idiom "voiced the opinion" is significant. It has a chilliness to it. It doesn't make plain exactly what the complaint is, but has a slight asperity to it, as if it were a mild version of "felt the need to tell us".