The most common figure of speech in a situation like the one you describe is to say that something (a conversation, a topic, or especially a joke) is (or goes) over someone’s head, as in sense 1 of this Oxford Dictionaries entry:
Beyond someone’s ability to understand:
the discussion was over my head
If the person who does not understand the conversation is the one saying it, it will often be in a slightly self-deprecating form, such as:
Woah … this is way over my head.
If others say it about him, they are quite likely to come off as fairly condescending.
From the notion of something passing over one’s head come two closely related gestures that are often used together:
- Moving one of your hands quickly over your head (from the front to the back), as if imitating something physically flying past you above your head; and
- Saying, “Whoooosh!”, imitating the sound of someone whooshing past very close to you.
Once again, this is seen as slightly self-deprecating if done by the person who does not understand, and definitely offensive if done by others.
An alternative to expressing that something goes over one’s head is to state that one is out of one’s depth (sense 1.2), an extended sense of a phrase meaning ‘standing in water that is too deep’:
In a situation that is beyond one’s capabilities:
they soon realized they were out of their depth in Division One
I find it difficult to talk in a situation like this—I’m out of my depth
This is a bit more polite and less self-deprecating; it’s also less of an idiom and more of a straightforward collocation. Something like the following is a quite polite way of saying that you don’t understand the topic:
I’m afraid I can’t follow you. I’m a bit out of my depth in this conversation.
Consider sail under false colors and [not] show one's true colors
sail under false colors: To behave deceptively; the “colors” of a ship are its identifying flags: It turned out that the door-to-door sales rep was sailing under false colors and was actually a swindler (The American Heritage Dictionary)
show one's true colors: reveal oneself as one really is, as in We always thought he was completely honest, but he showed his true colors when he tried to use a stolen credit card. (The American Heritage Dictionary)
Best Answer
They are "giving you the cold shoulder" (some history) or even "silent treatment" (more here).
These idioms refer to the behavior, as opposed to the person engaging in that behavior. I'm not aware of an English idiom meaning "someone who gives one the cold shoulder". Related, but not an exact match, would be the word "snob", which applies to someone who feels superiority over others, and displays that feeling through stand-offish behavior.