Why it’s wrong
Yes, it’s grammatically wrong. But your way of thinking about English grammar is right in a very profound way: wondering if singular and plural could be exploited to suggest some subtle distinction.
Here’s why it’s wrong. The pronoun their calls for a plural antecedent. Hair is singular. Also, movement is singular, so it has to be grammatically tied to an individual hair. If you tie it to the mass of hairs all together, then it suggests that the mass of hairs all together (what we normally call “hair” with no determiner) has one movement as a whole.
How to do it right
There are ways to use singular vs. plural to indicate what you have in mind. Here’s how you’d do it:
… each of his brown wavy hairs had a movement of its own…
The singular word each gives the singular movement something to attach to, and hairs is plural. This makes it clear that the sentence is talking about many movements, not just one.
You could also indicate separate movement of each strand of hair like this:
…his brown wavy hairs had movements of their own…
You are right that logic often trumps rigid grammar rules, leading a reader to interpret a sentence reasonably when too-strict application of grammatical regularity would lead to clumsiness. But since the language provides a straightforward way to indicate the intended meaning in this case, there’s no pressure to bend grammar.
The inevitable complexity
Their can take a singular antecedent when it stands for a person and you’re trying to avoid indicating “their” gender. However, to many people's ears, this usage sounds sloppy or ungrammatical, or at best informal, because their calls for a plural antecedent. There is currently something of a war going on in the language right now, to allow their to refer to a singular person as antecedent in order to avoid sexist language. Perhaps after that war is won, their will broaden to allow singular antecedents of all kinds, but today such a development is beyond the horizon.
The difficulties with your examples arise partly from the use of the verb come as well as with the tenses.
It's clear that you are speaking in Example 1 after you had gone to the party. You are no longer at the party. Therefore you need to say that your friend went to the party. If you were still at the party, your friend would have come to the party.
That's to say, you use come to mean towards me and go to mean away from me or in some other direction.
This is slightly complicated in your examples because you are imagining yourself at the party when you write them.
To avoid this difficulty, you might write:
If I had not gone to the party, I would not have met my friend who arrived there/showed up there/turned up there/was also present.
Example 2 is more complicated. You are writing it before you decide whether to go to the party:
If I did not go to the party.....
You conclude that:
I would not meet my friend
which is perfectly correct
But because you are not at the party, which you may or may not attend, you need to conclude:
I would not meet my friend who is going there/going to be there/will be there/will also be there, etc
But you cannot say: who came there because you are not there yourself and because this uses the past tense to describe an event that lies in the future for you.
Best Answer
Always refer someone's hair, in its totality, as a single unit.
Never refer to someone's hair, in its totality, in the plural.
Refer to a subset of someone's hair as hairs, a hair, or as hair.