Keep one's hair on indeed means
To stay calm or, to be patient.
History
As far as I know, it's a colloquial British English idiomatic expression, urging the other party not to lose their cool. However, it seems to have spread across the globe and is widely used across the US, Australia and other English speaking countries.
Acc. to Google Ngrams, the phrase first appears in 1868-69. To keep your shirt on has the same meaning and tone but doesn't seem to appear before 1870-71, according to Google Ngrams and in 1904 according to Etymonline.
NOTE: The expression "keep your hat on" predates them both to the year 1804.
Extended Explanation
(Disclaimer: This is some sheer guesswork, putting two and two together.)
The idiom seems to be constructed from that fact that one might lose their hair due to stress (check this), or even might pull it out in exasperation, anger or frustration.
The Ngram results definitely indicate that keep your hair on is currently more popular than keep your shirt on". And for what its worth, these expressions have nothing to do with to let your hair down.
Not only is it common, it even has a dictionary mention, however this mentions Say when rather than Tell me when and that's certainly what I have encountered most often (and use myself).
when, adv. (conj. and n.)
2. In an indirect question or clause of similar meaning: At what time; on what occasion; in what case or circumstances. Also ellipt.
say when, colloq. formula used by a person pouring out drink for another, to ask him to say when he shall stop; also ellipt., as a reply to this formula.
[OED]
That is, in answer to the colloquial formula "Say when", a person might answer "When!" as a humorous answer to being told to say that, as well as indicating "Now is when I want you to stop".
Best Answer
By request from the comments: here is a link to a Language Log post that, among other things, explains the origin of the phrase.
The explanation above taken from
Nigel Rees, Cassell's Dictionary of Word and Phrase Origins (2002), p. 90