Learn English – Why are reveries sometimes called “brown” studies

It's apparently somewhat of a private joke, due to the fact that the next show on the air was by the same creator:

In the closing moments of last night's Family Guy Peter Griffin introduced American Dad as "whatever Fox is limping to the barn with."

So, he means “whatever crap will air after this”, but is just humorous self-deprecation.

“Limping to the barn” is then understood as meaning “being of inferior quality”, or “performing badly”, or “tired”… A horse that limps back to the barn has been beaten, or is simply not in good shape altogether. I haven't heard this expression before, though… Other uses found by a web search include:

• Don't lie, by this time on this Thursday you're just limping to the barn at work.
• The running game will be "limping to the barn" at best this week.

You are correct about the current usage. You would not use it to refer to a mental patient in the technical sense, but apparently it was at one time acceptable to do so. I would have thought that the idiom started life in the "angry and agitated" sense, but there is one old (1804) citation that I found via an ngrams search which uses the second sense, apparently without irony. I do not know when the newer sense of the phrase took over, but it was certainly in use about 100 years after that first example.

"Mad" itself seems to have followed the same path. The earliest citations in the OED refer to the "mental disease" aspect, and the "uncontrollable rage" meaning comes later, followed by the colloquial sense of "angry".

It seems to be a common pattern, that a technical phrase for a mental condition gets borrowed for colloquial use and then falls out of usage in its original sense to avoid offense. "Retarded" is another example: the use of the word as an insult and "synonym" for "unintelligent" put it onto the Euphemism treadmill.