Attempting an answer à la the formidable Sven Yargs, who's an inspiration!
Under the entry gravy, Etymonline gives:
late 14c. (early 14c. in Anglo-French), from Old French grave, graue, apparently a misspelling of grané "sauce, stew," with -n- misread for -u- -- the character used for -v- in medial positions in words in medieval manuscripts.
The French word probably originally meant "properly grained, seasoned," from Latin granum "grain, seed" (see grain (n.)). Meaning "money easily acquired" first attested 1910; gravy train (1909) originally was railroad slang for a short haul that paid well. Gravy-boat "small, deep dish for holding gravy or sauce" is from 1827.
[Etymonline]
GRAVY TRAIN - "In the 1920s, railroad men invented the express to 'ride the gravy train' to describe a run on which there was good pay and little work. The words were quickly adopted into general speech, meaning to have an easy job that pays well, or, more commonly, to be prosperous. 'Gravy,' however, had been slang for easy money since the early 1900s." From Encyclopedia of Word and Phrase Origins by Robert Hendrickson (Facts on File, New York, 1997).
[The Phrase Finder]
Regardless of whether you think of gravy as a rich meat-based tomato sauce, or, as do most Americans, as a thick brown sauce made from pan drippings, especially good with biscuits, there is no disputing that we all love our gravy. In the early twentieth century, the word gravy came to mean “easy money.” This sometimes meant easy profits resulting from just plain old good luck, but it also could refer to easy, but ill-gotten gains, especially through conning your way into it.Gravy could also refer to any unexpected benefit, or poker winnings, and conversely, to a prison sentence, especially a harsh one, as in in the phrase “dish out the gravy” to mean deliver a harsh sentence.
When you combine gravy with train, then, you get the idea of easy money that keeps on coming in, with little effort on your part. It doesn’t stop, just like a train. We could leave it at that and feel pretty certain that we understand the origins of gravy train, but it turns out that the term actually originated with railroad workers of the 1920’s, who used it to mean an easy but high paying run.
Riding the Gravy Train
Pink Floyd used the phrase “riding the gravy train” in their song Have a Cigar, in regards to, ostensibly, record executives or other music industry hanger-owns urging the band to capitalize on the success of a previous hit for “easy money” and, perhaps, to the record company itself “getting a free ride.”
In the 1970’s, truck drivers had a similar expression. A gravy hauler was a truck driver who would only drive high-paying runs.
[Culinary Lore]
However, the redoubtable World Wide Words remains somewhat uncertain about the origins of the idiom.
The experts do generally agree that the phrase has its source in the slang use of gravy for something easy or cushy, simple to do, or an unexpected benefit. This is recorded in the major references books as appearing slightly earlier (1910) than gravy train (1914). As a result of the digitisation of old newspapers in very recent times, I can take these dates back somewhat. For example, advice to potential advertisers appeared in The Daily Independent of Monessen, Pennsylvania, in October 1906: “If you buy right and then tell an exacting public in a clear, concise way, just as you would over your counter, you are then getting in line for good gravy.”
There is some slight evidence that gravy goes back rather further than that. If it is the source of gravy train, it would have to, because I found the latter in the Courier of Connellsville (also in Pennsylvania) in November 1895, almost two decades before the previously oldest known example: “Johnston claims that Reuben Nelson and another tall negro were in New Haven the night of the escape and that they broke into the lockup. Johnson further states that the next day Kelson laughingly told him that the New Haven lockup was ‘a gravy train.’”
But why and how do trains come into the picture? We don’t know, which leaves the matter in an uncertain and unsatisfactory state.
[World Wide Words]
By far the earliest match for "punch a gift horse" in Google Books search results is from a 1972 issue of National Lampoon, a U.S. satirical magazine that grew out of The Harvard Lampoon, a satirical university student publication. Here is the snippet result that Google Books reports:
NEVER PUNCH A GIFT HORSE IN THE MOUTH OR SHIPS THAT GO BUMP IN THE NIGHT OR GAGA BABRUUUU FREEZIP BLESH DEEDEEDEE GEMP ... SOMETHING LIKE THAT .
Unfortunately, the snippet window that Google Books provides for this excerpt doesn't show any text at all. Still, given even the very limited context in which the phrase appears, it seems clear that this instance is an intentionally garbled play on the much older "never [or do not] look a gift horse in the mouth," which Christine Ammer, American Heritage Dictionary of Idioms, second edition (2013), says reached English around the year 1500:
look a gift horse in the mouth Be critical of something received at no cost. For example, Dad's old car is full of dents, but we shouldn't look a gift horse in the mouth. This term, generally expressed as a cautionary proverb (Don't look a gift horse in the mouth), has been traced to the writings of the 4th-century cleric St. Jerome, and has appeared in English since about 1500. It alludes to determining the age of a horse by looking at its teeth.
In the National Lampoon excerpt, the author leads with "never punch a gift horse in the mouth," garbling the proverb with a common descriptive phrase ("punch [someone] in the mouth"), then mashes together two unrelated idiomatic phrases ("ships passing in the night" and "things that go bump in the night") as "ships that go bump in the night," and then descends into complete incoherence ("gaga babruuuu," etc.). Ha ha.
The next-earliest instance I've been able to find is from an Elephind newspaper database search, which yields this instance from "Without Leonard, the Spotlight Shines Marvelously on Hagler," in the Columbia [Missouri] Missourian (February 8, 1983):
[Marvin] Hagler [at the time the undisputed middleweight boxing champion of the world] was as bleak as the Cape Cod weather last week. It's not as though he would punch a gift horse in the mouth. He just looks that way with his shaved head and goatee.
The Missourian is the student newspaper of the University of Missouri School of Journalism, one of the most highly regarded Journalism programs in the United States.
The expression next reappears as an "honorable mention" entry in the results of "New York Magazine Competition Number 709" in New York Magazine (January 19, 1991), "in which you were asked for Near Misses. Titles, names, phrases, and the like":
Don't punch a gift horse in the mouth.
[submitted by] H. R. Schiffman, Highland Park, N.J.
It is possible that these two and all or most subsequent instances of "don't punch a gift horse in the mouth" ultimately derive from people exposed to the 1972 magazine occurrence. A considerable number of college- and high-school-age young people did read National Lampoon back in the day. More likely, though, the periodic recurrences of the expression during the last three decades of the twentieth century are instances of absurdist minds thinking alike.
Best Answer
From the Oxford Dictionary of Words Origins :
Idiomatic expressions with hobby-horse:
hobby-horse:
get on (one's) hobby-horse:
ride (one's) hobby-horse:
on your hobbyhorse:
(The Free Dictionary)
From History of Rocking Horses: