Learn English – Origin of phrase “pulling for you”

A Google Books search finds two examples of the phrase from the 1890s and two more from the very early 1900s. From "Johnnie You've Lost," reprinted from the San Francisco Examiner as part of "The Sketch Book—Character in Outline," in Current Literature (February 1890):

Both [bare-knuckles fighters] were winded and blood was flowing in streams. As soon as the men were in their corners the human stake [a waitress to be claimed, along with a gold ring, as a prize by the winner] lavished encouragement on Buckner, and as he was returning to the scratch cheered him with the remark:

"Don't give up, Johnnie; I'm pulling for you."

From Edgar Fawcett, "Old Uncle Vanderveer," in Outing (May 1895):

“So you won't help me, Uncle?” he muttered, sadly though not sullenly. “You won't give me a pull, so to speak?"

“One would suppose I'd never done any kind of pulling for you till now, Larry," came the answer, blent with a mildly sarcastic smile.

From Arthur McIlroy, "Pan-American Babies," in National Magazine (August 1901):

"I like the idea of naming a child for a good man. Many's the youngster that has Bill Daly for a handle to the name he inherited. Their mothers would ask me for the privilege! Think of that, my boys, when the privilege was all the other way. I'd say to myself, 'Bill, old man, here's another little one pulling for you—pulling you in the right direction.'"

And from the "Abbeville, N.C." report in Typographical Journal (August 1, 1902):

The convention committee is grateful for the support it is receiving , and is especially pleased with the letter received from Charlotte. We are glad to know that our neighbors are with us and pulling for us. We have a worthy ally in Charlotte, and with her help will make "The Old North State" heard from.

Two other interesting matches come from the years 1904–1904. From George Ogden, Tennessee Todd: A Novel of the Great River (1903):

"Andrews, as I said, is strong. He's got a strong combination pulling for him, too. The department don't doubt his ability to carry out the contract, but it does doubt the ability of his trains to make as good all-around time, up and down, and all considered, as the boats. ..."

And from "Beta-Epsilon: University of Wisconsin," in The Caduceus of Kappa Sigma (December 1904):

One of the greatest magnets which tends to draw prominence to a Fraternity and give it strength, too, is the athletic fraction of college life. We certainly have the magnet pulling for us "with all fours." Bro. Vanderboom, half-back on the foot-ball team, in the recent elections to the Athletic Board secured the Presidency, and Bros. Archie Pearsons and Art. Miller received prominent positions.

A much earlier example uses the phrase "pulling for him" not in the sense of "pulling on his behalf" but "pulling in direct competition with another for possession of him." From Mary Vidal, The Cabramatta Store; A Tale of the Bush (1850):

"Just so, Mat. Not that I care, I am sure, which [girl] Parry takes ; but 'tis fairly spoiling the young man to have two girls pulling for him, as you may say. However, they say he hasn't been very steady, and there are others have more claim on him now than either of these girls: but, hark! don't I hear the rails down ? Aye, and here she is!"

Unlike in this last example, the sense of "pulling for [someone]" in the other examples is "supporting [someone's] efforts spiritually or materially or both." Less clear is what the the metaphorical pulling originally referred to. Candidates include participants in a tug-of-war (as Josh61 suggests), a horse or other dray animal hitched to a plow or wagon, a person rowing or plying an oar, and a magnet. Of these possibilities, the dray animal and the rower may have been the most familiar images to contemporaneous audiences, but I haven't found any source that convincingly resolves the question.