Samuel Johnson's Dictionary of the English Language (1756) expresses no doubt that the term comes from the word rub:
RUBBER, s. [from rub]
One that rubs.
The instrument with which one rubs. Swift.
A coarse file. Moxon.
A game ; a contest ; two games out of three. Collier.
A whetstone.
Twenty years earlier, John Kersey, A New English Dictionary: or, A Compleat Collection of the most Proper and Significant Words, and Terms of Art Commonly used in the Language (1739) gives only two rubber-related definitions:
A Rubber, a Rubbing-Cloth.
To play Rubbers, or a double Game at any Sport.
Earlier editions of Kersey from 1706 and 1720 have no entries at all for rubber; and Elisha Coles, An English Dictionary, Explaining the Difficult Terms that are used in Divinity, Husbandry, Physick, Philosophy, Law, Navigation, Mathematicks, and other Arts and Sciences (1717) likewise has no entry for rubber. At least in their early editions the pre-Johnson dictionaries focused on difficult words, and it is highly probable that rubber was widely used in England prior to its appearance in the 1739 Kersey dictionary.
Though both Kersey (1739) and Johnson (1756) offer definitions for rubber in the context of games, neither includes a definition for latex-based rubber nor for anything implying elasticity. This provides circumstantial evidence that rubber as applied to games and sports is not directly connected to what used to be called "India rubber." In fact, the earliest dictionary I have found that mentions "India rubber" is Noah Webster, An American Dictionary of the English Language (1828), where this item appears at the bottom of the entry for rubber:
India rubber, elastic resin, or caoutchouc, a substance produced from the syringe tree of south America; a substance remarkably pliable and elastic.
All of which tends to eliminate the notion of elasticity or bouncing as a possible element the original sense of rubber as used in the context of games—but it doesn't offer much additional insight beyond Johnson's surmise that rubber in games, like the other senses of rubber that he lists, comes from rub.
Two very early instance of rubbers in the context of games appears in Thomas Dekker, "Sloth or The fourth dayes Triumph" in The Seven Deadly Sinnes of London (1606):
Hee [the "Cowardly Generall" Sloth] then gau licenses to all the Vintners, to keep open house, and to emptye their Hogsheades to all commers, who did so, dying their grates into a drunkards blush (to make them knowne from the Grates of a prison) least customers should reele away from them, and hanging out new bushes, that if men at their going out, could not see the signe,yet they might not loose themselues in the bush. He likewise gaue order that dicing-houses, and bowling alleyes should be erected, whereupon a number of poore handy-crafts-men, that beore wrought night and day, made stocks of themselues of ten groates, and crowns a peece, and what by Betting, Lurches, Rubbers and such tricks, they neuer tooke care for a good daies worke afterwards.
and in Thomas Dekker, "Vincents Law" in The Belman of London (1608):
The Dycing ebeator, and the cozening Card-player, walke in the habites of Gentlemen, and cary the faces of honest men. So likewise doe those that are Students in the Vincente Lawe : whose Inne is a Bowling Alley, whose bookes are bowles, and whose law cases are lurches and rubbers. The pastime of bowles is now growne to common exercise, or rather a trade of which some of all companies are frée ; the sport is not so common as the cozenage used in it, which to have it live with credyt and in a good name is called the Vincents Law.
In this Law they which play booty are the Banckers.
He that Betteth is the Gripe.
He that is cozened is the Vincent.
The Gaines gotten is called the Termage.
The Bankers are commonly men apparelled like honest and sub / stanciall Citizens, who come into the Bowling Allies, for a rubbers or so, as though it were rather for sport, then for any gaines, protesting they not whether they win or loose : which carelessnes of theirs is but a shadowe to their pretended knaveries : whilst they are crying Rub, Rub, Rub, and a Great one, In come the spectators dropping one by one, and stand leaning over a Rayle to behold them ; of which oftentimes some simple men that never saw common Bowling Ally before may perhaps be of the number, and is brought in of purpose by one of their owne Brotherhood to be rid of his money : ...
Though Dekker here is describing a crooked game of "bowles," it appear that in honest versions of the game, too, spectators would shout "Rub! Rub! Rub!"—either for encouragement, because "rubbing" (perhaps striking the target pins with the cast bowle) may have been the source of points in the game, or for purposes of calling for wagers. In any event Dekker's description of bowling offers a possible explanation for how rubber (or according to him, "a rubbers") might have emerged in its idiomatic gaming sense.
In explaining how to say "Rubbers at bowls" in Latin, Christopher Wase, Dictionarium Minus: A Compendious Dictionary English-Latin (1662) provides some insight into how the term was understood in English in 1662:
Rubbers (two games) at bowls Gemina in ludo sphærarum victoria.
Best Answer
Its origin is actually from the card game 'rubber bridge' where in a three-competition game one team wins once it scores 100 points or more. If that is done before completing the three competitions, the remaining one is said to be dead rubber.
Dead rubber:
Rubber: (from Etymonline)