I initially thought it was French, but actually it's a draw between French and Latin. There have been many studies on this topic, which came up as you would expect, with different percentages, but the conclusion remains the same.
There is a paragraph about this topic in the Wikipedia article, and another dedicated Wikipedia article that lists English loanwords by country and language of origin. One interesting conclusion in the first article is that if you consider only very common words, the Anglo-Saxon proportion is more important, which is a clear indication of English's Germanic substrate.
You know, if you had asked the question a couple of centuries ago, it would have been French alone. The reason why Latin has caught up is because many scientific and medical terms are coined after Latin. Remarkably, the second largest contribution of Latin is actually the result of the introduction of Christianity from the 7th Century onwards and the Roman occupation comes only in third place.
The French influence really began with the Norman invasion of 1066; but that was only a start, as it was followed by a continuous influx of French immigrants in England throughout the Middle Ages and even after. Indeed, a few of the early Norman kings rarely set foot in their English kingdom and couldn't speak English. Saxon English nearly became an underground language. However, many of these immigrants came without women, and once they'd intermarried with the locals, after a few generations their descendants could not speak French any more.
All considered, this was only too fair because the Norman people were actually of Scandinavian origin and they had made French their language for the same reason: Norman armies had landed on French coasts with very few women. The date of the establishment of the French Dukedom of Normandy is 911. Compare this to 1066. A mere 155 years had turned the North men into the Normans.
Other notable contributions to English are Celtic (see below), Old Norse (due to the Viking invasions), Greek, Dutch (with William and Mary and the Dutch wars) and a variety of other languages especially from the former colonies.
Language origin specialists theorise that the proportion of loanwords in a language is a function of both the length of exposure to and the social status of the borrowed language. For this reason, there are comparatively very few Celtic words — whiskey being one of the most prominent ones. One clue to this apparent oddity is the double meaning of Wilisc in Old English, the Saxon word for Celt (⇒ Welsh): it means both "foreigner" and "slave".
Best Answer
The term is loan words; I am not aware of any language-specific term other than Latinate words. Loan is a bit of a misnomer, though, for words that have become part of the English lexicon -- we're never giving them back.