In general in play they were ignored or just treated as an abstract language with no further comment.
As to where they came from, here's an answer from Gary Gygax on Dragonsfoot!
As D&D was being quantified and qualified by the publication of the supplemental rules booklets. I decided that Thieves' cant should not be the only secret language. Thus alignment languages come into play, the rational [sic] being they were akin to Hebrew for Jewish and Latin for Roman Catholic persons.
I have since regretted the addition, as the non-cleric user would have only a limited vocabulary, and little cound [sic] be conveyed or understoon [sic] by the use of an alignment language between non-clerical users.
If the DMs would have restricted the use of alignment languages--done mainly because I insisted on that as I should have--then the concept is vaible [sic]. In my view the secret societies of alignment would be pantheonic, known to the clerics of that belief system and special orders of laity only. The ordinary faithful would know only a few words, more or less for recognition.
In other words, it was supposed to be more like religious languages, but wasn't really well thought through. It disappeared in Second Edition and was not missed.
Latin (and to some extent Greek) used to be the lingua franca during the middle ages. Later on, French became the language of diplomacy and nobility. Everyone that mattered(1) speaks a local variation of said language which should still be understandable by another speaker. For example, Quebecois and French or American and English.
So, you could have such a language that all the PCs speak. They should be able to interact with everyone else. Now, make sure that each PC speaks the language from where they will go adventuring. If not, they will have to find a teacher and learn the language. This does not take that much time. You can learn everyday grammar and vocabulary in about three to six months of (hard) study. This is what I do for all my games.
Well, almost all my games. If the game is set in a bounded location, then only those languages that are around said location will be relevant. If I set a game in 14th century Venice, I do not need to worry about the PCs speaking Japanese. If I set a game in the Crusades, you better believe that everyone will learn Latin, French, and Arabic pretty damned quickly if they want to get anything done.
If you have boogly powers (aka magic or psionics or whatever), then learning languages could be done via it.
As a side note, Middle Earth started as a setting to play with the evolution of different languages yet most characters manage to communicate quiet well -- and were delayed at the gates of Moria because of a translation error!
Philology is just cool. And just because it is hard to implement in game setting should not be a barrier to trying it out provided that it enhances the enjoyment of the game.
(1) Why, yes sire, I do have blue blood... What about Peasants? They don't need to speak to outsiders, they need to work harder and pay taxes.
Best Answer
Unless you and the players speak Elvish, you have three options:
To decide which option to use, think about the effects of each:
Think about your experience with hearing other languages. Have you ever heard German spoken? (Assuming you don't speak German.) You could describe the experience by simply saying it sounded like German and assume your listeners know what you mean. If the party members are likely to have heard Elvish before, you can assume they know what Elvish sounds like.
If you want to actually say some Elvish words to give them a feel for it, you have two further options:
A search for "Elvish language" will yield plenty of hits. You could take a few words from someone else's Elvish and say them to the party.
The other option is to make it up yourself. Making an example sound reasonable is more in the scope of linguistics (for which there's an excellent SE site), but I'll give you a few ideas right here. The "feel" of a language is mostly determined by two things: which sounds a language uses and what order they allow them to occur in.
For example, let's say we have a language that uses these consonants: /p t k sh ch y w r/ and these vowels: /u o i/. Next we decide that it allows only syllables that start with a single consonant or one of these consonants: /p t k/ followed by one of these: /y w r/. Let's say words can end in any consonant, and vowels can't clump up. The result allows words like these:
tyiwor, shutosh, wukwop, chokrut
Now we have just enough rules to make up some nonsense words with a coherent feel.
If you're going for this option, making up a little bit of language to use, be careful -- some players will assume that there's a language puzzle here to figure out, and they'll start taking notes and try to work out the grammar. Other players won't do anything with it at all -- one bit of random gibberish sounds much the same as any other. Find out what your players like and do more of that. If they have a lot of fun figuring out a language puzzle, go ahead and make up a whole Elvish language if you feel like it. If they don't care about language stuff at all, next time they meet some elves, just say "They're speaking a foreign tongue. Sounds like Elvish." and leave it at that.