This is a touchy and complicated issue. There is no simple answer. I'm gay and I probably would be far more offended by the use of "retarded" than by the use of "gay" that you're describing. The problem is not if you are offending anyone but if you might offend someone, and where and when. I personally don't mind people using the word "gay" that way, in great part because I think it's stupid, but not directly harmful; I think that is more damaging to the person saying it than to the concept of being gay. (Equating "gay" to something bad won't make being gay bad. South Park uses the word "gay" that way. When Cartman calls lame things "gay", Stone and Parker are mocking Cartman, not gay people.) I'm offended by the use of the word "retarded" for the same reason: as I am not retarded, if I were not offended, I'd be latently approving of the word being used that way.
But, in short, by using these words, you're always in danger of offending people, even if you haven't so far. Imagine: you meet someone you're getting along with. Let's call them person A. That person is with a friend of their own, let's call them person B. Your friend is with you. Your friend uses the words "gay" and "retarded" the way you described. Person B is offended by these words, either because a relative of his is retarded, or gay. Person A can offer you a job, or a piece of advice, or help you move, or anything of the sort. Person B later tells person A, "I don't like the sort of people that guy," (meaning you), "is hanging out with." Person A is now far less likely to help you.
Is it really a risk you want to take? From your question, it appears you are rather young. That leads to another issue: do you want to get used to that language? If using those words that way is not an issue now, do you want not to notice it when it becomes an issue to do so?
Think of bicycles and helmets. Someone might say they don't see the point of wearing a bicycle helmet because they've never come close to having an accident, but they wear them anyway. Maybe the two are related. Maybe the type of person who is so careful they've never come close to have an accident is the same type of person who wears a helmet. When you're in the process of falling down, it's too late to start wearing one.
You've never had a problem with that type of language and you probably won't until it's too late to do anything about it.
I don't have much advice to give, just a little. Don't try to change others. It generally fails to change anyone and alienates those who try to make others change. But don't change yourself either. You seem to think there's something wrong with using these words this way, despite not seeing much evidence that there is. My advice is: don't wait for the evidence to come along. Stay wary of that kind of language and don't start using it. If one day you run into a situation where it's clear that you're right, it will be too late to do anything about it. And if you never run into such a situation, what do you have to lose by being ready for it? The fun of using a few offensive words? I think it's a pretty good deal.
Best Answer
Both gay and homosexual can be used for both sexes, and they mean mostly the same thing; the differences are subtle.
Probably because the word lesbian exists as well, one is slightly more likely to refer to men when using the word gay; but note that it is very often also used for women. When used as a noun, gay seems to refer to men a bit more often than as an adjective, although it is still very often used to refer to both men and women. It rather depends on context. The word homosexual seems to be completely gender-neutral.
Gay is more informal: homosexual is more formal, and so it is more likely to be used in e.g. medicine and biology. As a consequence, using homosexual can sometimes sound a little bit as if you were describing patients, as if it were some mental illness. This effect is not very strong or ubiquitous, but it is sometimes there. It is stronger when used as a noun (an homosexual, homosexuals). Note that gay is currently in the process of being used more in formal contexts too.
The word homosexual is older. Because homosexuality was long treated as a disease or at least as undesirable, some of this old negativity still clings to the word homosexual, especially as a noun. It is as if you were referring to the past, although this effect, too, is not that strong.
I think this is also the reason why the word gay came to be used (around the 1960s? earlier?) to refer to homosexuals: they wanted a more positive-sounding word. You will find that most gays will mainly use gay, except in a scientific context; and even there, homosexual as a noun has become almost impossible. If you were to say homosexuals, and you weren't over 50 or a bit reactionary in general, I would think you were joking.