Excellent question! The short (and rather unhelpful) answer is that while technically, "a couple" does in fact mean two, it is not always used that way in practice and if you ask several native speakers you're likely to get different responses.
"A couple", "a few", "several"... Words like this are used with various intent. In the particular case of "a couple of weeks" I'm (personally) likely to interpret that as 2-3 weeks away. In any other case where you use "a couple", it depends on the circumstances. I'll get a general idea of what you mean, but we won't necessarily have the same understanding of the situation.
Bob and Marie make a good couple.
Okay, that one's obvious. When you're talking about two people in a relationship as a "couple", clearly there are two of them.
I'll see you in a couple of weeks.
As I said before, this probably means 2, maybe 3 weeks (in my experience). I think this is probably the situation in which you're least likely to cause confusion, though obviously that's not always the case since someone corrected you!
These pretzels are delicious! Can I have a couple more?
Assuming these are snack-sized pretzels... Chances are I'm not just asking you for exactly two, right? Generally people use this to mean "give me some more of them" with "some" being indeterminate. The most common response would be to reach into the bag, grab whatever pretzels you would naturally get at a time, and give them to the person. Sometimes, just to be 'literal' and make a joke, I know people who will carefully count out two pretzels in this situation and give them to you. You'd give them a look, and then they'd give you more. So even native speakers are aware of this disparity, and can find humor in it.
If that's not enough, consider the following xkcd comic, where the author makes fun of the ambiguity of "a couple" and such words:
The author also adds mouseover text to his comic, which reads: "If things are too quiet, try asking a couple of friends whether "a couple" should always mean "two". As with the question of how many spaces should go after a period, it can turn acrimonious surprisingly fast unless all three of them agree." ;)
So there isn't a simple answer for you, I'm afraid, but the answer is it's all very dependent on who you're talking to and how they interpret the word. If your friend corrected you then he has a different interpretation--but that doesn't mean you were wrong!
Best Answer
Typical use would be one of the following:
So your analysis is pointing you in the correct direction. The next section explains this in some more detail.
"First of all," has emphasis and indicates the primacy of the first point, especially when there are more than two points. A one-syllable "First," can introduce the first point, but some find it too short to introduce a long concept of several points. The introduction "Firstly,", which is considered less formal, is sometimes used as a compromise when "First of all," is too strong but "First," seems understated.
Second, many people consider "Second," as a more correct or formal method to introduce the second point, but many people use "Secondly,". The phrase "Second of all," is logically incorrect: it would literally be the "second to all but the first", and in that regard, many consider it an egregious mistake. But it can be used for emphasis:
And finally, one could use "And finally," if it were actually the third-and-final point. Otherwise, one would use "Third,". The use of "Thirdly," starts showing the problem with the adverbial since "fourthly, fifthly, sixthly" are strange. Also, "Third of all," starts showing the self-defeating point of a continuous use of "X of all". An actual use of "Third of all," similar to the above "Second of all," would have to be quite emphatic, perhaps to the point of being shrill.
The fourth and final point might look like this. But rules are meant to be broken, and I can imagine a comedic use of emphasis in dialog using "And fourth of all,", perhaps ending with "And finally of all," :-)
And fifth, one might say this.
And my sixth and last point is that, unless its absolutely necessary, numbering one's points eventually becomes distracting.