Learn English – Why use apostrophe-s to denote possession when using ‘of’

To me, "Friend of Peter" and "Friend of Peter's" mean the inverse of each other.

In "Joe is a friend of Peter", Joe is the active person in the friendship - it describes Joe's active relationship to Peter. Peter is one of the people Joe expresses friendship toward.

In "Joe is a friend of Peter's", Peter is the active person in the friendship - it describes Joe as being the object of Peter's friendship. Joe is one of the people Peter expresses friendship toward.

In most contexts, nothing is being implied about the inverse relationship, although friendship is usually reciprocal. It's usually more a matter of who the speaker knows about the relationship from. So if I'm introducing you to Joe, but we both know Peter, and Peter's talked about Joe, I might tell you that Joe is a friend of Peter's.

The distinction is probably most significant in high school. ;-)

This is perhaps best explained by providing the relevant extract from the ‘Longman Student Grammar of Spoken and Written English’:

The double genitive is a special construction in which either the independent genitive or a possessive pronoun occurs in an of-phrase:

This was a good idea of Johnny’s.

There’s a talk by this lady from Boulder who’s a student of Sandy’s.

The woman who owns Harte’s is a friend of ours.

As these examples show, the main noun phrase typically begins with the indefinite article. In fact, the definite article does not normally combine with the double genitive: *the good idea of Johnny’s is unlikely to occur.

The meaning of the double genitive can sometimes be alternatively expressed by other constructions. Thus, a friend of ours could alternatively be expressed as one of our friends.

Here is a further explanation from ‘The Cambridge Guide to English Usage’:

The double genitive seems to serve two purposes:

emphasis. This is the effect of paraphrasing “not Jo’s fault" as no fault of Jo’s, or turning “our friend” into a friend of ours. The double genitive unpacks the phrase and foregrounds the noun rather than the person. In conversational examples such as That book of Bill Bryson’s is his best yet, the construction helps to adjust the topical focus.

clarification. Clearly a painting of Lady Rich’s and a painting of Lady Rich mean different things. The first (a possessive) makes the painting part of Lady Rich’s collection, while the second (technically an objective genitive) says that it is a portrait of the Lady herself. The duplication of the genitive is thus not redundant but clarifies the fact that the first construction is a possessive genitive.