Historical trends in 'losing [one's] virginity'
Whatever the early gender-neutral understanding of losing one's virginity may have been, there is a very clear difference between the earliest occurrences of "lost/losing/lose her virginity" and the earliest occurrences of "lost/losing/lose his virginity." Here is a Google Ngram chart tracking instances of "lose her virginity" (purple line), "losing her virginity" (teal line), "lost her virginity" (yellow line), "lose his virginity" (green line), "losing his virginity" (red line), and "lost his virginity" (blue line) for the period 1600–2005:
As you can see, none of these phrases was widely used prior to 1758 (when "lost her virginity" suddenly spiked) and 1759 (when "losing her virginity" did). The 405-year period tracked in the first chart, together with the extremely high spike from 1758, compresses the later results to the point of illegibility, so I break out the same results for the years 1759–2004 below, first for the trio "lose her virginity" (green line), "losing her virginity" (red line), and "lost her virginity" (blue line):
and then for the trio "lose his virginity" (green line), "losing his virginity" (red line), and "lost his virginity" (blue line):
A couple of things are evident from these male and female charts. First, the male chart doesn't register its first blip until 1816, more than half a century after the female chart first explodes; and the male chart only occasionally registers any activity until the 1900s, while the female chart is full of activity from about 1800 onward. Second, even when the male chart shows increasing frequency of usage (from about 1900 onward), the scale of the chart is only half that of the female chart: the male chart tops out at 0.0000024%, while the female chart tops out at about 0.0000050%. It seems clear that even today, "lose/losing/lost [one's] virginity" is twice as common in written English in the context of a female person as in the context of a male person.
When did 'lose/losing/lost his virginity' become a recorded phrase in English?
Looking now at the earliest Google Book matches of "lose/losing/lost his virginity," I find that one of the earliest is from the entry for Pope Gregory (the Great) in A New and General Biographical Dictionary, volume 6 (1761):
He [Gregory] was so rigid in regard to the chastity of ecclesiastics, that he was not for admitting a man to the priesthood, who had lost his virginity, and had the candidates questioned on that head. Widowers were excepted, if they had observed a state of continency for some considerable time.
This description is actually taken (without acknowledgement) from the English translation of Pierre Bayle's Dictionary Historical and Critical (second edition, 1736), where it appears in Bayle's entry for Pope Gregory. But the notion that male virginity was a meaningful concept goes back father still, as these entries from Randle Cotgrave, A French and English Dictionary (1673) indicate:
Puceau : m. A man that hath not lost his virginity ; also, on that with a puff lights a candle newly put out ; also, a kind of Vine.
...
Pucelle f. A maid, virgin ; girl, damosel, mother ; also, the river-pilchard [a fish] ; or a young, or little shad-fish.
So the notion is first memorialized by way of explaining French nouns related to male and female virginity, and next in connection with a translation of Bayle's Dictionary. But the wording of these early English treatments of male virginity present the notion as though they expect the concept to be immediately comprehensible to their readers, and not at all as though it might require further explication, as something outlandishly strange might.
Perhaps the earliest home-grown English instance of "lost his virginity" appears in Thomas Fuller, The History of the Holy War (1639):
Albert of the house of Brandenburg was the last grand master of this order, and first duke of Prussia. He brake the vow of their order, losing his virginity to keep his chastity, and married Dorothy daughter to the king of Denmark. The other Teutonics protested against him, and chose Gualther Croneberg in his room: yea, Albert was proscribed in a diet in Germany, and his goods confiscated, but the proscription never executed, the emperor of Germany being the same time employed in matters of greater moment which more nearly concerned himself.
Another early instance appears in Timothy[?] Wharton, "A Treatise on the Celibacy of the Clergy, Wherein Its Rise and Progress Are Historically Considered," in Edmund Gibson, A Preservative Against Popery, in Several Select Discourses upon the Principal Heads of Controversy Between Protestants and Papists (1738):
I begin with Tertullian, who acknowledgeth, That in his Youth he had been guilty of all the Debaucheries of the Age, and laboured under a total Corruption of Manners. Eustathius was six Times deposed by so many several Synods, for his scandalous and enormous Vices. Heliodorus was deprived of his Bishoprick by a Synod of Thessalia, for writing a lascivious Romance. Epiphanius was inveigled in his Youth by the Artifices and Lusts of the Gnostick Women, as himself confesseth. St. Hierom acknowledgeth he had lost his Virginity, (although it was for many Ages celebrated by the Roman Church in her publick Offices) For thus he writes : I extoll Virginity to the Skies, not because 1 possess it, but because I the more admire that which I want. It is an ingenuous and modest Confession, to commend that in others, which yourself want.
Early texts involving 'lose/losing/lost her virginity'
Although the earliest Google Books matches for "lose/losing/lost her virginity" are not far earlier than Thomas Fuller's "lost his virginity" from 1639, they are considerably more numerous. Here are five instances from before Fuller's History of the Holy War. From Robert Cawdray, A Treasury, Or Storehouse of Similes: Both Pleasant, Delightful, and Profitable for All Estates of Men in General (1609):
As Dinah, when she wandered to see fashions, and thought to feed her fancy upon the daughters of a strange country, lost her virginity amongst the sons of the country: so some men, while they seek to feed and fill their greedy worm of covetousness and ambition with divers pleasures and profits of the world, lose their sincerity amongst them, and so make shipwreck of faith and good conscience.
From a 1620 translation of St. John of Avila, The Audi Filia, or A Rich Cabinet Full of Spirituall Iewells [combined snippets]:
And although, by pennance, a pardon of the sinne is obteyned ; yet reacheth it not, to a recouery of the crowne of Virginity, which is lost. And a poore thing it is, sayth S Hierome, that a virgin who expected a crowne, must be glad of a pardon, for not hauing kept it. As it would be, if any King should haue a daughter, whom he loued much, and whom he kept for marriage, according to her rancke, & when the occasion thereof should present it selfe, this daughter of his should tell him, that she asked his pardon, for that she was not fit for marriage, as hauing vilely lost her Virginity.
From Robert Burton, The Anatomy of Melancholy (1621–1638):
For a woman abroad and alone, is like a Deer broke out of a Parke, quam mille venatores insequuntur, whom every hunter followes ; and besides in such places she cannot so well vindicate her self, but as that virgin Dinah (Gen.34.2.) going for to see the daughters of the land, lost her virginity, she may be defiled and overtaken on a sudden, Imbelles dama quid nisi prada sumus?
From Geoge Sandys, editor's notes to chapter 5 of Ovid's Metamorphosis Englished, Mythologized, and Represented in Figures (1632):
This knowne, she [Ceres] ascends into heaven, and complaines unto Iupiter, who signeth Proserpina's returne; provided, that since her descent she had tasted of nothing: meaning, as fome suppose, if she had not lost her virginity, alluding to the markes thereof in that fruit : because a rape so consummated is no way repairable but by marriage.
And from John Weemes, An Exposition of the Morall Law: Or Ten Commandments of Almightie God, volume 2 (1636):
Lucretia killed her selfe that shee should not be defiled by Tarquinius, but it had been no sinne in her, if shee had not given her consent ; for if a woman that is forced against her will should lose her virginity, then chastity should not be reckoned amongst the gifts of the minde, but onely amongst the gifts ofthe body, as strength, beauty, and health; therefore she was guilty of selfe murther.
Conclusions
In Google Books search results, as represented in Google Ngram charts, "lose/losing/lost her virginity" and "lose/losing/lost his virginity" have surprisingly similar upward trajectories since 1940 or so—except that the "her" chart is on twice the scale of the "his" results. So there is about half as much interest among writers in the concept of "lose/losing/lost his virginity" as there is in the concept of "lose/losing/lost her virginity."
Ngram charts show the "her" chart springing to life in 1858 and remaining almost continuously active from about 1800 onward. The corresponding dates for the "his" chart are a start year of 1816 and a continuous activity date starting at around 1920 (or perhaps 1930, depending on how you read the chart).
However, gender-specific instances of "lost her virginity" and "lost his virginity" go much farther back—to at least 1609 for "her" and to at last 1639 for "his."
The evidence indicates that some and perhaps many English writers (and speakers) have been aware of the concept of a man or boy "losing his virginity" for well over 350 years. Nevertheless, judging from the relevant Google Ngram chart, the frequency of mentions of the phrase (and its variants) has been growing consistently only since about 1930. Both the "losing his virginity" group of phrases and the "losing her virginity" group of phrases became far more frequent in the second half of the twentieth century than they had been in the first half, and as of 2005 were (according to Ngram) at their highest level of frequency in at least 200 years.
Best Answer
To come from point A does mean that you start a journey at point A and finish somewhere else.
However, to be from point A is to state that point A is your current (and more-or-less permanent) residence; you could think of it as being the end of your journey to find your own place to live. To have arrived at that residence in point A "by way of" point B is simply saying that prior to living in point A, you lived in point B; so the usage is the same, really, once you sort out the directionality.