Learn English – Envy vs jealousy – has the meaning changed

To my understanding, nailed here means outwitted/beaten, but also carries a sense of being struck down as well.

In other words, her conversational wit is kicking him while he's down:

He says he wants to go outside and talk about this (with her).

She says then go ahead . . . (Meaning, go outside and talk about it, but she won't be joining him, so he'll be talking to the air.)

He then accuses his friend of schadenfreude at seeing his misfortune (both at being beaten conversationally, and having his heart broken).

See here: Defeat or outwit an opponent.

One possible origin of "boo" as a term of endearment is a shortening of "booful", also spelled "boo'ful" and "boo-ful", which is itself a baby-talk version of "beautiful".

History of "Booful" for Beautiful

"Booful" as a substitute for "beautiful" is at least 200 years old. Early on, this phonetic spelling variant was used primarily by white authors to infantilize the pronunciation of people of African descent.

For example, "booful" makes an early appearance in the first of James Fenimore Cooper's Leatherstocking Tales, The pioneers : or The sources of the Susquehanna ; a descriptive tale (1823), which takes place in New York State in the pre-abolition 1780s. In this snippet, Agamemnon, a slave, is excitedly relating how a mastiff dog named Brave was killed by a panther while protecting two ladies. He describes Brave's "copse" (corpse) as "booful":

"Oh! de Lor! Miss 'Lizzy and a Miss Grant -- walk -- mountain -- poor Bravy! -- kill a lady -- painter -- Oh! Lor, Lor! -- Natty Bumppo -- tear he troat all open -- come a see, masser Richard -- such a booful copse -- here he be -- here he be."

In John Sterling's "The Onyx Ring", published in Blackwood's Magazine in 1838, an old woman whispers frightening things to a sleeping boy:

She crept to the bedside, and, after seating herself, and making various signs, she began to mutter in a low voice close to the boy's ear. These were some of the words which the Englishman caught: -- "Now, white woman come you very booful much -- tell you take massa's money -- put in um tree -- now she gib you um kiss very sweet much."

In Caroline Hyde Butler Laing's The Old Farm House (1855), the servant Sybil is explaining to her current employer why she needs to visit her previous employer:

"We'se all got to die one ob dese days, de rich and de poor, de missus and de sarvent, and den mebbe when you'se laying dere on de bed in dis booful room, wid nobody 'bout you dat lubs you..."

Expansion of "Booful" Usage to Children

By 1870, we find white characters -- children, or childlike people -- uttering "booful," as in James Payn's Gwendoline's Harvest:

Now, Miss Marion had not asked to be allowed to join her 'booful Dwendoline,' as in her baby-talk she designated her new friend, until she had had that idea suggested to her by Susan herself...

Another example from Louise Chandler Moulton's "How the Girls Got Rid of Freddy", published in 1873 in her Bed-time Stories (for children), in which a mother and a boy are talking:

"Weren't you frightened, darling?"
"Not fightened so much as me hungy. Then me find booful berries. See ! and before I did eat any, something laid me down to sleep."

"Booful" As an Endearment

"Booful" gradually transformed from a pet name mothers called their children to an endearment used between spouses and lovers:

In 1910's Harm's Way by Lloyd Osbourne, Phyllis and her hungover husband Cyril are dreaming big plans:

[Phyllis] "We're going to pack up, poor booful disgraced genius -- and wife (as they add on hotel registers); and we're going to ... take a tourist sleeper to New York.... And if the top all hates him, and the middle is all full, why Booful will begin at the bottom, while Mrs. Booful will wash, and cook, and darn his socks...."

In January, 1916, newspaper readers across the country were entertained by the ongoing Supreme Court case involving a millionaire heir and his mistress. His letters to her were produced in court, as excerpted in this (New York, NY) Evening World newspaper story playfully titled ‘“Boo’ful Baby” Notes of Broker Boil With Love’:

My Dearest Precious: Just received a very swet [sic] letter from my ‘Baby Doll’ — S—U—IS my boo’ful baby. I only wish you were here and I would show you how boo’ful and sweet U—IS.

Another letter entered as evidence:

I know you, love, and want me with you and I soon will be with you. You don’t know, sweetheart, how happy I feel to know you are well and still taking the emulsion. What will my ‘Ittle dirl’ do when her ‘Booful Baby tomes home? I know what he will do. Take good care of your dear little self. Your own dearest and most loving sweetheart, Jack.

Boo* in the Arts

In the first half of the twentieth century, "booful," "boo-boo," and "boo boo boo" began to appear in songs and plays. The song lyrics and scripts are lost to time, but the titles remain in catalogs as evidence.

In the Catalogue of Copyright Entries for 1916, there is an entry for a comedic play called “My boo’ful baby”, by Jules Simonson and Jardin D. Rickman, copyrighted Mar. 7, 1916.

In the Catalogue of Copyright Entries for 1907, there is an entry for a song called “My boo-boo baby”, words and music by Emile H. Naatz, copyrighted May 4, 1907.

In the Catalogue of Copyright Entries for 1938, there is an entry for a song called “Listen to my boo boo boo”, words and music by Robert Lloyd Lingle, copyrighted Jan. 22, 1938.

The 1990s: "My Boo"

In 1996, the Ghost Town DJ's, a hip hop group from the Southern U.S., released a song called "My Boo." One of the group's members was Vickie Washington, nicknamed "Boo."

An Alternate Trail

The poet/playwright Algernon Charles Swinburne (1837-1909) is said to have had an inamorata whom he called by the pet name "Boo" in the 1860s. Several people over the past 150 years have tried to deduce the identity of Boo, but no one has arrived at a definitive conclusion.

Sources to track down:

• Mayfield, J. (1953). Swinburne’s Boo. English Miscellany, 4, 161-77.
• Rooksby, R. (1993). Swinburne's 'Boo' Rides Again. The Review of English Studies, 44(173), 77-82. Retrieved from http://www.jstor.org/stable/518444

Rooksby has written quite a bit about Swinburne, so this would be another avenue to explore, for a different answer.