Learn English – What does “headline” mean in a job application form

You certainly did your research!

Two crucial points that it's not clear if you came across, though:

1) SNL is an abbreviation for Saturday Night Live, a popular comedy show that features comedic stand-up skits and musical guests.

2) The cowbell, in addition to being a countable noun, is also an instrument.

Now, let's take a break and watch an excerpt from the skit in question.

The skit is a reenactment and parody of the studio recording of the song "(Don't Fear) The Reaper" by the band Blue Öyster Cult. The guy screaming "I gotta have more cowbell" is the producer. Because the cowbell player (Will Ferrell) and the producer (Christopher Walken) are popular actors, and because SNL was a successful show at the time, this skit became a big hit. The humor lies in that "more cowbell" seems like an absurd request - a background instrument becomes louder than the vocals, the melody, etc.

The phrase is "more cowbell" and not "more cowbells" because Walken is asking that the one cowbell that's there be played more loudly. This is actually standard usage - someone might similarly ask for "more bass" or "more guitar", both in the singular. (Or, in another context, someone might stay a stew needs "more carrot".)

The "more cowbell" joke can now be seen all over US pop culture, though at this point, I have to say it feels rather stale. While it's not an idiom, you could call it a "pop culture reference" or "meme".

Understanding it as a cry for "more excitement" isn't bad, actually, but I wouldn't read too much meaning into it. One of the appeals of SNL was its randomness, and people often quote it quite randomly.

The toast goes back more than half a century before the scriptwriters of Casablanca used it in 1942.

From Anonymous, A Holiday Skip to the Far West (1884), we have this scene set in the Southern Hotel in St. Louis, Missouri:

"Any friend of Mr. O'Sullivan welcome. Will you nominate your poisons, gentlemen?" Since poisoning was apparently a necessity, I nominated whisky and water, in preference to a native mixture. It would be more satisfactory to know the cause of death, thought I, should this uncalled-for fatality occur. "Wall, —— here's looking at you," says my newly-acquired friend of the white apron, winking over the bottle-nose, and emptying his glass. "Here's how," responds the porter, and the curious bacchanalian ceremony came to a close. I am not proud, and enjoyed the hospitality, as the porter paid the drinks, be it said to his credit.

From "Things Pleasant and Otherwise," in Ballou's Monthly Mahgazine (May 1884):

"Ah," said the colonel, as he threw out a chew of tobacco, and took the bottle. "Here's looking at you."

"Drink hearty," the young man replied, and taking the bottle he held it up, and added, "Here's to the hair of your head."

From "Editor's Table" in The Yale Literary Magazine (April 1895):

Are your glasses filled, gentlemen? Then, here's to the rose of the girlhood—Dorothy, and one more in which that jolly devil, Saint Elihu, shall join for all his shadowy background and his austere countenance. And so fellow Collegians—"here's looking at you."

From Edith Chase & W.E.P. French, Waes Hael: A Collection of Toasts Crisp and Well Buttered (1903):

New Mexico

Cactus and prickly pear and sagebrush,/“Men with the bark on” and hearts brave and strong,/New Mexico, here's luck, here's looking at you;/You're a deep note in freedom's world-wide song!

Early extensions of the toast include this one from The Harrow (1889) [result not shown in snippet window]:

French returns in a few minutes and finds the Sophomore has cultivated it up. Waldo, at Hillsdale— "Here's looking at you." George — "Yes, through a glass darkly."

And this one from Wilbur Nesbit, The Loving Cup: Original Toasts by Original Folks (1909):

To you, and you, good fellows of the quill,/Belongs the Homage of the Wine I spill;/Here's looking at you from the Rosy Side,/And may you never lose a Dollar Bill.

I imagine that in a heavy-drinking, toast-oriented social environment, few dedications would come to the befogged mind more readily than "Here's looking at you," since the person addressing the toast is in all likelihood looking at the person thus addressed.

An early filmed instance of the expression occurs in the movie Three on a Match (1932), as Michael Loftus (Lyle Talbot's character) attempts to woo/seduce Vivian Kirkwood (Ann Dvorak):

Michael: Here you are, darling. [He hands Vivian a martini.] Well, here's looking at you.

Vivian: At me?

Michael: Yeah! And liking it, too!

Evidently, the expression was widely viewed as being quintessentially American, which would make sense in Casablanca as another marker of Rick's Americanness. At any rate, in one of J.P. Marquand's Mr. Moto stories (by 1938), we have this bit of dialogue [combined snippets]:

"Boy!" called Uncle William. "Whisky soda. ... Here's to you, Mr. Moto." Mr. Moto laughed again.

"Here is looking at you, gentlemen," he said. "That is the American expression, is it not? What beautiful weather we are having!"

"Yes," said Uncle William.

UPDATE: 'Here's Xing' as an American form

Because so many readers have echoed the comment/question (from andy256 below) "Yes, but ... what does it mean? Or if it is quintessentially American, what does that mean about America?" I decided to look into the idiomatic use of "Here's Xing" in U.S. usage.

Since Americans are (as Fowler says of people who grew up with the distinction between will and shall) "to the manner born," the form "Here's looking at you" doesn't strike us as particularly odd. Indeed, the similarly formed expression "Here's hoping" is used not just as the start of a toast, but as an equivalent (in a multitude of settings) to "I hope so."

One of the earliest examples of "here's hoping" in a Google Books search appears in "Metaphors of the People," in The Galaxy (May 1870), which quotes Mark Twain's description of toasts given in various locales in the U.S. West:

The reaching of the popular mind after odd fancies is illustrated in Mark Twain's humorous explanation of the etiquette of the bar-room on the Pacific coast. He says:

In a mining camp in California, when a man tenders you a "smile," or invites you to take a "blister," it is etiquette to say, "Here's hoping your dirt will pan out gay." In Washoe [Nevada], where you are requested to "put in a blast," or invited to take your "regular poison," etiquette admonishes you to touch glasses and say, "Here's hoping you'll strike it rich in the lower level." And in Honolulu, when your friend the whaler asks you to take "fid" with him, it is simply etiquette to say, "Here's eighteen hundred barrels, old salt." But "drink hearty" is universal. ...

The sentimental method of asking a person to drink is in the formula, "Suppose we shed a tear." The operation, strange as it may seem, is identical with "taking a smile." There is a frequent toast in some places which seems to contain considerable truth, viz.: "Well, here's another nail in my coffin."

Of all these expressions, only the "Here's another nail" and the "Here's eighteen hundred barrels" follow (arguably) grammatical form, but it appears that the "Here's Xing" wording arose fairly early and has shown impressive cultural persistence. In fact, it appears in James Fenimore Cooper's The Pioneers (1823):

Toasts were uniformly drunk ; and occasionally, some one, who conceived himself peculiarly endowed by nature to shine in the way of wit, would attempt some such sentiment as "hoping that he" who treated "might make a better man than his father; or "live till all his friends wished him dead;" while the more humble pot-companion contented himself by saying, with a most imposing gravity in his air, "come, here's luck," or by expressing some other equally comprehensive desire. In every instance, the veteran landlord was requested to imitate the custom of the cupbearers to kings, and taste the liquor he presented, by the invitation of "after you is manners;" with which request he ordinarily complied, by wetting his lips, first expressing the wish of "here's hoping," leaving it to the imagination of the hearers to fill the vacuum by whatever good each thought most desirable.

Given decades of "Here's hoping..." in the context of toasts and in the ordinary affairs of life outside barrooms, the emergence of "Here's looking at you" can hardly have struck listeners as a strange innovation. That toast, in turn, may have prompted the toast, "Here's mud in your eye," which a Google Books search finds as far back as 1912 in the form, "Here's mud in your eye and a quick trip back to Texas!" There are numerous competing interpretations of that expression, however.