The term I've heard used for this between-songs talk is "stage patter." Instances of the term go back to the early twentieth century in a Google Books search. For example, from Israel Zangwill, "The Serio-Comic Governess," in Collier's Weekly (January 3, 1903), we encounter a character who says:
And the tales you tell me — how useful they'll come in for stage-patter !
The term seems originally to have referred to a carefully constructed series of remarks—as much scripted as the rest of the stage show. For example, in "The Point of View," Scribner's Magazine (August, 1920):
Isn't much of the slang of this generation manufactured in the word factory of some writer of musical comedy or stage patter instead of in the home or the village shop where our grandfathers' slang—and furniture—was made?
But more recently, the term has come to refer specifically to the type of half-scripted/half-extemporaneous between-music spoken interludes you seem to have in mind. Thus, for example, in Frets, volume 6 (1984):
Try out a little rehearsed "stage patter," to get an idea of what it's like to talk to an audience. It usually goes a long way toward relaxing everyone (yourself included), if it isn't overdone. There's a fine line between keeping the set moving forward and rushing through it. A bit of humor or a few words of introduction here and there within a set can keep the audience keyed up for what's coming next.
And from J. Randy Taraborrelli, Motown: Hot Wax, City Cool & Solid Gold (1986):
"Maurice King was just the best," says Mickey Stevenson. "He worked with Dinah Washington, Billie Holliday, and all of the big shots of the forties and fifties. As well as teaching the Motown groups their music, he taught them about stage patter, what to say in between songs. Diane [Ross] would work with him day and night on this."
And from Alyn Shipton, Groovin' High: The Life of Dizzy Gillespie (1999):
Like all seasoned troupers, his stage patter included an element of the well rehearsed alongside the spontaneous wit.
Best Answer
Theomastix:
Source - Grandiloquent Dictionary
Theomastix:
Source - Encyclo,co.uk