Learn English – “People decrease” or “The number of people decrease”

Strictly speaking, the subject of the verb here is the singular noun group; of fifty people is just a prepositional phrase modifying the noun. It is parsed as if it were written thus:

The group (of fifty people) is going to arrive Tuesday.

It's not that simple, however. For one thing, group is a collective noun, which may be construed as either a singular or a plural, depending on your emphasis.

This group is larger than that group. Here you are contrasting two groups as undivided wholes.
This group are all children. Here you are characterizing the members of the group.

So which verb you use really depends on what you mean; and that's something that can’t be judged from this little fragment of discourse. Including the phrase of fifty people might mean you are trying to emphasize the count:

There are fifty people, in a group, who are going to arrive on Tuesday.

On the other hand, the phrase may be parenthetical, just thrown in for additional information:

The group, which has fifty people, is going to arrive Tuesday.

Or the phrase may be restrictive:

The group of forty people is going to arrive Sunday; the group of fifty people is going to arrive Tuesday.

Say what you mean, and in this case at least you’ll be OK. As the great linguist H.Dumpty said, “Take care of the sense and the sounds will take care of themselves.”

[NOTE: Speakers of British English are more likely to use the plural than speakers of American English. But both populations will understand you perfectly well, and hardly anybody will even notice which verb you use. Or care.]

Most of the audience is/are sleeping.

"Of the audience" is a prepositional phrase and can essentially be struck from the sentence to look at the rest of the structure. So look at:

Most of the audience is/are sleeping.

Here it it obvious that the phrase should be "Most are..."