Learn English – “Let keep it clean” or “Let us keep it clean”

'Let go of me' is commanding someone to release you physically, in much the same way that 'Let go of the cup' commands someone to release the cup physically.

'Let me go' can be a physical command, but it can also be a metaphysical command. The difference in terms comes from the fact that someone can have different types of 'hold' on you. They could be physically detaining you, but they may also be holding you through authority and not letting you go to the cinema, for example.

The book is giving you some screwy examples. "Let his game be played by him" is correct but weird. You would only say something like that to make very unusual emphasis.

Here are a couple things to know.

How to make the subordinate clause

To make the object of let into the subject of its own clause, you need to put it into the objective case and put the verb into the infinitive. That's why you say:

Let him play his game.

rather than:

Let he plays his game.

Similarly, you would say Let him be helped by us, not Let he is helped by us. However, this sentence is equally as weird as Let his game be played by him.

Two (or three) senses of let

I think what the book is trying to do is teach two different senses of let at the same time that it's teaching you a tricky form of passive voice. Two of the main senses of the word let mean: (1) allow/permit the clause to happen; (2) suggesting or agreeing that "we" do the clause.

A classic example of the allow/permit sense: Let me go! is what a person who is being held against their will says to their captor.

A classic example of suggesting that "we" do something: Let's go! or Let's get started! is what you say when you want to start doing what you and your listener were just talking about doing together. This sense nearly always has us contracted to 's. (It has to be us rather than we because the subject of the clause has to be in the objective case, as above.)

Those are the most common and simplest examples to remember in order to learn the sound of the language. However, those examples can't be converted to a passive form.

Here's a more-realistic example in both active and passive form:

Let Dr. Kildare see the patient.

Let the patient be seen by Dr. Kildare.

Possibly your book has confused the suggesting/agreeing sense with a third sense of let, expressing a wish. A classic example of using let to express a wish is: Let peace prevail on Earth. Here's a realistic version of what I think your book is trying to demonstrate:

Let us beat our swords into plowshares.

Let all our swords be beaten into plowshares.

or, passively again, without using let:

May all swords be beaten into plowshares.

I can see why your book might have confused these senses. They really are a big, muddy mess. They are all variations on the basic sense of allow/permit, stretched to mean different things by repeated usage. The wishing sense can often be understood as the allowing sense and the suggesting sense simultaneously, where the request/suggestion is addressed to a deity, like Oh, God, please let there be peace on Earth.