Fallout – How to have 22 settlers

What seems to be the problem is that you haven't assigned the beds to anyone, which makes someone wanting a bed to rest look for the closest one and if it is occupied already, makes that person unhappy.

This is based on those 2 threads I found talking about the same problem and that the solution was to assign the beds.

First one :

So I took the time to put it to where I had exactly 20 beds, and then assigned each person a bed that night. It's been a couple days in game, and they haven't complained since.

Second one :

I've had some issues with beds as well. My latest problem was fixed by manually assigning everyone beds in the workshop mode (hit E on settler, then E on bed). Maybe try that.

I'd placed beds in every house in Sanctuary at the start of the game, thinking people might use them eventually so why not. But the settlers picked random beds all over the town, and something about that situation was making them unhappy. So I consolidated the beds into the houses in the center of town, made a town bell to gather my settlers, assigned them all to nearby beds, and put the unused ones into storage.

To answer your actual question, settlements DO grow slowly. If you are hanging around sanctuary, waiting for people to show up, you will be waiting for awhile. To do missions and side quests. When you return to rest, drop your scrap, etc. It will have grown.