What do the blue flags represent

You aren't clicking on them enough!

Or, more accurately, you might be clicking on their homes too much. Every time a villager comes out of their home to explore, they lose health. As they lose health, they weaken, and eventually die. That is what you're seeing when they sit down. They are in their final hours (seconds).

So make sure you click on them every once in a while! Especially if you see them sitting down. Or don't. You monster.

Clicking on a follower rejuvenates them! Or, more to the point, heals them completely (unless playing against an AI).

Your followers will weaken over time during exploration. Sadly, they will eventually die. If you see a follower sitting down, click on them to restore their health! That'll keep them going for a while. They will, over time, begin to walk slower, sit down, and die. Being seated means that they are quite near death.

It's worth mentioning that in matches against an AI opponent, your heals only count for 40, as opposed to 100.