Minecraft – How far do I need to be from the Iron Golem farm for it to work

Yes, it's called Iron Golem Farming and it's a widely used method of obtaining an infinite source of iron golems or iron ingots.

Quoting the Minecraft Wiki:

An Iron Golem farm is an easy way to acquire many iron ingots (and less useful poppies) with minimal effort after the initial setup. It is an artificial village (or several of them, spaced far enough apart to remain separate) in which golems are spawned and then either killed immediately, or moved to a holding cell (outside the village boundary, so that new ones can spawn in their place) for later killing.

This problem is caused by chunk unloading. Once you are far, the chunk with the spawner in it is unloaded from memory thus unable to provide you with iron golems or their drops. Try to shift your farm to spawn chunk and check if it starts working, as spawn chunks always remain loaded in memory.

This means, for instance, that redstone machines and iron golem farms will only work if a player is nearby. This can make effective farming difficult when the player spends a lot of time exploring or in another dimension. Since spawn chunks do not get unloaded, they continue to process events, even when no players are nearby. 

From Minecraft Wikia