Electrical – What do I need to replace 4 light switches on the same circuit

You say that, when you release the button, the contact opens again (resistance goes to infinity). But the light stays on, right? That means there must be some device which has a "memory" of the last action: you push a button, the light goes on and stays on, until you press the button again (or another button). I'm sure there is a device in the cabinet (Dutch: "schakelkast") which toggles its output on every pulse it receives, and which in this way controls the lamp.
The wire on the top right of the switch also hints in this direction; it's a signal wire, it's too thin to be a power wire. My hunch is that it carries the pulses to the actual switching device (I don't know what it's called).

Single switches that control a light are called single pole.

When there are two switches that both can turn a light on or off, they are called three way. (I am not sure about the origin of this label, perhaps because it required three wires; it does not mean that there are three or more switches - those are called four way!) In those switches, flipping the on off control on either switch will change its state: if it was off, it will go on, and vice versa.

When a dimmer control is part of a three way setup, one switch usually controls both the level and on/off, while the other switch only controls on and off. Both switches need to be the three way type.

It sounds like someone has put a single pole switch in a system that needs two three ways. Part of the problem is that there needs to be the right wiring between the switches. There has to be two hot wires*. There also may need to be a neutral wire, depending on the wiring configuration.

To determine the solution, we need to know what the wires are in each box that holds the switches, and what wires are currently attached to the switches.

SUPPLEMENT

It is possible to place dimmers at multiple locations if you use certain digitally controlled dimmers. You still need wiring between the dimmers that are able to have two hot lines, called followers, as well as a ground wire. You also need to provide a neutral wire to the fixture. That sometimes is part of the cable that goes through the dimmer boxes, but not necessarily.