Electrical – Is it safe to replace a switch with a dimmer on halogen spots

This is a bad idea.

The maximum power rating for low voltage lights is lower than that for normal lights because no transformer is 100% efficient, so some allowance is made for that inefficiency in the specifications. In your case, those four lights draw 200 W plus whatever overhead the transformers add, so you're drawing more current than the switch is designed to handle. More current through the switch means more waste heat generated by the electronics in the switch, which leads to higher temperatures inside the electrical box, which in turn will shorten the life of the switch¹.

To answer your question about the brightness: I'd expect that they'd get less current than they should normally which would make their light both dimmer and redder. Generally, you have bright lighting in bathrooms, so even if it weren't a problem for the switch, it might not be something you'd want.

^{1. If you look at the specifications for the switch, you may see de-rating information for using two or more of them in a single electrical box; this de-rating is caused by the waste heat generated by the switch.}

You probably need to replace the power supply with one designed for LEDs. According to various web sites, some LEDs have compatibility issues with transformers designed for halogens. In particular, the power supply may have a minimum load and your all-LED setup uses too few watts: "When replacing with LED bulbs they found they had to keep at least one halogen in the circuit to keep the load high."