Fallout – Underground Undercover: Canceling Possible

The Molecular Level quest both is and isn't a point of no return, fundamentally it requires you to make a choice - what you do need to consider is that speaking to a faction will provide you the infiltration quest for the Institute but also make the equivalent quest for the other factions unavailable. It is possible to reconnect with the factions you didn't choose after completing Institutionalized though.

The game is seemingly inconsistent in warning you when you're about to make permanent changes to your faction reputations. The actual point of no return varies by faction alignment.

In the case of the Institute quest line, speaking to Allie to start the Mass Fusion quest will pop up a nice warning stating you're about to become permanently hostile to the Brotherhood of Steel. There isn't a subsequent warning when you do the same to the Railroad, you'll just get lots of failed quest spam if you've got quests outstanding but in fairness, the mission is specifically to kill the Railroad so...

For the Railroad, I believe the switch occurs silently at the start of the quest Precipice of War. I definitely didn't receive a message and the conversation to start Precipice of War seemed to flow continually from the conversation that ended Operation Ticonderoga - the Brotherhood of Steel just stormed in and then I got lots of quest failure spam.

For the Brotherhood of Steel, the mission that makes you hostile with the Railroad is Tactical Thinking.

You can join all the factions without issues. You have to make a decision later, but not yet at this point.

At some point the quests you get will impact other factions, at that point you'll have to be a bit careful. But the initial joining and the early quests for the Railroad don't have any effect on your status with the Brotherhood.