I think I'm kinda stuck and I'd like to ask for help. Some preliminary stuff first:
I've already explored most of the Commonwealth and did most of the quests. I decided to side with the Railroad and they helped me build that teleporter thing. I promised Virgil I'd get him some FEV cure and Desdemona asked me to infiltrate them.
I entered the Institute and met Father. (that revelation got spoiled for me.) Throughout our conversation I was mostly 'unsure', neither accepting, nor denying his proposition to join the Institute. I did accept the invitation to look around though. He mentioned 4 names to speak to, which I did. The first 3 usually asked me if I was gonna join and I, again, said I wasn't sure yet.
Side note: Throughout my quest to find those 4 morons, I lockpicked my way into the FEV lab, in which I killed an Assaultron + turrets. But upon exiting that lab, nobody got hostile though. I also met up with that Patriot guy and agreed to help a solid dozen synths escape. Though again, upon going that far, nobody got hostile.
When I went to speak to the 4th guy, whenever I finish the conversation, I complete the 'Institutionalized' quest, though literally everybody gets hostile and starts shooting at me. I reloaded a few-minutes-earlier-save and exited the Institute, before speaking to the 4th guy. Then it says I've completed 'Institutionalized' as well. I went to speak with Desdemona again and my SS says he got kicked out of the institute, even though in that save file I did no harm. She then says I should ask Preston Garvey for help.
While still in the Institute, the 'Institutionalized' quest says 2 things. 'Enter the elevator', which I completed. Plus exit the institute, which seems to indicate the choice has already been made.
Where did things get messed up?
Best Answer
It sounds like this might not have worked out exactly as the developers planned it, which is not shocking in a Fallout game, but the Institute's hostility is likely due to your character's attitude.
Try loading a save before entering the Institute and being a bit more agreeable. This should prevent them from becoming hostile.