Fallout – Why is Edward Deegan hostile

It sounds like this might not have worked out exactly as the developers planned it, which is not shocking in a Fallout game, but the Institute's hostility is likely due to your character's attitude.

Try loading a save before entering the Institute and being a bit more agreeable. This should prevent them from becoming hostile.

I had something similar happen in Tenpines Bluff - there was a raider attack, and one of the settlers ran into the path of my VATS shot and turned hostile. Unfortunately, Codsworth was with me and so even putting away my gun didn't stop him from continuing to attack the settler, so I couldn't get them to un-hostile-ify. I tried running far away from the settlement and ordering Codsworth to wait, then returning, but by then the settler was too angry to calm down.

To solve the problem, I ran away again, dismissed Codsworth (instructing him to reside in Sanctuary), then fast-traveled to Sanctuary myself. I slept for 24 hours and fast-traveled back to Tenpines Bluff, and the settler was no longer hostile.

If Sanctuary is your only settlement so far, then just head back over to Vault 111 and sleep in the Overseer's bed, and the settlers in Sanctuary should calm down. Or find a bed in Concord, that should work too.