Skyrim Museum Quest in Dawnstar

Okay, in my fifth run through Calcelmos lab I finally found the reason.

Just in the second room (you enter the lab, go a bit further, turn right and go further) there is one Wizard Guard and, sadly, a normal Markarth City Guard. When it locates you, it places a bounty worth 5 on you (probably because you shouldn't be there).

When you talk to him and try to pay him to strike out the bounty you will be teleported out of the lab. Luckily the bounty is so small that you can convince him it's too much work to cope with you and he just leaves you alone.

After not killing this guard, I could have the bounty removed on any other city guard after leaving the lab and the other city guards won't be hostile.

I just encountered that glitch on the xbox. I didn't do any Dark Brotherhood quests, so I don't know how that affects the game, but this is how I got around it.

Forget looking for the guard; if he hasn't come yet, he wont. Go to Hjerim and pick the lock (or in your case, open it with the key you got from Tova).

Once inside, you can't interact with anything at first. Instead it should say "inspect item". Go ahead and inspect everything in the house. You should find a couple of butcher journals, a bunch of "Beware the Butcher!" fliers, a Strange Amulet, and the Butcher's mess beyond the fake wardrobe.

Once you've inspected and picked up enough evidence, the quest should move onto the next step and proceed as normal. You can confront Viola (the woman making the fliers) or any of the guards with your evidence.

One more thing - if you picked up the Strange Amulet, it's marked as a quest item and can get stuck in your inventory. The only way of getting rid of it is by taking it to Calixto's Curiosities and selling it to Calixto. The amulet is not required to finish the quest so don't worry about selling it.