Try sheathing your weapon. Or try using a calming spell on a guard.

Here are the standard approaches.

Getting arrested

If you have a bounty, guards from the hold in which the bounty applies will attempt to arrest you. If you run from the guards they will chase you. If the guards become hostile, you can yield by sheathing your weapon.

When you are arrested by a guard you have several options:

Declare Thaneship

If you are the Thane of a hold, you can declare this to the guard. This will immediately clear your bounty (if the bounty is less than 3000 gold), but can only be done once per hold (twice if the hold is captured during the civil war and you are made a Thane by the new Jarl).

Pay your bounty

If you pay your bounty, you will be immediately transported outside the nearest town jail, and all stolen items in your possession will be seized. Thieves Guild members have an additional option to pay a much lower fine, if they so choose. This additional option will allow you to keep stolen items.

Bribe guard

Thieves Guild members have the option to bribe the guard. The amount of gold required to bribe a guard is usually much larger than the bounty for small crimes, but less than one for murder and doing so will raise the Speech skill, immediately clear your bounty and avoid having items confiscated. The Bribe Guards Speech perk also grants you this option. Note, the bounty cleared is only for the crime you were recently caught doing. If you had some previous bounty as well, this won't get cleared.

Go to jail

If you cannot pay your bounty, you will be sent to the nearest town jail. You are able to keep one lockpick in your jail cell, which can be used to try to escape from prison. Escaping from jail can be much easier than in Oblivion with an option to pick the door or a loose grate on the floor which is much easier, however, escaping prison will still keep the bounty on your head. There are also other methods to escape, such as pushing down a crumbling wall. You can also pickpocket the jail key from a passing guard to escape the cell. However, it is still difficult to recover your equipment. To serve your sentence, simply sleep in the bed in your cell, and you'll sleep through your entire sentence. Depending upon the length of your sentence, your progress towards one or more skill advancements will be reset to zero. Unlike Oblivion, you will not lose any past skill increases; instead, it will take longer to obtain the next skill increase(s).

Resist arrest

Resisting arrest will cause all guards in the area to attack you. If you change your mind, yielding to a guard will make them attempt to arrest you again, presenting the same options. Sometimes, you cannot yield, and the guards will simply continue to attack. Using a strong enough Calm spell will stop the guards from attacking, though speaking to them results in an arrest attempt. Another method is to punch an innocent person, keep your fists raised and then try to yield again.

Persuade guard

Occasionally another option will appear if you have a low bounty. You can attempt to persuade the guard that you are not worth the effort of dealing with, due to your low bounty, and should be allowed to go free. Success is dependent on the Speech skill. If successful, you will raise Speech, but will not clear your bounty. If your speech is 75 or higher, this option will always be present (given that the guard has not been persuaded already). If you wish to persuade the guard again, wait 2 days and the option will be available again.

Bounty collector

If your bounty is above 1000 gold, you may be met by a Bounty Collector. Unlike the guards, they ignore the distinction between holds and will attempt to track you down regardless of which hold you are in. You have to pay him the bounty + 20% more as his cut. Your bounty will be erased as it would be when you pay to the guard, but you keep stolen items in your possession. If you kill the collector after paying him, he won't have that money in his inventory. Also, use his services at your own risk as the bounty may still be present in the hold that he collects the bounty from, even after paying him.

Bug

Occasionally, Guards will not recognize your attempts to yield and will continue attacking even if your weapon is sheathed.

Source: http://www.uesp.net/wiki/Skyrim:Jail#Go_to_jail

Picking the lock, or stealing the key to the museum will make the guards hostile to you. From this point, you will need to go the sneaky way, or kill people.

If you want to avoid this, you have other options:

• persuade Ancelmo (speech)
• do something for him (Kill the spider or solve The Book of Love quest)

He will then give you the key, and guard won't be hostile.