Finding the default in-game sound effects is quite technical. If all you want to do is create a resource pack with sounds, skip down to the Creating a resource pack section below.

The locations of the sound files changed in 1.7. The guide below explains how find the files in the latest version. For pre-1.7 versions, use the guide at the bottom of this answer.

Minecraft stores a list of virtual file mappings in %appdata%\.minecraft\assets\indexes\<game version>.json. To find the physical file for the sound you want, you have to search through this file.

The mapping contains the virtual path of each audio file, and the actual file it is located in. For example, if we wanted to find the second villager talking sound, we would want to look for the following in the file:

"minecraft/sounds/mob/villager/no2.ogg"

Underneath this line, there will be a line that looks something like this:

"hash": "c289aa46fcbcfaf21c8080d57953fd45a4939294"

The random characters in the second quotation marks are what we want.

To find the actual file, head to the .minecraft\assets\objects directory. This folder contains a big list of sub-folders. Find the sub-folder that is named with the first two characters of the has above (i.e the hash for the villager was c289aa46fcbcfaf21c8080d57953fd45a4939294, so we want to look for the folder named c2).

Inside this folder, there should be a file with the same name as the hash above. This is the sound file. These files are stored as Ogg Vorbis files, so to open them in Windows you need to have a capable media player such as VLC.

To use the file, first copy it, and rename the copied file to <previous name>.ogg (that is, add the .ogg extension). You might need to havve file name extensions visible in order for this to work (in Windows 8, click on the View tab, then check 'File name extensions' in the 'Show/hide' group).

Once this is done, you should be able to open the new file in VLC or similar.

Creating a resource pack:

To create a resource pack, head over to %appdata%\.minecraft\resourcepacks\. In here, create a folder with a name for your resource pack. Inside here, create three things:

1. A folder called assets.
2. An image file, saved as pack.png, containing a 'logo' for your resource pack.

3. A file called pack.mcmeta which should contain the following:

   {
      "pack": {
           "pack_format": 1,
           "description": "<a description here>"
       }
   }
   

In the assets folder, create another folder called minecraft. This is where all of your stuff goes.

Now, take a look back in %appdata%\.minecraft\assets\indexes\<game version>.json. If you want to change any sounds, you should first find the virtual path of the sound in question in this file, and then the folders and file in your resource pack look like the structure set out in there. For example, if you wanted to change the piston 'in' sound, you would create a folder called sounds, then another called tile, then another folder called piston, and finally, you would save your new sound as in.ogg.

Remember that all sound files must be saved as Ogg Vorbis files. You can convert sound files to this format with free programs such as Audacity.

Finding the default sound files for pre-1.7:

The default sounds and music for Minecraft 1.7 on Windows can be found in %appdata%\.minecraft\assets\virtual\legacy, sorted into folders and files. If this directory is not empty, you can also use it as a reference for the structure of your resource pack.