Do I need to get all the stray fairies

Note: I haven't gotten this far in 3D yet, so this is based on the original N64 version.

There are really three ways to speed up at this point in the game.

1. Roll as often as you can. Which can be tricky as there are a lot of traps around this area.
2. Get the Bunny Hood first.
3. Play the Inverted Song of Time first. You can learn this song from the Scarecrow in the Observatory basement.

To get the Bunny Hood in the 3DS version:

1. Go to the Laundry Pool during the first or second night and listen to the organ-grinder guy. He will give you the Bremen Mask.
2. Go to Romani Ranch on the third day (it's blocked until then). Specifically, go to the Cucco (chicken) Coop.
3. Talk to the guy there.
4. Use the Bremen Mask to start marching around.
5. March near the 10 Cucco chicks and they should follow you.
6. Once all of them are following you, they should all turn into adults.
7. Talk to the guy to get the Bunny Hood.

Edit:

Having done this now, the Bunny Hood does make this a lot easier in the parts where you don't need the Deku mask to proceed.

However, when he goes around corners quickly, it's still possible to lose track of him.

You have to use your ears in this instance. It will always play his helicopter umbrella sound from whichever direction he turned at that corner. As long as you're within a reasonable distance of him, you should be able to get through any doors before they close.

Put a bomb down, they are secret grottos.