PC not recognizing gamecube controller with mayflash adapter

If it looks OK in the windows game controller control panel, Joy2Key is the issue.

Try using xPadder instead, it is far superior to Joy2Key. There is a free version linked in this question: How can I emulate keyboard and mouse commands with a game controller?

Also if it's this adapter, it says something about changig the function of the stick by holding start and A for 3 seconds: http://www.mayflash.com/Products/PCUSB/PC051.html

Without knowing how exactly your Wii modified it's not possible to definitively answer your question. The Wii was originally released with GameCube support, but Nintendo removed this backwards compatibility feature late in the life of Wii. If your Wii is one of these older models then you would need a GameCube controller and probably a memory card to play GameCube games. They won't work with any sort of Wii controller. It should go without saying, but just to be clear you'll also need a GameCube game disc as well.

(While GameCube games and memory cards might be hard to find, getting a GameCube controller is actually quite easy at the moment. Nintendo recently starting selling them again so they can be used with the recently released Super Smash Bothers for the Wii U.)

It seems unlikely that your Wii was actually modified to play Nintendo 64 games. This would require cramming in a complete Nintendo 64 console inside the Wii case. More likely is that your console was modified so that it can run homebrew software and so can run a Nintendo 64 (or other) emulator. In that case the emulator should work with normal Wii controllers.

Nintendo also released a few Nintendo 64 games for the Wii as Virtual Console games. These game are purchased through Nintendo's online store and are run using Nintendo own emulator. No modification of the hardware is necessary for these game and they should work with Wii controllers. The store page for the game should spell out exactly which controllers are supported.