Fly speed won't help you at all while underwater.
Unless you have some kind of ability that allows your to use your fly speed while swimming, your different speed types do not help you on different terrains. Swim speed will not help you move on land or air, land speed won't help you move on water or air, and thus, fly speed won't make you run faster on land, nor make you swim faster on water.
Excluding the swim skill rules, which allow you to use half your base land speed as your movement rate, but does not grant you swim speed at all. But that's a specific rule for the skill.
To clarify, whenever a rule is talking about your speed or your base speed, it is a reference to a character's base land speed, not whichever speed is highest. This can be found on common game terms:
Base speed is your unencumbered speed for a specified type of movement. Your base speed for any movement type is calculated in a similar manner as described in Base Land speed. When a speed type is not referenced, base speed usually implies base land speed.
Here is the basic description of the three movement types:
Land speed is the normal mode of movement for creatures that do not burrow, climb, fly, or swim.
Creatures with a fly speed receive the Fly skill for free as a class skill.
A creature with a swim speed can move through water at its indicated speed without making Swim checks.
If we check the Fly skill description, it says:
You are skilled at flying, either through the use of wings or magic, and you can perform daring or complex maneuvers while airborne.
Finally, the Swim skill makes several mentions of your speed (and even your base land speed on the unchained rules), but nowhere it talks about using other speed types while underwater.
There are exceptions, of course. The Swim on Air spell allows a water-based creature to use her swim speed to fly as if swimming on air, for instance.
However, it isn't unheard of that a creature that could make more sense to use a certain speed type instead of another. Some water-based creatures have zero land speed, so the GM might have to give it at least 5 land speed when trying to do something extraordinary while on land, even though normally they cannot move on land at all. Or even house-rue to use a speed type as another if that would make more sense in a given situation.
Example, the Octopus has a special water-based movement ability called Jet, which allows them to move 200 ft in a direction, while this ability does not mention that it can be used on land (even if that doesn't make a lot of sense) it's perfectly fine to allow a flying octopus to use that ability while airborne.
Best Answer
Yes, they can
There are no rules saying that a bard cannot initiate her bardic performance while underwater, or that you cannot use abilities with audible components while underwater. On the contrary, we know that you can speak normally while underwater, because sound travels farther underwater, as evidenced in Blood of the Sea (pg.28):
And again, Aquatic Adventures (pg.45), when explaining how spells work differently while underwater:
Another evidence of that is the Sea Singer bard archetype, which is a bard who is at home next to the water, nor the Aquatic sorcerer bloodline, which is themed as being descendant of creatures from the ocean, have any abilities modifying how audible components work. While they have several water-themed abilities, none of them modifies the requirement of audible components or affects their bardic performance differently while being underwater.
Also note that neither creatures with swim speed or amphibious creatures have any rule to "allow" them to use audible components underwater, because they don't need to have such ability.
For spells, you only require concentration checks if you cannot breath underwater. Otherwise, that's not even an issue:
So, if spells, which are normally more restricted than supernatural abilities (such as bardic performance), work normally underwater, why wouldn't a water-breathing bard be able to sing just fine?
This is all assuming her bardic performance even requires singing, playing an instrument or oratory, if their performance only has visual components, the only restriction might be in the murkiness of the water, which won't affect if they can do it, but the maximum distance they could possibly affect their allies (or enemies).
From Aquatic Adventures (pg.46):