You're trying to use a D&D precedent in Pathfinder, but the Diplomacy rules were specifically changed!

The line "You cannot use Diplomacy against a creature that does not understand you or has an Intelligence of 3 or less" was explicitly added to the skill description in Pathfinder; it was not present in prior incarnations of the rules. Given this, it hardly matters whether it worked in 3.5 or not.

what is this witch ability good for if it doesn't allow for using diplomacy?

Um, talking to animals, same as the spell it emulates? You can still get information from them. I believe you could also cast [language dependent] spells on them, if you want a more specific game benefit.

Enforce the Handle Animal Rules

Getting those dogs to do what they want requires a Handle Animal check (DC 10 and a move action if it's a trick). Each PC can only do that to one animal at a time. Otherwise, the dogs will just generally do whatever you as the DM want them to. That might mean they all swarm something, or it might mean they find the target frightening and run away from it.

Remember, animals will only normally attack other animals, humanoids, monstrous humanoids, and giants. The animal has to be specially trained and use two tricks to have an attack command against anything else.

Kill The Dogs

Unless these are advanced animals with bonus HD, Dogs have 6HP. At 5th level, one Fireball will wipe them all out (the Riding Dogs you gave the party have 13 HP, so their survival chances are better but it should still kill some of them and make the Handle Animal checks harder on the others due to being wounded), and since Dogs are pack animals and one PC can't direct five dogs to attack five different things in one turn, they're likely to not be that far apart.

A Barbarian with Cleave could make short work of them as well. I'm not saying to go far out of your way to kill them, but they're low HP targets in combat, and they make a good target to even the numbers in a fight for certain types of enemies. If none of them are dying in combat at this level, it would be pretty strange.

You could pretty easily make an encounter that would eliminate the dogs in a turn or two, at which point the party will have to decide if it's really worth spending money to buy new ones.

Alternately, you could kill one or two of them in an encounter (which should happen normally in EL 5+ encounters), and see if the party decides that having dogs that aren't up to higher end combat around is a good idea. Any animal lover characters wouldn't want to put animals in harms way that aren't up to it, and the more frugal characters wouldn't want to have something die when they could sell it for gold and get more value out of it.

Initiative Shortcut

When this happens in my game, I roll one initiative for all of a given type (all the Dogs in this case), and have them all move at the same time. That cuts down the annoyance a little bit.

I also don't put much thought into what they're doing. Dogs are animals, not tacticians. They will either do exactly what they're ordered to do, or they'll attack, or they'll not attack. They're not going to get fancy figuring out how to reposition themselves for optimum protection of the PCs. (The PCs would have to use Handle Animal commands to do that.)

As they only have one attack, the rolling should be pretty limited and so you can keep the pace up by acting quickly without thinking about what they're doing.

Party Logistics, As You Mentioned

Is the party carrying enough food for five dogs? Are you enforcing that? Caring for five dogs in real life is actually quite a lot of work, make them deal with that.