Obviously, the soil needs to be graded away from the house (in the back yard). If that means you need a retaining wall, then you need a retaining wall.
Regarding the swale, improved drainage sounds helpful. The real question is, why isn't water shedding (what is the real issue)? I've had swale issues, and the problem in those cases, was not with my swale, but the neighbors'... so water backed up in my yard. In such a circumstance, you can talk to the neighbors about the issue, ask them to fix it, offer to help, and/or sue them for damages.
But if the neighbors aren't to be blamed, then another alternative to keep the grass cut very short (grass slows drainage). Or you could check into a trench drain (concrete ditch). This is just another option that I didn't see mentioned, although it may not be your best option. As Tester101 pointed out, being there to see the issue is probably necessary.
Regarding power outages, gasoline or propane generators are the most reliable answer. A moderate generator should be enough for a sump pump, the fridge, and should keep the toilet flushing too. As you know, the drawback with a battery bank is that once it discharges, then you don't have a simple solution, like getting more gas.
Edit- Here's a possible way to use a retaining wall to increase the grade away from the house.
I wouldn't think that even in bad conditions the water table rises all that much, I'd guess your infiltration is from the side walls as rain drains down into the earth. Re-grade next to the foundation.
If that doesn't help, then yes, you need a real sump. I'd need a pic to say if I'd make a new hole or not. French drains can be added later, but you'll need a sump regardless. Or not, if re-grading takes care of it. Order of attack:
Re-grade next to the foundation
Add sump
Add French drains
If you're serious about keeping it dry so you can use it as a rec room or whatever, arbitrarily do all three steps.
Seal the foundation with some caulk and paint it. Or skim coat the whole thing and then paint it. At the very least, some paint will show you where the trouble spots are once they discolor. If there is space permitting, I'd leave that old pump alone as a backup, and dig a new pit if you're going to put a new sump in.
But if you're 5~6 feet above a river, a real sump pump might be a losing battle... Caulk the wagon and ford the river.
Best Answer
You could certainly perforate the lower section of the sump basin with small holes (1/4 inch/6mm or less would be my preference, but some might go twice/3 times as large - depends in part what you are bedding it in)
Outside the basin, you'd want washed stone (depending on soil type, possibly filter fabric and washed stone)
Inside the basin, a concrete paver is a suitable pump support. Don't put anything else in there that would make cleaning more difficult.
In my personal experience, pick a submersible sump pump - all the ones I've met have been much more reliable than the pedestal style pumps. To get the most out of whatever pump you buy, don't cheap out on the pipe size - plastic pipe is cheap, no good reason to reduce 2 inch threads to 1-1/2 inch pipe, but I see it all the time - even 1 inch pipe; don't do that - stick with whatever size the outlet on the pump is for best efficiency. And make sure it discharges far enough away from the building - I've met quite a few that pump the same water in circles, from inside to outside, where it comes back inside, to be pumped back outside...
Also, run the pipe straight up as high as it needs to go from the pump, then slope it towards the drain at 1/16 to 1/8 of an inch per foot (or 10mm per meter.)
If you are concerned about pumping rate, you can either try to come up with some sort of test flow (difficult unless the weather cooperates) or simply mount the pump to one side, so that you can add a second pump a few inches higher (use two pavers) if the first proves inadequate when the rains come. If you already have a pump in the old sump, start with that and see how it does.