You didn't indicate your location or site characteristics (slope, hillside, etc), but the location of the country, even generally could be helpful, but not required.
You indicated the water bill was not high, so it could not be a water leak. That would only be if the leak was after the meter. However, the leak could be before the meter and impossible to detect through usage alone. A few tests to help determine if this is a spring:
Temperature: Spring water would be the average annual temperature (between 56-59°F). If it is different, turn on the cold tap water and measure it's temperature. If it is close to the temp of the water in the sump, that is more likely the source. If the water is a different temperature than either of those, it is likely ground water.
Chemical Composition: Have the water chemistries evaluated and compared to the houses tap water - if they are very similar, the water is likely a leak from the main, and not a spring. If there is chlorine, that is a dead give-a-way (but it off gasses quickly, so absence is not a complete indication)
If it is, in fact a natural source, then if could be spring or runoff. Most springs are intermittent to some degree with natural cycles. Perhaps this is a heavier time or from recent storms that are working their way underground and bumping into an impervious layer of shale that your house was built on top of. As a home buyer, you only have a point in time reference, and the neighbors may not know the details from the inside of the house as to how often and consistently the pump runs.
You also assumed it is only pumping a few gallons — some sumps hold 15-25 gallons. If it were built intended to handle the flow you mentioned (~900 gal/day) it likely would have been larger to begin with to run less often.
Is this a red flag / deal breaker? Is this just something we need to
keep an eye on, or is there a possibility that the foundation might
crumble at any time?
If it has been in place for 25 years, I don't think this is a major concern. If it was retrofitted, especially recently, I might have a different opinion. But this depends on where you live somewhat — is your area composed of a lot of Karst or cave features? Florida for instance is known for its sinkholes and springs are one possible concern with those.
In most areas, however, if the water is directed well, this should not be a concern because the water is directed away from the foundations and out of the house properly. Consider getting a home inspection & point out your concerns. They may find additional evidence of water damage, remodeling or other evidence of concerns that you could have overlooked.
Assuming it is not critical, what precautions should I take? I
already plan on buying a water-powered sump pump as backup, and
getting a couple alarms for both pumps. The current pump already has
a battery backup.
Alarms and backups are great. A small, but high-quality generator would be fantastic to keep things running for a longer period if needed. Make sure it's rated for constant duty. Be sure to have a plan to allow easy switching from the pump's current electric source to the generator. You could make this tie-in at the panel or the pump, but be careful not to backfeed into your system.
The water looks clear and fresh. What can I do with all this water?
Anything productive?
Do you have gardens? Make a rain-barrel type system using large holding tanks - plus, you'll see how much water you are actually removing seasonally. But be careful in planning that whatever you do that your system is able to handle cold weather without freezing up or you risk a damp basement and a blown pump.
But If you really are pumping what you claim (3 gal every 5 min = 960 gal per day) you will likely have limited use for all that water, unless you're in California and will share with your neighbors!
Good luck with the new home!
Obviously, the soil needs to be graded away from the house (in the back yard). If that means you need a retaining wall, then you need a retaining wall.
Regarding the swale, improved drainage sounds helpful. The real question is, why isn't water shedding (what is the real issue)? I've had swale issues, and the problem in those cases, was not with my swale, but the neighbors'... so water backed up in my yard. In such a circumstance, you can talk to the neighbors about the issue, ask them to fix it, offer to help, and/or sue them for damages.
But if the neighbors aren't to be blamed, then another alternative to keep the grass cut very short (grass slows drainage). Or you could check into a trench drain (concrete ditch). This is just another option that I didn't see mentioned, although it may not be your best option. As Tester101 pointed out, being there to see the issue is probably necessary.
Regarding power outages, gasoline or propane generators are the most reliable answer. A moderate generator should be enough for a sump pump, the fridge, and should keep the toilet flushing too. As you know, the drawback with a battery bank is that once it discharges, then you don't have a simple solution, like getting more gas.
Edit- Here's a possible way to use a retaining wall to increase the grade away from the house.
Best Answer
Sort of, but it probably doesn't work like you think. I'm guessing you want a pump that works like a shop vac. But once you put in a provision to drain off the collected liquid, the vacuum effect no longer works. One could concoct a system that alternately sucks, then drains, but I know of no such commercially available package system. Maybe something like this does exist? I'll bet it would be pretty loud!
There are vacuum primed pumps, they tend to be large, and when the first bit of air hits the impeller as the sump empties, all the remaining liquid in the suction pipe drains back into the sump. Something similar happens with the jet pump you linked to, but since the pipe is small, the volume of liquid drained back is small. But there's always the liquid remaining between the sump floor and the highest portion of the inlet opening, no matter which pump you use. Only manual vacuuming will get at that. In addition, most pump controls shut off the pump before there's any chance of sucking air, leaving even more liquid in the sump.
People typically use self priming pumps to suck out liquid from containers. These sorts of pumps all must retain a liquid reservoir in the casing from which they can re-initiate their prime. There's always some left over liquid sitting around somewhere when ever a liquid pump is involved.
It is possible to concoct a system where after the wastewater is mostly removed, potable water is added and pumped away in an attempt to flush away the remaining wastewater, so that only clean water is left standing. The reality is you are only diluting the remaining wastewater, so some small wastewater portion will always remain.
Jet pumps like the one you linked to are clever devices. You do need good water flow, and they do use a LOT of potable water. Also be aware that the integral check valve of this device will not be considered adequate by almost any building authority. Some sort of vacuum breaker will likely need to be added, further reducing available pressure and volume. These jet pumps are for removing relatively clean groundwater. The bits of food, hair, lint, etc. found in graywater will probably clog the device.