If the pump is a 3-wire submersible with a control box, you can do some basic troubleshooting with an ohm meter without pulling it. If you have a control box like this:
Then you can pull the top off. Inside there are terminals that have black, yellow and red wires that go to the pump. Inside the cover of the control box, it will tell you the resistance ratings (in ohms) between each of the colors, which you can check with any standard multimeter. If those readings are off, then the pump needs to be replaced. If they are OK, the next thing to check is the current draw. You need a multimeter with a current clamp on it, like this:
and you need to be able to clamp around one of the wires while the control box cover is on. The red wire is the starter wire, so it should have a high current then go down. The others should draw whatever the control box says is the normal current draw for the motor. You can measure either the wires going to the pump, or coming from the panel.
If the current draw is very high before it shuts off, the pump is probably jammed. You'll need to pull it and possibly take it apart to fix it.
If there is no or very low draw (and the ohm readings are ok), the control box is probably bad, and you can just replace it without pulling the pump.
I used to work for a company that did that kind of work. Anytime we installed a pump in a cistern like that, we would use a pitless adapter to attach it to the incoming pipe. They're normally used in drilled wells to be able to pull the pump.
If you are lucky, then yours has one too, and all you need to do is screw a threaded pipe into the top of the adapter to be the pull bar (typically 1 1/4"), and then pull it straight up and out. Because you are not using clean potable water, and especially if it hasn't been pulled in a long time, it may require some "gentle" persuasion to get it out. Wiggling and/or using a T-shaped bar (which you can hammer from the bottom) usually does the trick.
Before you put it back you should check the o-ring, and lubricate or replace if necessary. It's not really as important in a cistern as it is in a well, since if it leaks the water just goes back into the cistern, and the pump doesn't have that far to push water anyways, but it's just good practice.
I have run across cisterns where this wasn't the case though. If the pipes come in through the top, you may just be able to pull them and pull the pump out. If not, you may be able to cut the pipe, and then pull the pump from there.
If those methods fail, or if the pipe enters the cistern at the bottom, then the only way to get at it would be as @longneck says and drain the whole thing, then crawl in and get the pump out.
If you do have to drain it, it's a good oppertunity to clean it as well. There is likely a lot of sediment built up on the bottom that you can get out with a shopvac. Spraying it down with a pressure washer probably won't hurt either.
I would highly recommend that before you put the pump back in, you install a pitless adapter.
I think what is leaking is the drain. Does it look like the object sticking out to the side in the below picture? If so, this is used to drain water from the house side of the supply when the valve is off.
It sounds like you over tightened it in the first place which started the leaking. If you've broken only the cap, you might be able to find a replacement. If the stem and threads are damaged, long-term you will need to replace the entire valve. As a short-term fix, you just need to block it some how. Marine epoxy is handy for emergency plumbing repairs.
Best Answer
I would say a soft start controller would not change the problem I have many booster pumps and the only thing a soft start would be good for would be throttling the pump. Centrifugal pumps create a large hammer action in how they work.
A expansion tank may be your best option. The quality of the valve may also be the issue I can warn you I had a irrigation pump that when throttled to 45hz with a VFD (can be used as a soft start) that frequency would tear up 4” irrigation lines I could run fine from 50 up to 70 hz and had no problems (below 40hz the motor would over heat)
There is cavitation at start up on the pump this is normal but a soft start won’t stop that. Some water filters can not be used without a pressure tank or expansion tank as the vibrations tear them apart.
I did see the comment on the water hammer arrestor that may work but I would think a 2-3 gallon expansion/pressure tank would solve the issue better than a electronic method.
I thought I should add most 120/240v pumps are capacitor start with a centrifugal clutch pulling them in. A soft start in this case would probably overheat the motor they are designed to spin up quickly and change windings or the cap depending on the type.