All sort of issues here - while there's a short list of things that one normally looks for with extreme short cycling (such as a blown bladder in the pressure tank) that doesn't touch the 90 PSI issue...
If, in fact, water is going back down the well, the check valve in the pump (there is pretty much always one built into deep well pumps) is bad. Depending how your system is set up, there may or may not be another one inside the house, which would also have to be bad. My own system, after considerable reading and opinion-gathering, has only the check valve on the pump 300 feet down my well. When it goes, it's pump-pulling time. That would be the case even if there were others, and others might actually cause that to happen sooner according to a number of experienced well folks, which is why the other one I bought is still sitting in a box, not installed. In another few years I'll put a new pump on the shelf for when this one dies, so it's not a panic buy when (not if) I need it.
80-90 PSI is VERY high and almost certainly indicates a problem with the pressure switch. Certainly on my well the emergency relief valve lets go at 100PSI. On 40 off 60 is usually the upper range of domestic well pump "common settings" (30/50 and 20/40 being the other two commonly seen.) Ideally you'd know what your setting usually is or it may be recorded on a sticker somewhere on the system. You mention it going to the very high pressure when first switched on after being off for a while - what was it cycling between when you found it?
You could TRY hooking up a garden hose and opening that valve, then turning the pump on to flush out the crud that's been stirred up. It's possible-but-not-likely that this MIGHT clear a bit of crud that is jamming the check valve open. If nothing else it will get the stirred-up crud out of the well before the pump is repaired or replaced.
If your well is prone to throwing dirt, you may want to add a "spin-down filter" (I put mine after the relief, but before the rest of the system - this was contrary to the manufacturers instructions - they suggested after the pressure tank - but in line with the whole point of having it.)
That old PVC pipe is highly unlikely to be up to code for carrying a pressurized water distribution line.
You also do not want to place water service outside like that because it will freeze and burst the pipes when it gets cold in winter (unless you live in an Amazon tropical zone or Hawaii).
Proper installation is to install it inside near to the heated space of the home. Use of copper or PEX would be the preferred material for the water lines. If you use PVC makes sure that it is a proper type for pressurized water service.
Best Answer
I am guessing that is a small pressure tank (blue thing top of photo). The answer is yes yo can hook up a garden hose. I would add a valve so if the hose springs a leak you can turn it off. The next bit of good news it looks like you have 2 ports on the pressure side (top pipe right) if they unscrew add a hose bib and you can water all you want.