You can do it. Certainly.
There was a recent episode on this for "this old house". They basically suggested cleaning out the loose fill and then replacing the old mortar with fresh mortar. It looked tedious more than anything else - but doable by a homeowner. I've re-mortared and chinked stone foundations. It's a surprisingly high maintenance experience compared to poured walls.
The good news about stone foundations is that with proper bracing, you can fix up almost anything well enough to sell it to someone else who enjoys huffing dust, cobwebs, squirrel mummies and Radon Gas. But for the most part, it's like fight club - we just don't talk about it.
With improper bracing, you can experience the joy of being crushed to death, or lose an eye due to the magic of newtonian physics while you learn about shear loads and amateur metallurgy using those hydraulic jacks you bought at Harbour Freight. It's very educational.
A well built stone wall can always be repaired with less trouble than concrete foundations. The mortar and chinking is mostly for aesthetics, anyway.
Sealing the walls is fine, as long as you do it on the outside first, and then and only then do the inside. You'll have to dig down to the foundation to do this, and that bracing is an issue on the outside as well.
Now the part you won't like. I did this in high school for a relative who had a nearly 200 year old house with a sandstone & rock foundation, in a damp (flood plain) area. It looked great, but ultimately it was wasted effort. Exposed rock and mortar foundations will always breath, sweat and disappoint anyone expecting a modern basement experience. You can apply gallons of sealer, but it will simply create areas where it perforates.
These foundations were designed to keep out animals, hold up your house so it doesn't rot all that quickly and very little else.
They will always be home to spiders. You can seal it with gunnite or epoxy and it will still allow infiltration. It might look OK, in that "I love Halloween" fashion so popular with the gothic/serial killer/lonely artist set, but it will never be a comfortable space.
You might build a second wall that can be plumbed, framed and sheet rocked, but it won't solve much over the long run, even with substantial vapor barriers and ducting. The walls will still slowly shift, moisture will come and go, and you'll never be happy with the low ceilings.
So in summary: You can redo the walls. You can seal them. You can even box them in and add ducting, insulation, AC and lighting. It's still going to be a basement that weeps moisture due to temp and pressure differentials.
Your downspout extension may be installed incorrectly. It appears that the black pipe coming out of the hill is not sloped properly and is causing water to pool inside of the pipe.
Since the pipe is not draining properly, organic debris such as leaves will clog the pipe and render it inoperable. Since water takes the path of least resistance, it will back up and then come out of the bottom of your downspout instead of out the extension.
The stones are not doing very much, and they are mostly decorative. Many people put stone, or some other kind of ground cover under their decks because it prevents weeds and grass from growing.
If you want to keep the stone there, I would recommend removing the plastic tarp from underneath it and replacing it with landscaping fabric. The fabric will allow water to drain through, and prevent weeds from growing. That way, you would be able to reduce the amount of maintenance in that area.
Best Answer
Not sure about inside, maybe damp rid or a dehumidifier. I don't suppose it wouldn't hurt to pack concrete in that gap under the wall/foundation (but you should dig this out from outside first).
The truth is, water coming in must be taken care of from the outside. A picture of the outside might be helpful.
I understand that you are on bedrock, but just to be sure, whatever soil exists outside should be graded away from the house. If that's not possible (or is not helpful) then you need to (dig out the outside and) seal the outside of the foundation. Tar and rubber liners are good for houses built into the side of a mountain. Again, packing concrete in that gap under the wall/foundation would probably be helpful. French drains should be at the foundation level... so that sounds like work (cutting into rock?) for you. If this is in the side of a mountain or hill, a cement patio or sidewalk (sloping away from the house) with a retaining wall for the soil would be the most effective solution (for worst case scenarios).