Drylock is of questionable benefit, IMHO. It's definitely not a solution to a leaky basement...but it sounds like you've done some regrading already which should help with that.
The reason basements are often musty isn't so much from water from the outside but condensation. Basements are humid and cool, so the walls tend to be perpetually wet. The goal, then is to insulate the walls so help prevent that moist air from hitting the cool wall.
When using fiberglass, that's permeable to the air, so the solution is traditionally to put up a plastic vapor barrier. However, if water does leak in from outside, you've now trapped it.
As such, I strongly recommend going with the Building Sciences corporations recommendation of using XPS foam board against the wall and no additional vapor barrier. The XPS acts as a moisture retarder, yet will let moisture eventually dry through from one side to the other.
I went in to much more detail (including my recommendation to use metal framing rather than wood) with this answer here: https://diy.stackexchange.com/a/8644/1209
When finished, the other objective is to try and reduce the humidity in the basement. If you can bring the basement space into being part of the conditioned space, that's ideal. Alternatively, keep that dehumidifier running.
To answer the specific questions:
Is drylock sufficient to use R-15 batt? Or should I skip it entirely?
In terms of creating an insulated finished space, Drylock is of no use. The purpose of a vapor barrier is to prevent moist air from getting to a surface to condense on. The concrete wall will still be cool regardless of Drylock.
If not then should I put in EPS/XPS? (again Drylock?)
Yes to the EPX/XPS (see my linked answer above).
Would I really need more than that?
Code should dictate what is required. Check your local authorities regarding the amount of insulation you need. EPX/XPS will get you more R-value per inch than fiberglass.
In terms of comfort, the biggest culprits in a basement will be air infiltration, the upper exposed part of the foundation, and the space between the floor joists. Focus on those as well.
We can only guess at this point, but one theory is that when it rains, you have water seeping down the outside of your foundation until it hits the footer, at which point it's seeping into your basement.
At the point where you have this puddle, what is outside? Is there a gutter downspout there? Or perhaps the ground is sloping towards your house at that spot? Check those things first.
Best Answer
Absolutely it's worth insulating.
Concrete is a poor insulator. 8" concrete blocks have an R-value of 1.1 - about the same as 3/4" particleboard.
In my neck of the woods the frost line is about 3' down. So, assuming that a house here has a 40x40 foot square footprint and the basement is 8' high:
160 lineal feet x 6" = 80 square feet exposed to air; 160 lineal feet x 3' deep = 480 square feet exposed to ground above the frost line; 160 lineal feet x 4.5' deep = 720 square feet exposed to ground below the frost line.
Calculate the heat loss, using the formula (Area * (Tinside - Toutside) / (thermal_resistance) See here for a complete explanation
Assuming it's 0F outside and 70 inside (you said long and cold :) ), we'll say the above the frost line is 20F and below is 40F.
Above ground: 80*70/1.1 = 5091 BTU/hr Frost line: 480*50/1.1 = 21818 BTU/hr Below Frost line: 720*30/1.1 = 19636 BTU/hr Total: 46545 BTU/hr.
Now sheath and insulate with R20 insulation. Call it an effective R15 due to thermal bridging by the studs and because you don't do this every day so there are some gaps, etc. Then your total comes to 3414 BTU/hr. You save more than 16,000 BTUs/hr, or almost 400,000 BTUs/day.
In a 6 month winter that's 72 million BTUs.
Your $ savings will depend on the price of energy in your area, what you're heating with etc. $15/million BTUs is not unusual: that's higher than it was in Alberta where I'm from, but lower than California where I'm at now.