Check the packaging on the caulk you used for your bathroom. From what you described, it was most likely silicone caulk, which is more difficult to work with than other types of caulk. Silicone requires alcohol rather than water for cleanup. I'd try acrylic caulk, which is the most common choice for exterior applications.
Two obvious options:
1) Dry well. Give the water a place where it can flow into the ground quickly, then leach away at whatever speed the soil will permit. Basically the same principle as a septic tank, but for the unwanted groundwater. If you really want a bottom-of-foundation drain, the well has to be that much deeper; not a small project unless you have a hill you can take advantage of. Does require that the soil be porous enough that the well will drain faster than you're filling it, at least most of the time.
2) Sump. Give the water a place where it can be collected and then pumped away. My area forbids connecting sump pumps OR roof gutters to either sanitary or storm sewers, so a common practice is to pump the water to the edge of your property in a place where it will drain across the sidewalk, into the street, and then into the storm sewers. Does burn some energy and requires replacing the pump occasionally, but avoids the need to excavate that well; the sump only needs to be large enough to gather enough water to make operating the pump worthwhile (it'll last longer and burn less energy if it's doing fewer, longer cycles rather than many short ones). Theoretically, you could even run some of the water from the sump out to rain barrels for gardening use, since it was in the ground anyway... but that's probably overcomplicated.
If you talk to the pros in your area, they may have other suggested solutions appropriate to your terrain, climate, soil, etcetera. I've found that a good contractor will sometimes be willing to advise even on projects they won't get, in order to build goodwill so they're the people you call when you do need more expertise/manpower/coordination.
Best Answer
I wouldn't be in any hurry to caulk that up just yet. I'd be investigating why there's water down there. Check your entire bay window for any cracks in the frame or any cracked caulk on the outside. I'd also dig that old caulk out and slide in some paper towels just to get an idea of how much water was getting in there. If you do find some areas outside that are bad and recaulk them, you would be able to monitor your success with the paper towel method.