It took me a lot of tries to figure out how to hunt these whales effectively.
The reason the whale flees is because you are running out of harpoons. Yes, you have a limited supply of harpoons at your disposal; if you run out, you'll be forced to abort your hunt. So don't get too hasty, keep your calm and take your time to aim.
As for how to avoid getting damage from the whale:
The White Whale knows only 2 attack patterns, one of which is using its tail while your line is still attached. To prevent it from using its tail, you need to hit it soon after it lifts its tail. Take too long, and its tail will damage your boat, even if you managed to hit before its tail hits the water.
The White Whale uses its second pattern only after it manages to get rid of your line and after it submerges. It will emerge soon after, either right in front of your boat, or some distance away. When it appears right in front of your boat, that's when it's launching its attack. You have to hit it twice to prevent it from using that attack, so you should keep aiming towards the front of your boat while the whale is submerged, lest you have too little time to fire the second shot before its attack connects.
Before you go hunting for whales, I recommend you upgrade your harpoon storage to the maximum ( enabling you to carry 40 harpoons) and the harpoon strength to at least level 2. Rowboat armor is a fine idea as well, but if you're a skilled whaler, you'd probably be fine with a level 1 rowboat.
The key to this mission is patience. As long as you are not detected, after you have found your targets (actually see them), they will start to take a predetermined route around the town. They will be walking around together so they should be easy targets to air assassinate when in range. You should not get detected during this part of the mission otherwise you'll get everyone riled up and your two targets will stop taking that route and split up.
You should (silently) take out guards that are covering the roofs or are within visual range of them. That way you can easily get into position to the nearest roof when they come by. Then lie in wait for them to get close enough and air assassinate them.
If you're having difficulty taking out enemies silently, take advantage of your sleep darts (and not your berserk darts) so you can close in and finish your targets easily. You don't necessarily have to take them out one at a time. If you spot a group, shoot them all with darts and finish them off.
I recorded a quick video demonstrating how easily this can be done:
Best Answer
To use the mortars, first zoom in using the Mortar Zoom button. This will zoom in as if you were using the spyglass with the addition of a target circle. Move the target with the right thumb stick and, when you are satisfied with your target, press Fire to shoot. The target circle will turn red when an opponent is within range.
Buttons
Mortar Zoom
Fire