I initially guessed 50% health was considered "low health." If you were to melee attack light enemies (i.e., non-heavy enemies like most Bullymongs) at 50%, it would cause them to stagger. Though after experimenting, this doesn't seem to be the case.
Based on my observations, I'd say "low health" is ~33% health (or 1/3 health).
Fortunately, it is pretty clear when the ability was applied in a kill. Normally you deal a fixed amount of damage to everyone. When the ability is applied, you see a boost in the damage dealt and you get a nice "buzz" sound and a slashing animation for every hit.
To test this, I respec'ed Zero and went off to fight making sure I didn't level up any more during the tests. As a baseline, I fought off enemies with only the Deception skill and no other skills. Every melee attack I dealt gave 55 damage.
I saved and restarted to go on another run. This time I added Killing Blow Level 1 to the mix. Again, most hits dealt 55 damage. Using a weak gun, I chipped enemies' health to around 50% (as close as I can get it) and hit them with a melee attacks. Still did 55 damage. I then tried for 25%. Again, as best as I can, I chipped them down to as close as I can to 25% then hit them with a melee attack. I got boosted damage and other indicators of a successful hit. So it's definitely applied at least at 25%.
However I noticed that at times when I wasn't quite at 25% but a little higher, I was still able to get the successful hit. So apparently it was higher. After a few more tests, I came to the conclusion that it is at 33% at max.
To see if the level of the skill affects this, I tried again with Killing Blow Level 2 and up. You get even more damage of course but it still seemed to apply at 33% and didn't change.
Beyond skill points and level, I'm aware of no other factors that directly affect Axton's Turret damage (ignoring things like Slag, which are properties of the entity being attacked, not of Axton or his turret).
It's worth noting that the Dahl Engineer class mod provides Sentry skill points, which will increase the turret's DPS (though not the damage per hit).
Best Answer
According to this post in the borderlands wiki, all of Moxxi's weapons return 2% of damage back as health. The Rubi is much higher (12%), in case you were wondering.