There are a few corpse references we can look at that might help you make an informed decision.

First is the Animate Dead spell, PHB pg.212;

Choose a pile of bones or a corpse of a Medium or Small humanoid within range. [...] The target becomes a skeleton if you chose bones or a zombie if you chose a corpse.

In this case, a pile of bones does not qualify as a corpse for the purpose of reanimation and the two terms are not interchangeable.

Next is the Create Undead spell (PHB, pg.229) which uses the corpses of humanoids to create ghouls, ghasts, wights, or mummies. Each of which is generally depicted as being more than just a pile of bones. Since it doesn't say that a pile of bones (or skeleton) can be used to create these creatures, we can infer that once again just the bones don't qualfy as a corpse.

As such I would conclude that the Speak with Dead spell cannot be used on a skeleton. But hey, if you want it to work than as the DM you have the power to make it work.

Knowledge is key

Obviously, the target can only tell what the man knew before death: preventing him from knowing the one who paid (make him actively prevent others telling their names?) will prevent him from compromising the person. But of course, maybe that is no longer possible. In that case, let's look at the limitations of the spell.

Semantics for the win!

The spell has very specific triggers when it will not work, and you might exploit them:

If the corpse has been subject to speak with dead within the past week, the new spell fails.

This can be exploited: an item that casts "Speak with Dead" triggered by the death of the wearer will stop tests within the first week. The caster now would be a person that is not present, so he wouldn't answer any questions, just stand by for a few minutes. Even better, if dying triggers the spell to be cast once a week, it will prevent any questioning.

[...] the body must be mostly intact to be able to respond. A damaged corpse may be able to give partial answers or partially correct answers, but it must at least have a mouth in order to speak at all.

Destroying the body enough to not being able to speak is a sure way: have some sort of spell triggered at the point of death that immolates the head or blows away the jaw.

An other variant to prevent speech while keeping the body mostly intact would be to use a baleful polymorph spell triggered upon death, that just merges the jaws and lips. Mind, it targets "one creature" — not one living creature like polymorph!

This spell does not let you actually speak to the person (whose soul has departed)

This makes reincarnation not a protection against this spell, but...

This spell does not affect a corpse that has been turned into an undead creature.

Having a triggered spell upon death that raises the ninja as an undead, even for a moment, would prevent the spell from working: it has been turned into an undead creature and if it dies a moment later, it is the corpse of an undead creature without any memories.

Conclusion

Either prevent knowing the boss's name, or render the corpse unable to answer. To do so, you would need to find yourself a magician who is willing to make a custom magic item. This will cost up to (spell level × caster level × 2,000 gp) as it might be considered a "use activated/continuous" effect, or it might be as little as (spell level × caster level × 50 gp) for a "single use, use activated" item. So it might be a bargain.

• having an item, that destroys the head (or the whole body via disintegrate) upon death would surely prevent the usage of the spell — or rather the effective use at least.
• having an item that turns the body into an undead upon death will perfectly prevent the casting.
• having an item that uses baleful polymorph to remove the ability to speak (e.g. removing the jaws & mouth) upon the body upon death prevents answering, save for another baleful polymorph.
• Using a custom magic item that casts speak with dead once a week will prevent other questioning, but it is a more complex solution than a magical explosive leash (TV Tropes warning) that triggers upon death.