When boarding, I tend to aim for the Oxygen subsystem room - it's typically small, and critical enough that I will quickly attract attention. If they don't respond, then I've pretty much sealed their death quickly.
Meanwhile, I'll target whatever critical system I choose (usually weapons, shields, drones, or engines) with my laser weaponry. Once I've hurt one or more of their crewmembers, I'll target the medbay with missiles or bombs. That way, I can finish off heavily wounded crewmembers while disabling their ability to heal. Fire bombs and subsystem bombs are the best for this sort of attack.
I do believe that firing on the rooms where your boarders are will hurt them as well as the enemy, so I tend to avoid this. Trying to get the timing right for lasers (to reduce shields) + beams (to do damage) is also kind of tricky if you're also monitoring a boarding party.
Also, it helps if you've got a couple of teams of boarders - while you're healing the first squad, send the second squad in to keep the pressure up.
Some ships are particularly bad for trying to board, however. If they've got significant crew, especially a lot of mantises, I may give up trying to win by wiping them out. On the other hand, getting into an engagement and realizing they have no medbay pretty much means I've got a win locked down.
As you've noticed, Rock and Mantis crewmembers are the best to send as boarders. Don't forget that you can pause the game with the space bar, so if you think getting your crew back on board is going to be a close call, pausing and unpausing to slow the game down may help you slice the timing razor-thin.
Possible spoilers if you've never been to sector 8:
It's also been noted in a couple of strategies for fighting the final boss that the weapon systems are in isolated rooms with just a single human guarding each. If you send 2 of almost any race of boarders into these rooms, you can kill the guard and disable the system, thereby making this fight easier. The best weapon to start with is (in my opinion) the missile launcher, which is third from the left.
The Anti-Ship Drone is armed with a laser, which does damage to a single room targeted randomly by the drone. It also takes the shields down one level if it hits a shield..
The Anti-Ship Beam Drone is armed with a beam weapon, which does damage to multiple rooms along the path of the beam, but which does not affect shields at all. Instead, shields affect the beam, in that each level of shield reduces the damage of the beam. Since the Anti-Ship Beam Drone only does 1 damage per room, even a single active shield can render the Anti-Ship Beam Drone completely useless.
Best Answer
Does 60 damage to crew members of either ship.
From the FTL Wikia:
I think it is safe to assume your own crew will take damage while boarding. If you have drone bay feel free to go drone boarder crazy.
Tip: If you have anti-bio beam AND boarder drone put your boarder drone in a 2 by 2 room to get more enemy crew in one spot. Then hit that room with anti-bio beam and all 4 will be hit.