When boarding, I tend to aim for the Oxygen subsystem room - it's typically small, and critical enough that I will quickly attract attention. If they don't respond, then I've pretty much sealed their death quickly.
Meanwhile, I'll target whatever critical system I choose (usually weapons, shields, drones, or engines) with my laser weaponry. Once I've hurt one or more of their crewmembers, I'll target the medbay with missiles or bombs. That way, I can finish off heavily wounded crewmembers while disabling their ability to heal. Fire bombs and subsystem bombs are the best for this sort of attack.
I do believe that firing on the rooms where your boarders are will hurt them as well as the enemy, so I tend to avoid this. Trying to get the timing right for lasers (to reduce shields) + beams (to do damage) is also kind of tricky if you're also monitoring a boarding party.
Also, it helps if you've got a couple of teams of boarders - while you're healing the first squad, send the second squad in to keep the pressure up.
Some ships are particularly bad for trying to board, however. If they've got significant crew, especially a lot of mantises, I may give up trying to win by wiping them out. On the other hand, getting into an engagement and realizing they have no medbay pretty much means I've got a win locked down.
As you've noticed, Rock and Mantis crewmembers are the best to send as boarders. Don't forget that you can pause the game with the space bar, so if you think getting your crew back on board is going to be a close call, pausing and unpausing to slow the game down may help you slice the timing razor-thin.
Possible spoilers if you've never been to sector 8:
It's also been noted in a couple of strategies for fighting the final boss that the weapon systems are in isolated rooms with just a single human guarding each. If you send 2 of almost any race of boarders into these rooms, you can kill the guard and disable the system, thereby making this fight easier. The best weapon to start with is (in my opinion) the missile launcher, which is third from the left.
Yes, you will be able to gain new crew members. When you get a crew from an event, e.g., an interface will appear at the end of the event showing you every new crew member + the new guy (who will helpfully be flagged as "new"). You then have to chose to dismiss a member since you are over the limit. If you see crew for sale in a store, you can go into your ship screen and dismiss a member there so that you can buy a new one.
In the comments, ChargingPun confirms that when you are on the "crew dismissal" screen, you can still mouse over crew members to see their skills.
Also please note that as of patch 1.5.4/the recent add on, potential crew you encounter can come with skills which have gained experience. It is possible to get a crew with max [insert skill]. This is tremendously useful, as it both allows one to more easily make up for later-sector mistakes as well as maybe find someone with decent shields skill (heh).
Best Answer
Piloting
A crew member who is controlling a helm gains one point of experience for each incoming projectile that is dodged during combat. This includes asteroids, so long as you are in combat at the time. (Note that the dodges do not count while you ship is under the effects of +60% evasion from a cloak.)
This can be explicitly trained by finding an enemy that cannot break through your shields, and letting him fire at you. Increasing your dodge chance (i.e. more power to engines) will help this occur quicker, but even with the default chance you can max out the skill eventually.
Engines
Exactly as for piloting, a crew member manning the engines gains one point of XP for each projectile evaded while uncloaked in combat - and so it can be trained in the same way.
Weapons
A crew member who is manning the weapons station gains one point of experience for each weapon that is fired. It doesn't matter whether it hits or misses, or whether it can do damage (e.g. a beam weapon fired at a shield).
(Multi-shot weapons such as burst lasers still count as a single fire order and hence a single XP.)
This can be trained by firing non-lethal weapons (e.g. ones that do ion damage) repeatedly against a target that cannot damage you.
Shields
A crew member manning the shields station gains one point of experience for each shield "bubble" that is brought back up during combat. It doesn't matter why the shields were down, be that direct enemy fire, ion damage, asteroid strikes,
or even manually removing and then restoring power to the shield systems.This can be trained by letting a target that cannot break through all of your shields repeatedly take one layer down. (For fastest results, move your pilot away from the helm, or reduce your engines to 0 power. This will reduce your dodge chance to 0%, ensuring that everything hits your shields.)
Note that you used to be able to manually remove power and restore it in combat, to get points, but this has been patched.
Repair
A crew member gains one point of experience for getting the "finishing blow" when repairing a (sub)system. Note that hull breaches provide no repair experience.
There's no specific way to train this as it requires your systems to be damaged. Though you can focus the XP gains by always having the same person run around and repair everything after a battle.
Combat
A crew member gains one point of experience for getting the finishing blow when fighting an enemy crew member or (sub)system.
This generally gets trained in "real" situations. However, since XP is gained for taking out systems, a few extra XP can be gained from boarding if one tries to take out the enemy's systems before the crew.