on purpose is what's used. This is a legitimate expression in English with its own entry in the dictionary. It means intentionally. I myself use it all the time. As for the other one, I have never ever heard anyone say in purpose in place of on purpose. That would just sound wrong.
However, you could say in purpose in certain situations, but that would mean a totally different thing. For example:
This thing is completely useless in purpose as well as in use.
Meaning, it for whatever reason does not have a purpose and thus is considered useless.
Generally, off the top of my head is usually used to refer to the first thing that pops into your head.
What's the capital of Mauritania?---I don't know off the top of my head, but I could go and look it up.
This is a valid thing to say but you could also say:
I don't know off the top of my head, but it will come to me.
Meaning that it's not immediately available in your mind, but you do know the answer.
By heart implies something slightly different. To know something by heart it to know it inside out. You'd never hesitate to remember something you know by heart because you know it so well.
These are just my opinions and interpretations of the phrases (since you have already supplied the dictionary definitions).
However, given those slight nuances, I would lean towards using:
Our teacher was a really smart person. At the second session, he know everyone's name by heart.
...since your sentence seems to imply that he had picked them up really easily and knows them inside out, rather than just that that he had them in the forefront of his brain.
Best Answer
They're both correct, but with slightly different meanings.
"Head to" connotes that you're confident that you'll actually arrive. A college student might say, "Classes are over, let's head to the bar."
"Head for" connotes that you're going in that direction, but you might get waylaid or you're unsure of the route. A tourist in New York might say, "We're lost; let's head for the Empire State Building so we can get our bearings."