Learn English – How to end a letter beginning with “To whom it may concern”

I don't think there is a "black and white" rule there. With that out of the way, here's my opinion.

"To whom it may concern" is used where the letter is issued for use by someone without regards to who receives it, whether a person, or an institution etc.

"Dear Sir or Madam" is a more specific address.

According to Debrett's General Rules for Writing Letters:

The sign-off depends on the salutation. As a broad rule, if you addressed the letter to 'Dear Mr Debrett' the sign off is 'Yours sincerely'. If addressed to 'Dear Sir/Madam', then 'Yours faithfully' is correct.

If you don't know the name of the person to whom you are writing, start with 'Dear Sir/Madam' and end with 'Yours faithfully'. However, I would strongly urge you to find out the name of the HR manager. Individually addressed letters are obviously more personalised and, while it won't make a huge difference, will appear better than standardised ones.

EDIT (CLARIFICATION): The only traditionally appropriate way to start a letter to an unknown person is 'Dear Sir/Madam'.