Learn English – “to dinner” vs “for dinner”

To expand a bit on what's been said so far, either to or for are equally used in this context and are both correct. With the exact wording you have here, I think to is the better option.

As for the actual distinction, it's a little fuzzy. Worded like this, the difference is that to indicates where/what you are being invited to, while for indicates the reason.

He invited me to dinner at 9pm.

This specifies what the invitation is for and tells you what the speaker is being invited to.

He invited me [over/out] for dinner at 9pm.

Although you'd be understood without the addition of the words "over" or "out" this makes it a little more clear. You've been invited over to his house or out to a restaurant, and the reason is to share dinner.

The reason you can use them interchangeably in this context is that dinner is both an event one can receive an invitation to, and an activity one can be invited over for.

Just to complicate matters a little, to can also be used to refer to intended actions or reasoning, as I mentioned for could in the previous example.

He invited me over to help fix his stove.

vs.

He invited me over for help fixing his stove.

Notice that when I used to it's referencing a verb (to help fix...). However, I chose for when I was referencing a noun (for help...).

It is confusing, and most native speakers will have trouble pinning down the actual rule of when they use one or the other. Your best bet is just to keep listening to sentences that use it until one starts to sound right. But, when it comes to dinner specifically, either is fine.