Understand first that both "go to work" and "get to work" have multiple meanings. Both can refer to the process of traveling to your place of work. Both also mean "start working".
In the context of traveling, the difference is similar to the English use of "go" vs. "come". If I say I need to go to work then I picture myself moving toward my place of work. If instead I say I need to get to work then I picture myself already at work and the fact that I am not actually where I should be.
In terms of the second meaning, the difference is similarly subtle. "Go to work" is used when starting a project, while "get to work" can be used anytime to mean either "start work" or "work harder".
Which to use can vary with context:
All right team. We've prepared everything, now let's go to work building it!
Steve, stop chatting with Mary and get to work.
We're already way behind on this project; we need to get to work if we're going to finish it on time.
The team of lawyers went to work on the contract to see if they could find any grounds for a breach.
Note without any additional context, simply using "went to work" instead of "got to work" implies the process had just started. I could instead say:
The team of lawyers got to work on the contract ...
which would not necessarily change the meaning, but it also would not as strongly imply that it was just the start of the process.
In the example with your coworker, since it's assumed that he was already working and stopped to chat with you, then "get to work" is correct. Although, it probably would have been even more appropriate to say "get back to work":
Steven I'm really busy. Could you stop distracting me and get back to work? Thanks.
Best Answer
Go in to work has a very specific meaning that implies physically travelling to one's place of business.
Go to work can be used in place of Go in to work, but it may also be used to imply focusing one's attention on a task. For example, one may Go to work on a taco if one is very hungry and the taco is very large. This usage could also be expressed as Get to work, which oddly enough rarely refers to arriving at a place of business; almost always referring to focusing on a task or responsibility.