New Jersey is not generally known for farming. It is generally thought of as small cities and suburbs.
The origins of the nickname "the Garden State" are not clear. They apparently go back to the late 1800s. Maybe at that time the nickname was highly appropriate. It was declared the official nickname in 1954. I just did a bit of checking and find that the bill making it the official nickname was vetoed by the governor on the grounds that most citizens of New Jersey do not think of gardens or farming as more representative of the state than other industries, but the legislature overrode his veto.
As someone who grew up in New York, I can tell you that most New Yorkers think of New Jersey as the poor step-sister of New York, as a place to be looked down on. They think of it as dirty, poor, unsophisticated, etc. I recall a New York friend of mine referring to New Jersey as "the armpit of America", which may have been extreme but is pretty typical of the attitude. (Of course, New Yorkers tend to view the entire rest of the world as the poor step-sister of New York. They have the curious idea that the fact that they live in a place with a higher population density than most of the rest of the world makes them superior human beings, an idea I didn't understand when I lived there, much less since I've moved away.) I don't know if the phrase "go back to Jersey"'s firat appearance in a movie was the one Stoney B cited, but the idea of insulting someone by telling him to "go back to Jersey" is consistent with the way New Yorker's thought and spoke since I was a kid in the 1960s.
And yes, New Yorkers disdain people from farm country even more than they disdain people from New Jersey. They routinely see farmers and people from farming areas as stupid, corrupt, and uncultured. Watch TV shows or movies made by people from New York where the characters travel to a rural area. A very common plot theme is that the local governments in such areas are corrupt, with the "outsiders" arrested on trumped-up charges to cover up some nefarious plot, or just because the locals are suspicious of all outsiders. Considering that statistics show that New York City has a much higher rate of official corruption and street crime than almost any rural area in the country, it's a curious stereotype, but there it is. Oh, and rural people are always assumed to be ignorant of modern life, and New Yorkers are routinely amazed to discover that people in rural areas have computers and cell phones and cable TV.
Best Answer
It's not a well known joke, nor are you missing any nuance of the English language. Presumably Andrew was alluding to an earlier meta thread of his - How come I never see any Aussies on here?, where he put a humorous slant on a genuine question.
This would be known as an in-joke - a private joke that can only be understood by a limited group of people who have a special knowledge of something that is referred to in the joke.