(This is all based on reading about ashcans on a variety of indie RPG sites a year or two ago. I have purchased a few.)
The word "ashcan" comes from the comic book industry. In recent usage in the comic book industry it tends to mean a smaller format comic (for varying meanings of "smaller") used as a freebie to try and hook new readers.
In the indie RPG scene, an ashcan is a sort of pre-release, a weird sort of public beta-test. A designer might not feel their game is done or has uncertainty. The designer wants feedback from a wider group of people. This would normally be done by playtesting, but: the designer may have exhausted his playtesting pool, the designer may want opinions from "normal" gamers who wouldn't normally engage in playtesting.
Generally speaking an ashcan looks "finished," frequently with production quality comparable to the final product.
By and large you pay for an ashcan RPGs. While this limits your player base, the people who purchase it will almost certainly be more invested. The problem with, say, a free online release is that you'll get lots of downloads from people who will never play it. They might offer feedback, but the designer wants feedback from actual play, not feedback based on just reading the rules. If you've spent a few bucks on the game, you're presumably genuinely interested. You're more likely to seek out a group, to try it repeatedly, and to be interested in providing serious feedback.
Finally, many ashcans come with a discount on the final game as a sort of thank you.
It is indeed a new term used to describe conventional, normal RPGs in the "traditional" tabletop RPG format as opposed to newfangled indie games. It is not pejorative in nature, though it is used a little grudgingly as it mainly exists to distinguish "games that work like most every RPG ever as opposed to whatever crazy new variation you've come up with" in Internet discussions.
A trad game likely has:
- strongly differentiated GM and player roles
- task resolution via dice against skills, ability scores, or other metrics
- some basic nod to realistic simulation of the game world
- character advancement
- other stuff common to most RPGs ever made
D&D, GURPS, Rolemaster, and the vast majority of games published before the year 2000 are trad games. (Notable exception: Amber Diceless Roleplay). There are many new "trad" games too, from Savage Worlds to Eclipse Phase, that bring new genres or systems to gaming but stay within the traditional tabletop RPG format.
There is no clear legalistic differentiation between trad and indie. Most would say that the White Wolf Storyteller system, though it had initial aspirations to being narrative, is in retrospect a completely trad system. However, many new games have some aspects of traditional RPGs but innovate in one or a couple ways - there's no real clear "this crosses the line" criteria, it's more an attribute of self identification by the game's creator(s).
Best Answer
An itch.io game jam is a distributed challenge, with a central organizing point but no expectation of coworking or collaboration.
A similar distributed challenge that you're likely to have heard of is a project called National Novel Writing Month, which was started around the year 2000. People would sign on as individual writers with the goal of reaching a certain word count on a personal novel project that began and ended in the same month. While a common online space was provided, though the specifics of that space changed as the years passed, there was no expectation that all participants in the project would make use of it, or that cross-collaboration and criticism would occur among participants.
itch.io game jams organize around similar principles - while there is a theme, a deadline, and the expectation that work will be novel work done exclusively during the jam period, there is no expectation by people who create a jam that anyone who works on it will be taking a dedicated timeblock to do it or maintain any kind of shared physical presence. Sometimes they offer up a collaboration space like a forum or Discord channel, but the expectation is that the jam will take up your spare time over a longer time period rather than being the only thing you're doing.
This differs from the original "game jam", effectively an extended coworking "jam session" where people collaborated to make games.
The idea of an informal improvisational session and refinement workshop among craftspeople is much older than Gilded Age jazz, though the "jam session" terminology dates back to then, and perhaps apocryphally to one regular participant's desire to improv around Clarence Williams' "(I Ain't Gonna Give Nobody None O' This) Jelly Roll".
Similarly to a musician's jam session, game jams began with the idea of a group of people taking a small but dedicated block of time to do game development work, with the same sort of open criticism, commentary, and knowledge sharing you'd get from a musician's jam session. They also involved physical presence in the same space. Rather than just ad hoc refinement of the craft, however, these game jams had a goal - that everyone involved would participate in creating one or more complete games, which would take more than just a few convivial after-hours hours.
The only commonality itch.io game jams have is this same goal - to produce something novel under time constraints.
This isn't to dismiss any of the work that goes on there - the Internet can connect people with common interests even if they don't have the desire or the means to be commonly physically present, and as you've noted many products on itch.io are connected to one jam or another. But if you're looking for the opportunity to have a short burst of active, focused collaboration with fellow game design enthusiasts, itch.io game jams are better viewed as an opportunity to initiate one yourself, rather than one that has been set up for your participation by an involved organizer.