Hoodie with no hood

The OED says "A hooded sweatshirt, fleece, or other garment.", and therein lies the answer, I think.

A hoodie is a "hooded sweatshirt" or "hooded fleece", (in contradistinction to the original kind of each, which had no hood) and the abbreviation is natural.

As far as I know "hooded coat" or "hooded raincoat" have never become established phrases, and so have not become "hoodies". This may be because on an outer garment (especially a raincoat) a hood is unremarkable, whereas on a sweatshirt it is much more distinctive; but that bit is speculation.

A sweatshirt is made with sweatshirt fleece, which is a heavy fabric that is finished on one side and has a soft, fluffy nap on the other. The garment is designed to be worn with the finished side out and the napped side close to the skin, so the air trapped by the nap will help retain body heat. Despite the name, it does not have to be wool; sweatshirts can be made of cotton, synthetic fibers, or other materials.

A sweater is simply a knitted garment worn over the torso, typically one in which the yarn is thick enough that the knit pattern can be seen easily with the naked eye (unless it is obscured by the nap, as with fabrics such as angora and cashmere).