I think the lay (non-technical) person still thinks of "computer" as being a laptop or desktop. Of course, anything with a processor in it is technically a computer, which these days means your phone, gaming console, tablet, but also quite possibly your television, media device, and even devices like washing machines, microwave ovens and so on.
However, since you posted this question in the English stack exchange, I'm going to focus on how the general public understands the terminology, rather than technical definitions. I don't actually know of a single word which defines the concept of general-purpose computer, but that's what most people understand by "computer".
It is unfortunate that "PC" has come to mean a personal computer running Windows as its operating system, but that's the connotation in the public consciousness.
There's also "microcomputer" which once conveyed the same meaning, but it's become archaic since technically most servers now are also microcomputers.
According to Bablenet, these are called clause elements:
These are well worth learning about, as you will certainly want to use them to explain the syntax of language data you are studying in exams or investigations. If you are not able to describe or identify clause types, it is usually acceptable and always helpful to consider how these elements work together. You may use them to explain how sentences work, also. They are: subject (S), object (O), verb (V), complement (C), adverbial (A)
Best Answer
That 'part' is the computer. The other devices you describe are peripherals connected to it.
If the form factor is a traditional, vertical case, as pictured, 'tower' is often used as well. And of course, if you're looking for a term specifically for the housing, well, that'd be the 'case'.