Asking the basic questions is what gets us started.... not silly at all.
If they are as simple as nail holes from picture hanging and the like, and there are only a few of them, use spackle. It comes in small quantities, pints for example, so much will not go to waste. It is intended for only small holes.
If there are a lot of repairs, you may want to go with joint compound, it is pretty much the same as spackle, it is easy to sand, water based for easy clean up, but you will get a bunch more for the about the same money. It is intended for larger repairs, or new drywall install, can be built up, but it does shrink a bit, not so that it will fall out. If the shrinkage is noticeable, add another coat, no biggie.
Plaster is way overkill for this type of repair, it requires a skill to use. I have tried to use it before years ago, and it set up as soon as it touched the wall, quite surprising. I now know were it is to be used nowadays, and nail repair is not one of them.
Expanding foam can get messy but it may be your best bet, if you can keep it in place and they do not chew through it.
You may want to either incorporate a heavy gauge metal mesh wire, neatly cut around or into the place the mice go through. You might could even "bed" the wire mesh in the foam as it is expanding to lock in place that way. Doing this will make a potentially messy task and make it even messier.
This will not look nice in any sense of the word on the larger holes but it will certainly slow them down for a while. Otherwise a cat, as mentioned in the comments section, will also take care of the problem.
Thin sheet metal screwed in place will work as well without the foam too. You could try nailing it down, but in the cabinet it will bounce a bunch, screws are better here.
Best Answer
It is to remove ashes from the fireplace that is above it. So there will be a shaft up to the fireplace. I would only fill in the hole shown and not fill the shaft.