Sorry, but this will be more difficult than you may think, at least if you are looking for it to look great. (I am a perfectionist on these things, and everyone has different goals and flaws they are willing to accept. However, you might be disappointed if you hang a door that you are not proud of.)
Suppose you fill (as indicated) with matchsticks, and any type of dowel. Those spots will have end grain showing, which will be extremely noticeable, especially if you then stain. The end grain will absorb stain differently from the rest of the wood.
Similarly, suppose you fill with wood filler or just a paste made of sawdust and glue. Again, it will not have the same characteristics as the surface wood. It won't look right. Of course, it depends on how large the holes are.
You might try a plug cutter in a drill press, used to cut face grain plugs to match the grain of the wood. Now, drill the holes out with a sharp drill bit, so the plugs will fit perfectly. A tiny dab of glue on the plug, and it is there to stay. Now you will have face grain set into face grain. If you are careful to get a nice grain alignment match, this can be made to work very well. And since you can buy plug cutters of various sizes, you can fill holes of many different sizes.
Once the plug is set in the hole, sand it down to be perfectly flat to the door surface. I rarely even wait for the glue to set, since it will do so in good time. Anyway, the sanding dust as I sand mixes with any glue present to fill in even any tiny imperfections around the perimeter of the hole. Note that plug cutters do not work well in a hand held drill, but you can get an attachment for many drills to make them work like a small drill press.
In the case of a really large hole that needs filling, you can always use a router to cut away that area to get a region with nice clean edges. Now if you do it properly, a template guide in the router will let you inset a Dutchman, again, with perfectly matching face grain.
Alternatively, one might try veneer patches, but for an outside door, veneer will likely not hold up well.
For those with some artistic skill, you might also consider faux painting any patches. Here you essentially paint on fake grain in any patched areas, made to match the door. Such faux grain can be very convincing, and with a good finish on top to protect is, this might be acceptable.
Make sure you find a piece of wood for these plugs that matches the grain of your door. All oak is not the same. You said oak, but there is white oak and red oak: even within those subspecies, there are different colors and grains. It even matters how the wood was cut, thus quarter sawn versus flat sawn wood. These woods have very different looks. Find a hardwood lumberyard to select a piece of wood that will match. If absolutely necessary, bring the door along and get help from someone there to find some wood that will be a perfect match. Or you might bring a close up color picture of the door to compare the grain & color.
A few final thoughts:
Use a good glue that is rated for exterior use, so it will not be a problem with potentially wet conditions.
Since you are looking to match the existing wood, I expect you will apply varnish (or some variation of clear coat) on the door when done. Use a high quality coating that is rated for exterior locations. This should have components in it to resist exposure to the sun, rain & snow, as they are tough on such finishes.
Best Answer
I would take the little piece of pad out and place a scrap piece of 3/8" or 1/2" plywood in there if you have it. The pad will compress, the plywood will not. The doors are height adjustable, the bottom pivot in the door has threads that allow it to rise up or drop down. The original holes are not critical to keep. Place the bottom pivot guide onto the carpet and set the screw into the plywood through the carpet, the bracket may go at an angle, but press it down at the jamb end and set the screws into the jamb. Adjust the bottom pivot for height if needed.