Sorry, but this will be more difficult than you may think, at least if you are looking for it to look great. (I am a perfectionist on these things, and everyone has different goals and flaws they are willing to accept. However, you might be disappointed if you hang a door that you are not proud of.)
Suppose you fill (as indicated) with matchsticks, and any type of dowel. Those spots will have end grain showing, which will be extremely noticeable, especially if you then stain. The end grain will absorb stain differently from the rest of the wood.
Similarly, suppose you fill with wood filler or just a paste made of sawdust and glue. Again, it will not have the same characteristics as the surface wood. It won't look right. Of course, it depends on how large the holes are.
You might try a plug cutter in a drill press, used to cut face grain plugs to match the grain of the wood. Now, drill the holes out with a sharp drill bit, so the plugs will fit perfectly. A tiny dab of glue on the plug, and it is there to stay. Now you will have face grain set into face grain. If you are careful to get a nice grain alignment match, this can be made to work very well. And since you can buy plug cutters of various sizes, you can fill holes of many different sizes.
![plug cutters](https://i.stack.imgur.com/WHbpK.jpg)
Once the plug is set in the hole, sand it down to be perfectly flat to the door surface. I rarely even wait for the glue to set, since it will do so in good time. Anyway, the sanding dust as I sand mixes with any glue present to fill in even any tiny imperfections around the perimeter of the hole. Note that plug cutters do not work well in a hand held drill, but you can get an attachment for many drills to make them work like a small drill press.
In the case of a really large hole that needs filling, you can always use a router to cut away that area to get a region with nice clean edges. Now if you do it properly, a template guide in the router will let you inset a Dutchman, again, with perfectly matching face grain.
Alternatively, one might try veneer patches, but for an outside door, veneer will likely not hold up well.
For those with some artistic skill, you might also consider faux painting any patches. Here you essentially paint on fake grain in any patched areas, made to match the door. Such faux grain can be very convincing, and with a good finish on top to protect is, this might be acceptable.
Make sure you find a piece of wood for these plugs that matches the grain of your door. All oak is not the same. You said oak, but there is white oak and red oak: even within those subspecies, there are different colors and grains. It even matters how the wood was cut, thus quarter sawn versus flat sawn wood. These woods have very different looks. Find a hardwood lumberyard to select a piece of wood that will match. If absolutely necessary, bring the door along and get help from someone there to find some wood that will be a perfect match. Or you might bring a close up color picture of the door to compare the grain & color.
A few final thoughts:
Use a good glue that is rated for exterior use, so it will not be a problem with potentially wet conditions.
Since you are looking to match the existing wood, I expect you will apply varnish (or some variation of clear coat) on the door when done. Use a high quality coating that is rated for exterior locations. This should have components in it to resist exposure to the sun, rain & snow, as they are tough on such finishes.
Best Answer
White silicon may work to plug the holes but if you use it make sure to get the surface finish the way you want it before the stuff dries. It is not a material that will sand very well after the fact.
You may also want to try a bathroom tub and sink sealer product. These products bridge gaps fairly well and dry staying a nice white color. The product that I have used for sinks and tubs is the one pictured below.
Also be aware of several things. If screws had been applied into the PVC window frame the screw threads probably made a portion of the PVC surface stick out at the hole edge. This may need to be trimmed carefully so that your patched holes will be flat. If the window frame was made only of PVC with no internal wood structure the only thing the patch material will be adhering to is the edges of the hole. This will mean that the patch could pop in or out of someone were to be poking at it.