Caulk is a bad choice, because it can't be sanded. You can't get it perfectly even with the flat surface, because it stick to everything it touches. You will end up with a slightly raised area around the hole, and it will show more readily than putty, because caulk has such a different consistency. And, yes, caulk will shrink just like putty, but unlike putty, you can't put a second coat on.
This is not about the trim, but…
Danger! Danger! The top flange of the window has no flashing over it to direct water running down the wall onto the exterior of the flange rather than the interior. As a result, any water that runs down the wall will go right into the inside of the house. For that matter, the whole window looks un-flashed. That exposed wood is going to rot and let water into the house. You need a water-resistive barrier (WRB), like Tyvek or tar paper something, to cover this wall.
The typical safe way that windows are installed in solid masonry walls is to deeply recess them in the wall, sit them on top of a sloped sill, and install "block frame" windows that can be directly caulked and sealed right to the wall, rather than in "nail fin" form--a surface-mount window style that was designed to be nailed onto wooden wall sheathing on the outer surface of the wall.
The easiest approach is probably to turn the window into a sort of artificial block frame window by relocating window and frame alike deeper into the wall, caulking all the gaps, then covering the exposed flanges and wood in PVC trim.
If you insist on going to keeping the window in place, then to make this installation work you need to treat it the same way you install such a thing in a wood-framed wall: with flashing and a water-resistive barrier (WRB) integrated with the window flanges. Now, this looks like a foundation wall, where such coverings are not typical, so I understand that there may be aesthetic considerations at play. If you are not willing to add a WRB and some type of cladding to cover the wall, you need to do the best you can to salvage this installation. Flash those window flanges using high-quality window flashing tape to cover up the wood, lapping them so that gravity will not carry water into the window or infiltrate hind the tape. The most important one is the top flange. You should caulk that seam and then apply the window flashing tape over that, and then caulk the top seam of the tape where it meets the wall. But even all of this may not be enough since concrete is not waterproof. Water that soaks into the outermost inch of concrete above the window will bypass these flashings and drip into the inside of the basement. I strongly recommend relocating the window or adding a WRB and cladding.
Best Answer
Not every corner is perfect, and some adjustment is needed for every cut in some cases.
Either the cut is not square, and the saw is still not adjusted, or the wall is bumped out at the bottom. If that is the case the wall can be carved back behind the base or the base can be thinned down on the back to get the corner to come together.