How to fill a gap beneath windows

Unfortunately, the windows probably should have been addressed before the new siding went on. Doing the windows before putting on new siding would have given you the option of using new construction vinyl double hung with J-channel build into the frames. It also would make repairing or replacing rotted framing members much easier.

However, since you already have the new siding up (would like to see a pic of the window with the new siding in place) you may have to use replacement type windows. If you have rotted frames, those will have to be replaced and the new replacement windows sized to the opening AFTER the frame is fixed. Of course you can carefully calculate the opening based on what you intend to use to reframe the opening, but that can be risky for an amateur. Keep in mind, replacement windows usually have to be ordered to you exact size and are not returnable if you measure them wrong. Always a good move to have supplier measure and order the windows for you. After you install the new replacement windows, you may have to install some new extension jams and exterior trim in order to fit snugly to the new siding. Don't forget to use Ice & Water shield under the finish frame/ over rough opening framing and caulk all areas where water could possibly get in.

These windows must have been installed into window casements that were built in place or reused from previous windows. The reason I comment about that is that if the windows were new units with the windows themselves pre-hung in the casements it is hard to believe that they would have the misfit that you describe as delivered from the factory.

In any case you would do well to see if you can talk one or two other contractors to come out to look at the installation to see what feedback they would give to you.

Sounds to me like you need to argue the case for a replacement set of new windows. If by chance you have not paid for the job you should withhold such payment until the problem is corrected to your satisfaction. If you have already paid then you have lost a certain amount of leverage but threat of a civil suit may be in order if you do not get a suitable response from the contractor. Of course you have to weigh the cost of getting involved in litigation against the size of investment already made in the windows.