Should I fill in gap between window frames

I am having a bit of a problem understanding why plaster board or sheetrock would be finished to door and window jams without a casing/molding frame. In the few instances I have seen flush fitting walls to doors/windows, the wall covering is a thin, finished edge panel of some type, not 1/2" rock. In your case, the pic looks like the edge of the was was simply finished with some type of caulk or sealant to the jam, and either the jams have cupped or the caulking has shrunk over time causing the gaps to appear.

My first option would be to case the doors and windows with a molding. This would cover the gaps and secure the "flapping" wall board. The second option could be to use an elastomer paintable caulk/sealant to fill the gaps. I might use some finish nails to tighten the wall board and counter sink the nails slightly and fill and finish the holes. Hint: cover the jams and edge of the wall board with blue painter's tape before applying the caulk. Smooth the caulk into the gaps with a putty knife, then immediately remove the tape, leaving a nice crisp, straight line.

You will want to clean the area first. Scrape and moderately sand any peeling paint. Afterwards use a latex caulking which you can find at any home improvement store. This caulking is paintable which is the main reason I would use it and also shrinks and expands with temperature and moisture changes. Once the caulk is completely cured according to the directions on the tube, you can paint over the area. The better caulk products tend to have more flex giving them better longevity and decreases there chance of pulling away from the wood/molding.

Also, one more thing to add; What is the cause for the cracking in the first place? You may get more cracking if there is insufficient insulation between the rough opening and the window frame. Usually this can be solved easily by spraying foam insulation in between the two. This would be a more time consuming and difficult task, but may solve the root cause since this type of foam both insulates and seals the gap eliminating moisture and decreasing temperature change, both common causes of major expansion around windows and doors. The process for this fix would start with removing your trim, insulating with the foam, re-installing your trim and then following the steps above. This may not be worth the work depending on where you live, etc.... but it is a notable option.