Are there drawbacks to using solar panels as a roof without asphalt shingles underneath

Oh, wow. I'm sorry, but your roof is probably bad. If you can get the money you paid four years ago back (doubtful), I would, but you probably need to get that entire mess torn off all the way down to the decking, and probably quite a bit of the decking near the edges of the roof too. You can tell because the shingles look "lumpy" and have a rolling look to them -- like someone's melted them, especially near the edges -- and the valley is curved instead of sharp. I can also see the lines running across where different runs of roof sheathing panels meet (every 4' or so); that it's sagged in between these lines is a sign of sheathing damage.

Your roof isn't the shingles, so it doesn't matter that the shingles are only four years old. Your roof is actually the tar paper and ice and water shield that is underneath underneath the shingles. Multiple layers of shingles are bad; it means that they didn't replace the actual stuff that keeps the water out of your house, they just replaced the stuff that you see that goes over top.

If you go up into your attic when it's been raining quite a bit, see how much more of the underside of your roof sheathing is damp. If there's a lot, you need to get it torn off and replaced as soon as possible. If there isn't much, you have a few years to save up for it, but it looks bad now from the top. Make sure when they re-do it that they tear everything down to bare wood, remove all the nails and staples and junk, use Ice & Water Shield along the roof edges and in the valleys, and then put new 30 year tar paper and shingles over it. The tar paper and ice&water shield is what actually keeps the water out of your house. The shingles only protect the tar paper from abuse.

"Shingles torn" "Nails showing"

You very probably have leaks you have not yet noticed. If you re-roof now, you might not rot out the roof deck and need to replace it as well. If you wait until you have leaks you notice, you may find that you're rotted out the roof deck and be looking at a far larger bill.

However - don't hire ANY contractor that drives by, looks at your roof, and rings your doorbell offering to replace it for you. That sort of "drive-by marketing" is typically associated with the absolute lowest form of "wannbe contractors", often mobile units that won't be around when the new (shoddily installed from the worst materials) roof leaks. Look up established local firms and invite them to make a bid.

Your insurance adjuster is wise to the tactic you plan to engage in - if the roof was not maintained in good condition, they will probably deny the whole claim (you neglected the house) rather than buy you a new roof. They know what old shingles look like, even after they have been torn up by bad weather. They probably even have "before" pictures of it, with dates, in their files.