How are signs affected without swords

There are definitely repercussions.

I looted some things in the military camp near the first village, wanting to see if there were any repercussions (keeping in mind I did it in plain sight) and I promptly got my arse handed to me by the entire encampment.

They took some cash from my pockets and rather than reloading from last save, I 'woke up' in the encampment (sort've like how dying in GTA and other games works) and they weren't aggressive towards me any more.

Dunno...for the most part, the various witcher school gear is amongst the best you can craft. They are typically over-spec'd as it is, probably usable well into your 40's. However, if you venture about and do monster contracts and explore ruins/ tunnels etc. you'll find higher level gear than even that of your master crafted witcher school gear. I can't remember the name, but I found one such sword in a vampire's crypt, I think. Far south of Midscope, in an abandoned village overrun by ghouls. Do the monster contract and there and you should get this sword as a drop and/ or in the nearby chest. It's one of those relic sword designs that has an "S" shape running the whole length of the blade, like waves...looks cool. Kind of like a Celtic design. It's a level 37 steel relic sword and does more base damage than my master craft witcher cat school steel sword, although, my Cat School has the best attributes. What I wonder, though, is if your level determines your loot. I have run through this game 3 times(story) and the last two times I was getting a pretty sweet looking level 27 relic sword that had knuckle guards and a serrated blade, from near the lighthouse that is south of Novigrad, on an island and in a serpent's nest. This last time through, I was very low level and didn't get that same sword and instead, got some other level 17 relic sword. So you might not get the sword I got from that crypt, until you are around 30.