Skyrim – Do Shops or NPCs Replenish After Thievery?

When they're confiscated, they're placed in an evidence chest. These chests can be found in the jails themselves and in guard houses.

If you have the chutzpah, you can re-enter the jail once you serve your time and seek it out. It's usually around the same area as the belongings chest: the location is different depending on the jail, but checking the barracks or the entrances (or even sometimes right next to your cell) are generally a good bet.

So I tested @Mr Smooth's recommendation and it is indeed true.

If you have been caught, then you need to trigger a load* by doing a large zone transition. A small one (such as going from the ragged flagoon to the ratway) was not enough to cause the flag to be reset. However, transitioning from the cistern to Solitude was enough to cause the reset and I was able to immediately attempt to steal again.

In short, if you are in a town with a gate, then exit the town and go back in. Simply going into a house in the town is probably not enough to reset the flag. If you are not in a gated town, just fast travel to somewhere far enough that requires a load.