The Books are Silent
As far as I can see, the only rules for construction costs are on pg. 128 of the DMG. They detail the cosat for building various 'stronghold' structures. Nowhere does it tell you how much individual structural components of buildings cost.
Things to Consider
Given that the books say nothing, we will have to use common sense to determine a reasonable cost. The first thing to note is that this is a fine. The purpose of a fine was, and still is, to discourage people from committing the crime. Thus a fine usually charges much more than the stolen or damaged item was actually worth. I take an extract from this website I found with a quick Google search:
The thievery code in the Dalarna law called for a fine of three marks for pilfering one öre (a mark was worth eight öre).
As you can see, the fine for stealing 1 öre is 24 öre - a marked increase.
The next thing to determine is whether the property they damaged was that of a peasant or that of a nobleman (or other wealthy person). If it was a peasant's home, then it is safe to assume that they built it themself. If it was a nobleman's home, then he probably paid to have it built. For this reason, there would probably be a greater fine for damaging a nobleman's home than for damaging a peasant's home (a peasant would also be less able to press charges).
You also have to consider the character's ability to pay such a fine. Low level characters will not be able to pay a fine in the thousands of gold. Court systems today take the criminal's ability to pay into account. Maybe your court does too?
You might also want to consider whether a fine is really what you want. Other medieval punishments included:
- Flogging
- Time in the stocks (a pop culture favourite)
- Cutting off a hand or ear
One of those, or another inventive punishment based on your setting might be more interesting than a fine.
A Reasonable Cost
Let's look at the price of things in 5e. You can buy a chicken for 2cp (yes, that's copper pieces). A chicken was a valuable animal to have in medieval times. It would consistently lay eggs for food. If you bought a few and bred them, you could have a consistent supply of meat. A goat is 1gp (milk), a sheep 2gp (meat and wool), a pig 3gp (meat), a cow 10gp (meat and milk). All of these very valuable animals are extremely cheap by the player's standards (how many chickens could your players right now?).
A door is much less valuable than a cow. Jack wouldn't have got the magic beans for selling a door now would he?
So where does that leave us? We're looking at a fine of probably 1gp or less per door. The problem, of course, is that that sum is miniscule. You players will pay it without batting an eyelid. So what you need to do, is set a much much higher fine, not based on the value of the property destroyed, but on how much you think you need to charge to discourage this behaviour.
A Personal Experience
Once, my players decided to take some planks from a peasant's house. I can't remember exactly what they were for, but I am sure that the player's were well intentioned. Regardless, I decided that the town authorities would not stand for such vandalism, so I charged them a fine of (something like) 150gp. 150gp is an absurd fine for a few planks. It would have paid for the planks over a hundred times over. But that was not the point. I knew that my players could pay the fine, but also that it would discourage further such behaviour.
A Conclusion
Don't think about the fine in terms of how much the property they destroyed cost. Instead, think how much you will need to charge them to make sure that they do not do it again, because that is what fines are ultimately for.
Best Answer
If you accept answers from 3.5e
then
in 3.5e the door will break first, in 5e it's up to DM.
From the description of the immovable rod:
(emphasis mine)
Table: DCs to Break or Burst Items:
The DC for breaking the strongest door is less than the DC for moving the immovable rod. Both are strength checks. It means that the rod is “stronger” than the door.
Though I've never played 5E, I've looked into the 5e SRD and found that the description of immovable rod is almost the same there:
Unfortunately, I haven't found the same table for breaking objects, the SRD only says:
So, in 5e it seems that designers left the descision to a DM.
This is what we can know from the rules and their interpretation. To get more we need to refer to real world physics, but it is not safe, because real world physics often contradicts DnD physics.