Rigid foam board expansion

I am also in the Northeast and have had to tackle this type of situation a few times. There are several ways to attack it.

In your case, the best solution, but unfortunately the most expensive would be to use an open cell foam kit to fill the joist bays. They are fairly easy to use and give you about an R-4 rating per inch. They are a two tank kit with a hose and wand. Here is an example product: Touch N Seal 1000 Kit Open Cell Spray Foam Insulation

These type of kits start around $400, depending on the square footage and depth required.

A cheaper method would be to use 24 inch R-19 unfaced blanket insulation cut to your custom widths. You can use wire springs to hold it in place and keep it from falling. I would then cover the entire area with 4 to 6 mil plastic, stapled to all the joists. Tape the joints, if any. Moisture should not be a major problem since you have already done what sounds like a good job sealing the dirt floor. In the summer months, some ventilation in the crawl space will also help avoid excess moisture.

Rigid foam glued to the foundation walls would also help if the area is fairly air tight. If there is free air flow, insulating the brick walls would be a waste of time and money.

Yes, Rigid foam (XPS and ISO) boards are frequently used in exterior applications, and can be a great option for improving efficiency. There are a couple of issues with your proposed approach, though:

1. Vapor barrier location. The vapor barrier should be on the warm side (probably inside, unless you live in a very hot & humid climate). Ideally, this is immediately behind your interior drywall. You also do not usually need an extra vapor barrier layer adjacent to rigid foam sheets, they are typically highly resistant to moisture.
2. Attachment. You cannot rely on glue to support foam and siding panels. These need to be mechanically fastened.

On a wood frame house, you'd want to use this order:

• interior drywall
• possibly a vapor barrier (e.g. poly sheeting), though the wisdom of this is debatable (see the comment from iLikeDirt below)
• wall studs (optionally with batt or cellulose insulation in between)
• exterior sheathing
• rigid foam
• wooden strapping for attaching siding, screwed to studs through foam. This provides support for the siding and also allows air circulation behind the siding, so any moisture that gets behind the siding can dry out.
• siding panels, nailed to strapping

Here's a visual from Fine Homebuilding, which has a couple of good articles on this:

Note that adding foam on the exterior of your house complicates some things. You will likely need to redo the trim on your windows and doors, which will become more in-set. These inset areas will need to be flashed properly or you will have leaks, which have more potential to do damage when you've got multi-layered, thick walls that could trap moisture. If done well, you can get very good insulation. But make sure to work with someone who knows what they're doing!