Stair stabilization

How much load can 2x3's or 2x4's attached to concrete block support?

Typically a 2x on end can handle about 1250 PSI for doug fir. This is about 6,000+ lb for a 2x4, 4,500+ lb for a 2x3. Crushing wise, it depends on how often it is braced. You anchor a 2x4 into a concrete wall every 6" for 5' and I am sure it will actaully hold those loads.

Is this different than simply summing up the fastener tolerances and accounting for safety margin, i.e. 1590lbs / 4 (safety) = 397lbs per stud?

Yes. The 2x's can hold one load, the fasteners can hold another. Typically you specify fasteners based on the actual loads to be held, not on the possible loads each stud can hold.

Does the choice of stud (2x3 vs 2x4) AND the choice of orientation (long side vs short side facing out) matter in terms of supporting this load? If so, how?

Yes and No. The failure mode for a 2x is usually buckling. So putting it flat against the wall would make it stronger laterally by bracing it in the weak direction allowing more load before buckling. But if you brace the 2x laterally via wood and face them out, this is a moot point.

But if you framing walls inside of a fully supported basement, why does any of this matter? The walls are not supporting any other laod than the wall finish. Just shoot concrete nails into P.T. members placed vertical on the wall to fur it in and cover a suitable finish. Maybe I am missing something.

Yep, doing another pair of sticks under the landing where the chimney used to be is quite enough. Slip a galvanized plate under the existing & new leg to prevent moisture wicking. With your new sticks, anchor the first stick to the cinder block, even with just Molly or Mushroom Head Bolts before attaching the other stick.

This will avoid any possible future racking or pull-away type movements. Not that it would, but you can also avoid the anchors by doing an X-brace of sticks between the then 2 supporting legs or posts, so you don't make any holes in the walls.