After consultation with my master-carpenter-looking-over-my-shoulder, Dan, the following solution was arrived at:

1.The newel post - built up from 1x3 clear oak boards glued together. Two are full length, one is beveled (before gluing) to sit on top of angled knee wall. Attached by two lag bolts at base and one lag bolt into handrail.

2.Base - shoerail molding (with trough cut out to hold balusters)

runs up the angled knee wall. The shoerail is cut into two sections directly below the spot where the handrail meets the ceiling. This will allow the railing to be separated into an upper section, bounded by the base and top rail, and a lower section, bounded by the base and handrail. It is attached with 1 screw in the middle of the upper section between two balusters. The lower section is attached with two screws, also inserted between balusters.

3.Top - shoerail is also used at the top in the same manner as the base. Two screws hold the top shoerail to the ceiling (screwed directly into studs at the ceiling/stairwell edge).

4.Handrail - Standard handrail (with trough) held in place at newel by one lag bolt and at ceiling by one long screw upward into stud.

5.Balusters - These are toenailed into base shoerail and top shoerail. For extra strength, blausters are glued before nailing (Nailholes will be covered by fillet pieces).
6.Fillets - these are the filler strips put into the shoerail moldings top and bottom to lock in the balusters. They are nailed into place after the balusters are inserted in the shoerial. The fillet peices that sit over the attachment screws, top and bottom, are not nailed in, but held by small recessed screws so that the can be taken out.

All nailing was done with an 18 gauge nailgun.
Removal involves
a) unbolting the newel post,
b) removing the newel post,
c)removing the fillet pieces covering the screw holes in the base and top shoerails,
d) removing the two screws holding the lower shoerail base in place,
e) removing the upper handrail screw at the ceiling,
f) removing the lower handrail/base/baluster assembly,
g) removing the two screws holding the top rail,
h) removing the screw holding the upper section of the base rail,
i) removing the upper top/base/bauster assembly.
Something for your consideration, if you can have the railing modified or corrected. Don't mount it to the stairs at all if you absolutely don't need to, which you don't. If the railing hit's the house & overruns the bottom step or is long enough. Attach it to the building's exterior wall & the ground or pavement at the bottom of the stairs.
Bridging the stairs is one of the proper ways to do a short railing. The other way is for stairs that closely match the width of the door. The railing only stands next to the stairs but is attached to & into the ground's pavement.
Best Answer
Use the core drill you mentioned before, make sure it is deep enough, say 4" to get to the bottom of the iron stub. That is the standard mounting depth.
To keep the drill in place, drill a hole in a piece of 3/4" plywood big enough (1'X4'??)to lay on the stair you need to drill with the hole in the place that the old post is located. Stand on this while you use the drill to cut the hole. Get another body on it too if you feel you need to, but if you can maintain good balance you will be fine. drill it down about 1/2" and set aside the plywood and finish the depth of the hole. you can always chip a little of the concrete out, to go deeper, but you will find that a well placed tap with a hammer and chisel or steel pin to the side will release the plug that you drilled.
The plywood will stay in place very well with the pressure of you standing on it, it does not take a whole lot of pressure to keep it from "dancing".
If you can use a water fed core drill.