There's no problem, conceptually, with removing wiring that doesn't do anything. But your problem isn't the should I, it's the can I.
So to answer your question: Yes, feel free to remove it.
But you have to figure out how to squeeze your arms behind plaster walls and pull staples. :(
Arc-Fault Circuit Interrupter (AFCI)
An arc-fault circuit interruption device is designed to detect dangerous arcing within the protected circuit, and open (turn off) the circuit to prevent damage caused by the arcing. It does this using special circuitry to analyse the electrical characteristics of the circuit, looking for characteristics that match specific pre-programmed values. If the AFCI detects suspicious goings on, it opens the circuit.
AFCI breakers are typically combination devices, meaning they also provide similar overcurrent and short-circuit protection to a standard breaker.
Installing a combination AFCI breaker on a circuit containing knob and tube wiring would be a great idea, and could potentially prevent a fire.
Ground-Fault Circuit Interrupter (GFCI)
Ground-fault circuit interruption devices are designed to detect ground-faults, and open (turn off) the circuit when a ground-fault is detected. They work by using a current transformer (CT) to detect current imbalances between the ungrounded (hot), and grounded (neutral) conductors of a circuit. This blog entry might help you understand how GFCI devices work.
GFCI breakers are typically combination devices, meaning they also provide similar overcurrent and short-circuit protection to a standard breaker.
Installing a GFCI breaker on a circuit containing knob and tube wiring, probably won't provide any benefit. GFCI devices are designed to prevent electrocution, not to protect the wiring.
Combination AFCI GFCI Circuit Breakers
Circuit breakers that provide AFCI, GFCI, overload, and overcurrent protection are becoming more widely available. If you can find them for your panel (and afford therm), these would be the best option.
Best Answer
It can be and frequently is replaced piecemeal, but I'd certainly be in favor of wholesale replacement if it were my house.
If we accept "1880s-1930s" as the range of its general use (wikipedia, not any particularly great research effort) it's 80-130 years old. And any impact it's having on your insurance rates won't end until it's all gone.
Plus, you can insulate the walls, too, once it's gone. And you'll have ground wires.