Your player has the right of it, mostly
All of the following information can be found for free in the D&D 5e Source Reference Document PDF as well as on the official 5e online supplement D&D Beyond. Page references are included for the information in the SRD PDF. All of the content quoted from the SRD can also be found in the core rulebooks.
Paladins and clerics both say the following under their spellcasting class feature (SRD pgs. 16, 31)
You can use a holy Symbol (see "Equipment") as a spellcasting focus for your [cleric/paladin] Spells.
The general rules for spellcasting foci as related to material components are as follows (SRD pg. 102):
A character can use a component pouch or a spellcasting focus (found in “Equipment”) in place of the components specified for a spell. But if a cost is indicated for a component, a character must have that specific component before he or she can cast the spell.
In particular, the holy symbol states (SRD pg. 67):
A holy symbol is a representation of a god or pantheon. It might be an amulet depicting a symbol representing a deity, the same symbol carefully engraved or inlaid as an emblem on a shield, or a tiny box holding a fragment of a sacred relic... A cleric or paladin can use a holy symbol as a spellcasting focus. To use the symbol in this way, the caster must hold it in hand, wear it visibly, or bear it on a shield.
These are the only rules as far as the book is concerned. Clerics and paladins can use a holy symbol as a spellcasting focus to replace the non-costly, non-consumed material components of their spells. No stipulation RAW is made that the holy symbol must be of an exactly certain quality, type, or even a match to the deity that the cleric or paladin worships. From these quotes, we can determine that:
For spellcasting, the holy symbol is only required when you need material components
The holy symbol is a spellcasting focus. Spellcasting foci by design are only used to replace non-costly, non-consumed material components. If your player is casting a spell without material components, they do not need their holy symbol at all and can cast it normally. If the player can provide the actual material component, they can also cast it normally, even without the spellcasting focus.
speaking of material components...
Component pouches can be used by anyone
The rules for component pouches are as follows (SRD pg. 67):
A component pouch is a small, watertight leather belt pouch that has compartments to hold all the material components and other special items you need to cast your spells, except for those components that have a specific cost (as indicated in a spell's description).
There are no other rules for pouches, aside from those general "spellcasting foci" rules I quoted above. Any spellcaster can use a component pouch for basically the same purpose as a focus. More on that topic can be found in this question.
If the player gets a component pouch, they can use it to cast spells that they would normally use their holy symbol for. The flavor is that the character stores the actual non-costly material components in the pouch. So, as long as the character has access to the pouch, they are actively providing those material components that a spell calls for. This, as was described above, allows the character to cast spells that require material components, even without a focus.
However, non-spell channel divinity features may require holy symbols
For example, the cleric's turn undead channel divinity says (SRD pg. 16, emphasis mine):
As an action, you present your holy symbol and speak a prayer censuring the undead.
If the player has any non-spell action such as turn undead, they need their holy symbol to use it. In this case, your cleric cannot turn undead until they have a replacement holy symbol, since they have no holy symbol to present.
Additionally, these options are not spells, so the component pouch cannot be used as a substitute in these instances.
How they replace the lost symbol is up to you.
Prices for holy symbols are found on the general equipment list (SRD pg. 69), so in theory it could be as easy to replace as going to a friendly church and buying a new one. There are no rules for "blessing" a holy symbol or the like. The description of the holy symbol item does say they can be "carefully engraved or inlaid as an emblem..." so it would imply that the symbol is more complicated than "crudely carving it out of a piece of wood", but ultimately, the DM is the one who gets to make the call on if such an attempt would work or not.
Worldly Focus
Weird name for the effect, but this feat from Faiths of Eberron allows you to cast cleric spells without a divine focus. Note that it does nothing for turn undead, or any other usage of a holy symbol, which seems like an oversight to me but then so too does the lack of holy symbol in Vow of Poverty exceptions. But then, who actually uses turn undead to, ya know, turn the undead?
Summon Holy Symbol
I had originally suggested summon holy symbol, as follows:
An orison from Complete Champion that does what it says on the tin. Since the summoned holy symbol is a temporary magical creation per the Conjuration (Summoning) rules, it has no value and so (arguably) should not count as a “material possession” and cause trouble for a cleric with Vow of Poverty. This is a real holy symbol that can do everything a holy symbol can, but only lasts 1 round/level.
However, careful reading of the Conjuration (Summoning) rules indicates that summoned objects—unlike creatures—do not go away when the spell ends “unless the spell description indicates this.” Note that, even in core, there are no Conjuration (Summoning) spells that seem to remember this rule: summon instrument reads almost exactly like summon holy symbol, for instance. This makes absolutely no sense, but that is what the rules say, both in Player’s Handbook and Rules Compendium. My guess is that this rule was intended for instant summons (which, unlike most summoning spells, brings a particular item with your arcane mark to you) and maybe secret chest (which has its own special rules anyway), but it’s in the general Conjuration (Summoning) rules, not in the description of those spells. As a result, since summon holy symbol says nothing about the holy symbol disappearing, rules-as-written this orison can just create a permanent holy symbol out of thin air, and its round/level duration is meaningless since nothing happens once the duration ends.
That means the summoned holy symbol is no longer temporary, and there is no particular reason to think it should be value-less (Complete Champion indicates that it is not “particularly valuable” but that seems more to mean that it is the ordinary 1-gp holy symbol rather than one of “particular” value, rather than a holy symbol that is literally worthless). It is therefore difficult to argue that this spell is safe for those with Vow of Poverty.
Best Answer
It's not required
The rules on concentration can be found on page 203 of Player's Handbook. Concentration is only specified to end in these four ways:
There is no mention of one having to maintain any of the components, including the material ones, of the spell to maintain concentration.
Item descriptions are found on page 151. Under Holy Symbol, it is simply stated that the item can be used as a spellcasting focus, without special rules regarding concentration.