Yes, you can cast spells with a Material (M) component without issue.

Your Holy Symbol takes care of it.

This is one of the more difficult things to look up in the current 5e rules. It starts on page 58, under Cleric spellcasting:

You can use a holy symbol (found in chapter 5) as a spellcasting focus for your cleric spells.

In the equipment section, page 151:

... A cleric or paladin can use a holy symbol as a spellcasting focus, as described in chapter 10. To use the symbol in this way, the caster must hold it in hand, wear it visibly, or bear it on a shield.

Finally, page 203, under Material (M):

A character can use a component pouch or a spellcasting focus (found in chapter 5) in place of the components specified for a spell.

Put it all together: You can use a visibly worn holy symbol (no hands required) in place of the material components for a spell.

Somatic Components

What's more likely to trip clerics up are somatic components, which ALSO require a free hand but aren't covered by the holy symbol.

Fortunately, the rules are pretty lax on this point. Page 203 outlines interacting with objects:

You can also interact with one object or feature of the environment for free, during either your move or your action. For example, you could open a door during your move as you stride toward a foe, or you could draw your weapon as part of the same action you use to attack.

(See this answer for more info)

A very strict reading of this would allow the cleric to sheath his weapon, cast his spell, and then draw his weapon again the next turn (giving up opportunity attacks for a turn).

OR, you can drop your weapon (free), cast your spell, and then use your free interaction to pick it up again.

OR, pick up the War Caster feat (as you have).

OR, just ask your DM to go easy on you...