[RPG] n (online) SRD for Cthulhu d20

My background: I've run some adventures using insanity themes. I once had a character who used insanity-magic and was about to go insane, when his player decided the campaign was too dark for him and stopped showing up at my table. I once had a character who used insanity-magic and went insane on purpose because he wanted to kill the party and destroy the world. Here's what I would do in future if I wanted to use insanity themes again.

I think you should start by looking at the Same Page Tool.

D&D 5e is (usually) about a group of heroes who work together to save the world. Call of Cthulhu is (usually) about a group of normal people who stumble into eldritch horrors, go insane, turn on each other, and die horribly. These are very different games. Which game do you want to be playing?

Suppose that a player deliberately drives their character insane, then uses that insanity as justification for killing the party and unleashing eldritch horrors on the world. Is that: (1) totally appropriate and in-genre, or (2) annoying because that player ruined everyone else's fun?

Suppose that the party as a whole embraces the use of insanity magic, becoming more and more dysfunctional and evil until eventually the group shatters and everyone dies. Is that what you wanted from the adventure? Or do you want something more along the lines of "people go a little evil for flavor, but not so much that it makes them lose"?

Decide in advance what sorts of actions are "in-genre" for your game, then make sure you and your players are in agreement about that. Once you've all decided what you want your game to look like, you'll have a much easier time getting it to look like that.

In terms of mechanics: the problem you're going to have is that D&D doesn't really do "permanent damage". Most reduction to ability scores goes away after a short or long rest. Even if you house-rule that certain spells cause permanent damage, the greater restoration spell can cure that stuff.

A lot depends on what your players will tolerate. If they're on board with it, you could improvise some sort of Sanity Points system for representing permanent sanity damage. Most players don't like having their character's impending insanity hanging over their heads, though. This could cause them to lose attachment to their character, or it could cause them to lose interest in your game.

I think a more awesome approach would be to impose narrative costs: when you use the forbidden magics, you take temporary Wisdom damage and you make a Wisdom check. If you fail, you release evil into the world. The evil could take the form of another monster you have to fight, or it could be that something you relied on has become foul and twisted and corrupted, or maybe it ignores your character and makes life miserable for the nearby villagers (depending, perhaps, on how badly you roll). I've done similar things in Apocalypse-World-based systems and it's been a lot of fun.