How does this homebrew cantrip compare to other cantrips, and certain levelled spells

Answers in order

1. The shaken condition caused by demoralizing a foe via the Intimidate skill (facilitated by the blistering invective spell) is replaced by the frightened condition caused by the bard employing a combination of the bard's supernatural ability dirge of doom and the benefit of the feat Improved Dirge of Doom. At least, the condition is replaced for as long as the supernatural ability dirge of doom continues to affect the foe; the affected foes remain demoralized—that duration continuing alongside yet subsumed by the dirge of doom effect—but not also technically simultaneously shaken as fear conditions escalate.

   That is, the supernatural ability dirge of doom doesn't care how the foe gained the condition shaken nor how long the foe would have remained shaken. The dirge ability only checks that foes are shaken and, if they are, then the dirge ability thereafter follows its own rules—until, of course, it can't any longer, like when an affected creature's out of range.

2. A creature that escapes the range of the dirge of doom ability is no longer affected by the dirge of doom ability. If the duration of the creature's demoralization has yet to expire, the creature will become demoralized upon exiting the area. Time spent fightened by the dirge ability, however, counts toward the demoralize effect's duration.

3. To be affected by the dirge of doom ability, "an enemy must be within 30 feet and able to see and hear the bard’s performance," but "[t]he effect persists for as long as the enemy is within [range] and the bard continues the performance." When the enemy who was affected by the dirge ability isn't any longer within range, the dirge of doom effect ceases for that enemy, and that enemy "cannot be affected by it again for 24 hours."

   Thus a bard that's playing spooky music can't just chase after a dude who's already successfully gone beyond the dirge of doom ability's range so as to have the bard's spooky music scare the dude again, despite that chain of events seeming to mesh seamlessly with the adventures of Scooby-doo.

4. The trait Memorable, in part, says, "Whenever you create a fear or mind-affecting effect that imposes a penalty or bonus with a duration of at least 2 rounds, it lasts 1 additional round after it would normally end." While the frightened condition engendered by the benefit of the feat Improved Dirge of Doom does impose a penalty, the supernatural ability dirge of doom itself does not (cf. the spells bane and stage fright that each possess the descriptor fear and that each cause a creature to suffer a penalty directly).

   With this in mind, this GM would err on the side of caution and grant the bard that possesses the Memorable trait no particular advantage for the trait when he uses the ability dirge of doom. However, this GM admits that this ruling may be controversial (q.v. this 2016 Paizo messageboard thread discussing what seems to be an outdated version of the trait Memoriable), so I suggest asking your own GM for a ruling.

First, I want to say that I like the idea and effort put into making original and thematic abilities for this subclass. But, I think, there may be some balance issues.

Expanded Spell List - nice spells.

Hemmorhagic Shock - nice utility ability to have. You forgo a chance to stand up on rolling 20 on stabilization to guarantee that you won't die. Teammates would still have to spend resources to heal you back into combat. On the other hand - due to this ability they may not worry about hitting your unconscious body with AoE spells. I'd give it 3.5/5 power-wise, since it's a very useful ability, but limited only to certain situations - and healing of those, who are dying is relatively easy in DnD 5e.

Blood Boil - damage is relatively low to be meaningful past starting levels, but against many low-hp enemies may be nice. Recharging at long rest means that it's impossible to spam this ability. 3/5 power-wise - because ability has a nice damaging potential (even on level 1, if there are 8 enemies around warlock, it can deal 32 ((1lvl+3CHA)*8) damage) and it's not dice-based, but it requires to be extremly lucky with positioning and such position (in the middle of enemy group) tends to be quite risky. Also, it damages nearby friendly characters.

Also, most warlocks get only one ability in adition to Expanded Spell List at first level. Exceptions are only Hexblade, Celestial and Fathomless

Crimson Tithe - that's a very good ability, maybe too good. At 6th level, when you get it - it can almost double warlock's hp ( and being recharged at any rest, means that it can be spammed quite often. I can think of several schemes to abuse it. (For example - take one of 'elf' races that, according to MPMM can finish long-rest in 4 hours, buy/capture a number or rats or other small critters. Upong finishing long rest squeeze one of rats, opting for knocking it out non-lethally. Use Crimson Tithe. Do short rest. Start the day rested, with temporary hit points and a charge of Crimson Tithe ready again) 4.5/5 power-wise. Celestial at 10th level grants more temporary HPs (54-60 at 10th, gettign to 90 at 20th) - but they are spread over warlock and 5 companions.

Sanguine Endurance - that one is also good. While warlock doesn't rely heavily of concentration-based spells that require constitution saves, they are still among three most often encountered in game (along with Dexterity and Wisdom ones). Extra hp is also good. 4/5 - power-wise. At level 10 warlock has 17-80 hp (not counting CON mod) if you roll for hp and 53 hp if using fixed values (not counting CON mod), so 10 hp is about 10-20% increase of HP pool.

Scarlet Dominion - that's ability that is too good. It's DoT with huge AoE that doesn't hit friendlies and allows warlock to evade hostiles, control them and buff friendlies. You don't even have to concentrate to maintain it, it just exists. Downside is that damage is not that great, but it doesn't allow any saves and can soften enemies - still it's an only reason to not call ability OP, but it's still - 5/5 power-wise, as it's hard to compare to anything what other warlock subclasses have.

Result - powerful subclass focused on combat. 5/5 in combat-oriented campaign. Many abilities give spikes of power at the level when they are granted that go down with further levels.

I'd suggest to consider flattening those spikes, by making abilities more level-dependent