This should work, if it's the last thing you do before going to sleep.

Outside of combat, we typically don't account for actions, but this invocation would require keeping track of whether you do something that would be an action. For example, you can eat during a rest, but that would use an action if done in combat, so it would break the invisibility (because the magic doesn't care whether you call it an action).

However, if the warlock goes into an enclosed, dark space, lies down, invokes the magic, and then goes to sleep, yeah, they can be invisible until waking up.

A caveat about doing this during a short rest

(not actually the question, but it seems relevant)

The short rest rules mention "tending to wounds" as a possible activity, and allow regaining hit points by spending Hit Dice, but don't actually say that you have to tend to your wounds to regain hit points. I would argue that the rules imply that wound care is an expected part of resting, and if you insist on spending the entire rest invisible instead of caring for your wounds, you can't spend Hit Dice. But depending on how your table treats the rules, and on how you interpret hit points, this could go either way.

Devil's Sight can see through Maddening Darkness, but doesn't protect you from the damage

Excerpted, Maddening Darkness has the following effects:

Magical darkness spreads from a point you choose ... Non-magical light, as well as light created by spells of 8th level or lower, can't illuminate the area.

... Whenever a creature starts its turn in the sphere, it must make a Wisdom saving throw, taking 8d8 psychic damage on a failed save, or half as much damage on a successful one.

As the first effect of Maddening Darkness is simply to produce magical darkness, that particular aspect does not affect a Warlock with Devil's Sight, who can see through magical darkness without issue.

However, the second effect does not depend on the darkness, it simply applies to the same area (it's in a separate paragraph from the description of the darkness and doesn't reference the darkness at all, either). Anyone starting their turn in the area takes the psychic damage that the spell deals - seeing through the darkness doesn't change that, and the spell has no exception which makes the caster immune. You could even illuminate the area using a light-causing spell cast at 9th level, but that would not prevent the damage.