Yes, the Star card can permanently increase your ability scores beyond the normal limit of 20.
The Player's Handbook, on page 15 in the section about applying ability score improvements, states the following rule:
Some of these features allow you to increase your ability scores, either increasing two scores by 1 each or increasing one score by 2. You can't increase an ability score above 20.
Note that, specifically, this does not say "you cannot have an ability score above 20"; it states that you cannot use ability score improvements to increase an ability score above 20 - the rule only has importance at the moment you are altering your stats.
Following standard "specific beats general" interpretation, the Star card offers you an ability score improvement which specifically clarifies that it can, in fact, raise your ability score above 20, up to 24. Having applied the benefit of the card, if your ability score is now 20 or higher, you can't improve it further with a normal ability score increase, as that's subject to the general rule that you can't increase the score above 20. But there's no rule that states that, if you have legally increased your score past 20 by some means, it has to go back down again. It's not really so much a "cap" as a threshold - it's difficult to cross but you're not forced back if you do it.
Fix it in the real world
You're already basically off of RAW with a +4 weapon. 5e RAW only has +1 through +3 -- the Moonblade artifact even specifically calls out that you reroll if you roll the 'increase bonus' rune and the weapon is already at +3.
But laying that aside for the moment: This isn't a situation you can really solve in-game. You need to have this discussion outside the game, player to player. At least some of the players are already unhappy with what's going on, so this is the time to stop thinking of yourself as the sole arbiter of the game, and collaborate.
If you try to fix this by fiat, all you're going to do is upset your friends. Putting together a plot that ends with the moonblade and other powerful weapons lost, destroyed, or depowered is fine -- but you're going to have to get player buy-in for that first.
Have a discussion with your players. You don't have to work out plot details, but talk it over. Figure out what THEY are willing to do. Lay out the problem, and ask for suggestions. Not everyone will be excited to lose their cool gear, but only the most childish player will absolutely refuse to give up their awesome sword if it really is causing a problem with the game.
First off, realize that 5e is designed so that nobody NEEDS a magic sword to be perfectly capable. A +2 to attack and damage is nice, but not necessary. It's really not a big deal to be a 7th level fighter with a plain old steel sword. Your damage output is going to be only slightly lower than with the sword. If you really want to get a magic weapon into the Eldritch Knight's hands, that's fine, just let him use a standard weapon until it's narratively appropriate. There really shouldn't be any rush.
For the rest, the easiest answer is to just retcon the overpowered weapons into less powerful, but similar, weapons.
For the moonblade, I assume you rolled it as specified in the book, with d6+1 runes, with the first giving it +1 and the rest rolled on the power table. If so, then maybe the player will be willing to hand-wave away a few runes. If you rolled 6 runes initially, and it's messing stuff up, maybe your player would be willing to cut it down to 3 runes, with only the initial +1 giving it the damage boost, and the others giving powers rather than more plusses. Just retcon the whole thing to a less powerful version of the item, agree out-of-character that it's always been that way, and move on. (Moonblades are specifically useful that way since they can scale.)
The holy avenger might become a slightly less powerful 'holy sword' or some such thing, say a +2 weapon that deals +1d6 radiant damage to fiends and undead, but doesn't do the whole save bonus thing.
I don't know what a Zariel's cursed sword is but I imagine you can do the same sort of thing there.
If you and your players are unhappy with a simple retcon, then I'd suggest going the route of coming up with a reason the weapons have to be relinquished or destroyed, but again, you need to have at least some part of that discussion at the table, not just spring it on the players. In this situation, I'd collectively come up with a story that you can all be happy about, and then play it out (rather than you as DM running off to plot and plan and come up with a surprising story) -- but if you really don't want to do that, at least get player buy-in on you going off to develop a plot around removing the weapons in some way. If you end up destroying the weapons, it might be worth it to let the players keep some shards of their original weapons (or some such thing) that let them forge weakened versions of the weapons, as I discussed above.
But I suggest getting the players to help craft the story of why the weapons are gone (if you don't just retcon them). That's how you keep the players happy -- because what's happening isn't being done to them, but by them.
Plus that avoids the biggest problem with a fiat removal of the weapons -- somebody trying to weasel out of it. Worst case, somebody manages to succeed at a roll or comes up with a plan you didn't expect, and you're forced to concede that they managed to save their weapon from destruction, which negates the whole point (often supported by liberal use of 'My Guy syndrome' -- "My guy wouldn't just let that happen! He tries to escape!"). If the players made up their own fate, there's no motivation to try to dodge it. It was their own story.
Best Answer
The Deck of Many Things is in the DMG for a reason - it requires DMG adjudication. There are no further rules because it's up to the DM to decide. It's a plot/story call, not a rules/mechanical one.