Your Interpretation is Correct

According to Jeremy Crawford:

Dispel magic is cast on a creature, an object, or another phenomenon that is under the effect of a spell. You don't cast it on that spell's caster. To dispel a spell like banishment, you'd have to somehow cast dispel magic on the banished target.

Of course, since Banishment is a concentration spell, there are options that you could take to attempt to end the spell if you are on the same plane as the spellcaster but not as the target (like casting Magic Missile at the person concentrating on it). And like you said, if you could somehow end up on the same plane as the banished creature, you could dispel the Banishment spell there. But yes, specifically the Dispel Magic spell wouldn't work unless cast upon the target of the Banishment spell.

It is rarity "nobody will want to use it"

Let's be honest, it doesn't really matter if a weapon is Common, Uncommon or Very Rare, as those are simply terms the DM can use to determine when a character should get an item, and it sounds like this is an item you'd really like your players to have. What really matters is how desireable an item is, and this thing is not desireable at all.

At first glance it might seem like an awesome weapon for somebody to use, but that novelty will quickly wear off the first time the fighter accidentally banishes the big bad into another dimension for a minute.

Because you have no control over when it happens, this weapon is going to lead to a lot of frustration when it happens at the wrong time and nobody can attack the thing they were fighting for a full minute, causing concentration buffs to run out and making concentration spells that were already affecting the creature end because it's suddenly on another plane of existence.

It will throw carefully planned attacks into complete disarray and the effects are entirely too random to be of any help. You can't even use it to make a quick escape if things go wrong, because you would have to be able to roll a 20 on demand somehow, and if you can crit on demand, you might as well just kill whatever it is you were fighting by critting a few times.

Want to make it attractive?

If you want players to actually use it, giving them the option of turning this effect on or off would go a long way at making it more attractive. It's still going to be extremely random, but at the very least you won't accidentally ruin your party's plans.

You'll likely also want to make it a +1 weapon, at which point it could be a decent enough Rare weapon for a fighter, and the effect might prove to be useful once every dozen or so sessions.