If possible, run the side session on another time or day with just those participating
We've actually done this a number of times in a campaign I'm playing in. The players or DM occasionally come up with side quests relevant to a select few players, who we then find a time to play through an another day of the week. This prevents it from disrupting the group as a whole by keeping them from playing, while allowing for the side-quest to still take place.
The smaller group also helps the interaction go much faster. One session was literally the DM and I on a 15 minute car ride, playing in the theater of the mind. Another one was just my coming along to assist on another player's side quest, and only took us half an hour. Yet another of mine took about an hour, and was very similar to what you described above (me, the rogue, on a secret thieving quest).
So yes, this is entirely possible and normal. As I mentioned above, just do your best to avoid disrupting the other players, either by making them sit through it, or by making them have to skip what would have normally been a group session altogether.
Be aware though that by allowing this for one character, it may open up the door for others to request the same. This doesn't mean you have to oblige, especially if you only intended this to be a one-off opportunity for help the other player connect to their character better. But be prepared to know how you intend to respond and handle such requests.
Impact to the main story
This part is a little more dependent on you as the DM, and how you want this to affect your world. In the same way that backstories can (and should) affect the overall plot to increase character engagement, so can and should present stories. Ideally you might want to avoid consequences that take agency away from other players in their part of the story, but that doesn't mean it can't be relevant.
For example, in my thieving side quest I mentioned above, completion of the task got me into a city-wide underground criminal network. This allowed me to help other members of my party as an informant, who could get information on quests and goals we as the whole party sought, and helped increase our group dynamic. This had an affect on the story by helping to improve my investment in it and provided another resource for the other players, but didn't take anything away from the others.
Side-note: Handling XP
As NautArch suggested in a comment below, one thing to keep in mind is how this might affect XP gain. Depending on what method of leveling you use (standard XP, milestone, 3 pillar, etc), you will want to be careful that these side quests don't give someone an unfair lead over their fellow players. In our campaign, side quests typically only granted XP to those who were behind the others, often with the quest itself being an intentional way for that player to catch up.
As someone who typically attended every session and was at the lead in XP, I would often agree with my DM to forgo it altogether in order to simply enjoy the adventure, or in other cases would join on someone else's side quest (if they were ok with it) as a way to help them level their character faster. Again, this will all depend on your group and how leveling works, but so long as the side quests aren't giving them an unfair/special advantage, you should be good.
My first major campaign was almost exactly this structure, and it was a lot of fun. The setup was, the land was in danger of destruction due to the loss of eight magical load-bearing MacGuffins, so the rulers of the land offered enormous prizes for anyone who retrieved one of the MacGuffins. In my case, the PCs weren't aware that the rival party were evil until around halfway through the campaign, but most of the methods I'll describe will work even if the PCs already know the rival party are evil.
Exit Strategies
The rival party doesn't want to get squashed by the PCs - they have a goal to achieve, and unless the players manage to set up a situation where the only possible way for the rivals to win is by murdering the PCs, the rivals will avoid direct conflict. They'll take alternate routes to those used by the PCs, or try to get there first (or let the PCs get there first, if a route is dangerous). If the PCs do manage to encounter them, they won't stick around to talk or fight - they'll have teleportation magic, flight, or other fast-exit options at hand, and will use them early.
Minibosses and Goon Squads
Like you, I didn't want my players to immediately kill the rival party, but I still wanted to have conflict with them. I did this by giving the rival party a number of minions in the form of goon squads (lower-threat faceless mobs), plus a couple of named minibosses. In my case, the PCs didn't know right away that the minions worked for the rival party, and had an angry aha! moment when they figured it out. In your case, you can use minions to build up (good) frustration in your players as they keep having to fight the minions rather than the rival party itself.
Side Arc Antagonists
Your PCs don't always have to be directly opposing the rival party or its minions themselves. Especially on a MacGuffin quest, you can add in side arcs: find the key to open the dwarven door in Mount Doom, traverse the abandoned mines to reach the Sage of Sight, get directions from the Sage about how to avoid the traps protecting the MacGuffin, etc etc etc.
In each of those arcs, the PCs will be fighting not the rival party, but a unique antagonist to that arc: the dragon whose hoard contains the key, or the goblin king who's taken over the mine. The PCs can still make progress toward thwarting the rival team, either in obvious ways (by getting the key first) or subtle ones (by convincing the Sage to give the rivals incorrect information). But they aren't ever in a situation where they could kill the rivals - only hinder them.
Common Goals
(This suggestion is less useful in your specific case, but including for completeness)
In my game, many, many "teams" (aka NPC parties) participated in the quest for the eight MacGuffins, and the specific rival group was only one of those myriad teams - albeit one which the PCs kept running into. Those encounters allowed me to hint at the rival party's true, evil nature, while the fact that all eight MacGuffins were needed to save the land meant that the PCs were more comfortable pointing the rival party at some other MacGuffin rather than outright killing them (at least until the reveal). By giving the appearance that the PC party and the rival party are working toward a common goal, you can build up a (maybe not-so-)friendly rivalry that isn't hostile enough to warrant murder.
tl;dr
When running a campaign with a rival party, give your players lots of antagonists to focus on which aren't the rival party themselves, give the rival party multiple effective exit strategies, and build in story reasons for the PCs to work with the rival party instead of murdering them outright.
One final note: your players may surprise you by finding a way to kill the rivals early despite all your precautions. If so, roll with it! Your plot doesn't have to be over just because the PCs outmaneuvered your villain. Add in a surprise Greater-Scope Villain who was using your original villain as a cats-paw, or a reveal that the MacGuffin itself is evil or dangerous, and the rival party was actually trying to stop it being unleashed, or some other twist. Your players will always surprise you, so make sure you're prepared to surprise them back.
Best Answer
Sounds like Epic Adventures from Adventurers League.
An Epic Adventure has a structure consisting of 1-hour missions, each played by a different table. Every mission has a different objective and grants different bonuses when completed. There are also events that, when triggered, grant immediate bonuses or penalties to all tables.
You need another DM
Each table has their DM, so they can play simultaneously. You need to have another DM to take care the other party and communicate if the other party affect your party ("The guards successfully turned on the emergency light, your stealth rolls now has -2 penalty").
However, if you are planning to solo DM, I suggest to convert the "hide-and-seek" showdown to a combat showdown, ended with either the capture of the thieves or their escape. It is more manageable, without losing the narrative conclusion.
If your group is not into combat, you can change it to a skill showdown.
Using this method, you as the DM have to adequately interpret and narrate the result of their checks to build the tension toward the final skill contest.
Last note, "hide-and-seek" with traversing the map round by round will be tedious and takes a lot of time, even with two DMs. That's why I suggest you to ditch the hide and seek and use alternative.