It's neither the first attack nor the first round. As far as RAW is concerned, if you're surprised, you're surprised at least until the end of your first turn. From the Player's Basic Rules, page 69:
The DM determines who might be surprised. If
neither side tries to be stealthy, they automatically notice
each other. Otherwise, the DM compares the Dexterity
(Stealth) checks of anyone hiding with the passive
Wisdom (Perception) score of each creature on the
opposing side. Any character or monster that doesn’t
notice a threat is surprised at the start of the encounter.
If you’re surprised, you can’t move or take an action
on your first turn of the combat, and you can’t take a
reaction until that turn ends. A member of a group can
be surprised even if the other members aren’t.
The primary effects of being surprised last until the end of your first turn. It's not explicitly stated whether surprise ends when its effects end, but assuming the opposite would allow surprise to last for an arbitrary amount of time. If the duration of surprise isn't linked to the duration of the effects of surprise, we don't actually have any way to determine when surprise is supposed to end. So if the DM decides that you were surprised, you're surprised until the end of your first turn. Any attacks the Assassin lands on you before that will be critical hits.
For example: Alex the Assassin surprises Bob the Barbarian and Fred the Fighter.
They roll initiative. Alex gets a 15, Bob gets a 20, and Fred gets a 10.
Initiative count 20: Bob's turn. Bob can't do anything, but at the end of his turn he is no longer surprised.
Initiative count 15: Alex's turn. Alex could attack Bob, but the attacks wouldn't be critical hits, because Bob is no longer surprised. Alex could attack Fred, and those attacks would be critical hits, because Fred is surprised.
Initiative count 10: Fred's turn. Fred can't do anything, but at the end of his turn he is no longer surprised.
Initiative count 20: Bob's turn. Combat proceeds as usual.
Basic
I'm going to address the general case rather than the highly specific example in the question. General guidance will be more useful to other readers with a similar question.
That said, let's start with the rules text...
If neither side tries to be stealthy, they automatically notice each other. Otherwise, the DM compares the Dexterity (Stealth) checks of anyone hiding with the passive Wisdom (Perception) score of each creature on the opposing side. Any character or monster that doesn’t notice a threat is surprised at the start of the counter.
By definition, the whole group of player characters are trying to surprise the monsters, or none of them are. The less-capable ones can remain stationary and further away to facilitate that surprise, but they're still involved and still need to roll. The surprised/not surprised status is determined before initiative is rolled.
The reality is that the entire party is trying to be stealthy - some are just trying to do it while moving. The thunky rear guard group still needs to make Dexterity (Stealth) checks, but there are some modifiers that should be applied - advantage and disadvantage.
- The complete lack of motion could grant the folks hanging behind advantage on their checks - in all likelihood simply washing with the disadvantage from the armor they're probably wearing.
- The distance and obstructions could impose disadvantage on the passive Perception of the enemies, but only against the rear guard and only if they're far enough back that the distance is a hindrance to the targets. If they're just outside the room, no disadvantage should apply. If they're down the hall or around a corner, then it just might.
In effect, the advance team rolls Stealth with a DC of the targets' Passive Perception. The rear guard rolls Stealth with advantage against a DC equal to the targets' Passive Perception - 5.
From there, simply follow the normal rules for surprise. What they can do when the round begins can vary - maybe they don't have enough movement to get close enough to do anything, maybe they're forced to dash, maybe they have to use ranged attacks. There is no "Surprise Round" in 5E, as in previous editions.
Advanced
Because the rear guard is also ignorant of when the advance team actually strikes, it may be desirable (or at least somewhat more realistic) to consider them as three "sides" in the engagement, despite the fact that two are naturally allied.
Compare the rear guard's Passive Perception (you may consider giving them +5 for advantage because they know something is coming, just not exactly when) to the advance team's Stealth rolls, too, then follow the normal rules for surprise. This makes it possible for the rear guard to be surprised during the first round, making them a bit slow to engage because they have to wait for the sounds of conflict to know when to move in.
Alternatively, another method to reflect the rear guard's imperfect information would be to impose disadvantage on their Initiative checks. This could be interpreted or explained as them waiting until they're absolutely certain they heard the fight begin before moving in.
Advantage & Disadvantage
Advantage and disadvantage are among the most powerful tools at the DMs disposal for rewarding player thoughtfulness. They exist as a quick way to grant a benefit or drawback if the situation is good for a given participant. It has an almost minimal impact on speed of play, but can make players feel like their non-mechanical ideas have impact. What the exact advantage is doesn't matter. What the exact disadvantage is doesn't matter. If they have a big enough one to matter, apply it.
Best Answer
This is what the PHB says about scenarios that demonstrate one group surprising another:
So it's being shown to us that the encounter wherein creatures can be considered surprised begins when one group takes hostile action against the other group that has not noticed the first. And note that surprise can only happen when you are in the first round of the encounter, which means that, if you want to take advantage of surprise, Initiative cannot be rolled before the start of the ambush.
It also doesn't make sense to give someone the surprised status when nothing is surprising them (ie, the players are planning, not ambushing). When you are surprised, you cannot move. Does the act of the players planning somehow paralyze the other group? No, it shouldn't.