You've got it. Just make sure that in the light fixtures you use all black in one connector, all white in the second and all bare wire in the third (from your hand-drawn wiring, it looks like that's how you plan to do it). That way, the lights are wired in parallel so that they both get the full mains voltage across them when you turn on the switch.
Also make sure that the switch is connected to the live (black in this case) wires in the two cables, which means that no electricity can reach the light fixture when the switch is off (which could save your life if you accidentally touch the live terminal in the fixture when changing a bulb).
Since you didn't provide a picture, or a very helpful description of what you're looking at. I'll try answering your question by explaining how the switch itself works, which will hopefully help you understand the problem better.
Single Pole Single Throw (SPST) Pull Chain Switch
The pull chain switch that controls the light(s), is a single pole single throw (SPST) switch. It has two positions ON (Closed), and OFF (Open). Drawn simply, it would look something like this.
Switch shown in ON (Closed) position.
When the switch is in the ON (Closed) position, current is allowed to flow through the switch, through the light(s), and back to the the source (via neutral).
When the chain is pulled and released, the internal contact rotates 90° (1/4 turn) into the OFF (Open) position.
When the switch is in this position, current is not allowed to flow through the switch, and the light is not lit.
This is why the pull chain switch that controls the light(s) only has two leads.
Single Pole Multiple Throw (SPnT) Pull Chain Switch
The pull chain switch that controls the fan, is a single pole multiple throw switch. It has multiple positions, which allows it to control the speed of the fan. Draw simply, it would look something like this.
Switch shown in OFF (Open) position.
When the chain is pulled and released on this switch, the internal contact rotates 45° (1/8 turn) to the next position.
Another pull, another turn.
Pull again, turn some more.
One final pull brings the switch 180° around, and again to the OFF (Open) position.
By manipulating the output of this switch, the fan is able to whirl around at various speeds depending on the switches position. The number of output leads, will depend on the switch. How those leads are connected to the fan motor, will depend on the fan manufacturer. This simply illustrates the basic principle of how the switch works.
As always electrical work can be dangerous, never be afraid to contact a qualified Electrician
Best Answer
So your existing fixture (assuming standard US wiring, you'll have to verify) has a hot (black), neutral (white), and ground (bare) and has constant power. The additional lights will need 4 wires - constant hot to travel to the switch (black), a neutral (white) and a switched power that will come back from the switch (red).
The existing wire (assuming 14G wire for lighting circuits) is sold as "14/2" cable. The ground isn't counted in the number scheme, so it's considered to have 2 conductors. What you will need for the rest of the run is "14/3". This new cable (black, white, red) will run from the existing light to each of the new lights and then to the switch box.
At the existing light, black connects to black and white to white, but the red will just be capped off and unused. Red will be power coming from the switch. In the future if you wanted the pull chain light to be controlled by the switch, you could hook that light's black wire to the new red.
At the new lights, white will connect to white and the black on the fixture will connect to the "switched" red wire. The black wire of the 14/3 will not connect to the new light and will just "pass through" the box. In each box, all ground wires are connected to each other and the bare or green wire from the fixture.
At the switch, the white wire will be capped (unless you use a smart switch that needs the neutral) and the black and red wires will connect to the two screw terminals on the light switch.
Older wiring practices would allow you to do all of this without the extra red wire, but now all switch boxes require a neutral so you must use 14/3 to do it properly and allow for future flexibility. You could do this now with a mix of 14/2 and 14/3, but if you just use 14/3 you don't have to buy 2 rolls of different cable and again, it gives you flexibility later.
Diagram added by request:
The "dotted" red wire is optional and if you never want the pull chain light to be on a switch it will serve no purpose. The white wire going to the switch is not used unless you have a smart switch but it is required by code.