I have to load several million child records distributed over about 2000 parent records, so obviously I'm running into locking issues. I'm trying to find a way to avoid locks as much as possible, so I came up with this:
I split my child record file into one file for every parent record. then I use Dataloader CLI to start a separate serial bulk API job for each of these files. My idea was that each batch job would run independently from the others while the individual batches for each job would not interfere with eachother.
However, from small scale testing it looks like the jobs don't run concurrently. A batch from Job A runs alone and only when it is done does a batch from Job B start running, and so on.
Is this expected behavior or is it just an anomaly? If several serial jobs are running at the same time, will they still finish in the same time as one large serial job?
Best Answer
Whether they will run parallel or not is something that will be beyond your control. The general answer is that it is not likely for that to happen. The reason for that is because of how queued asynchronous jobs are processed in a pod. Rather than repeat that explanation here, I'm going to refer you to my answer to Dramatic increase in AsyncApexJob latency which explains how queued jobs are processed in a pod. The summary being that more jobs you have queued, the larger delays you can expect to occur between processing of your jobs.