[SalesForce] Governor limits when inserting/updating records using Dataloader

Yes, each batch has its own limits and each batch consumes an API call, they are unrelated to one another.

Reduce the batch size if you hit governor limits, this will of course increase the time it takes to process them.

You might find this useful for understanding the impact of that: http://limitexception.herod.net/2011/12/15/talend-vs-apex-dataloader-bulk-uploaddownload-benchmarks/

So I think the problem is that the bulk data API is running multiple asynchronous batches and thus kicking off too many Batchable Contexts.

It looks like your batch process doesn't actually care what Id's you pass in and instead just processes everything?

If this is the case and you don't need real time (or close to realtime) updates, it might be a better idea to just schedule this to run.

Otherwise, if you need it to happen in near real time, you could use batch "chaining" in order the have the batch keep running as long as there are records to process.

The basic steps would look something like this:

1. In the finish method, add a check to see if there are more records to process. If so kick off another batch.
2. In your trigger, have it first check if the batch is already running. If so, you don't need to do anything. Otherwise, have it kick off the batch.

Update

If you have access to the JobInfo concurrencyMode param, you can set this to "Serial" and it will prevent multiple batches from running in parallel. I think this might solve you problem. If that alone doesn't work, try also reducing the batch size. Obviously, this will cause the import to take much (much) longer.

On the dataloader this Serial can be turned on by checking "Enable Serial Mode for Bulk Api".