[SalesForce] Problem loading million records in Salesforce using Bulk API

Hi I am loading some millions of data in Salesforce using Bulk API. Is there any limitation on that? The external system has failed to insert the records. Records is around 1 million.

Is there any setting I have to let external system know when they use the Bulk APi? Can any external system insert millions of record in salesforce. I have a full copy and enough space to store the data.

Best Answer

Based on the volume of the data, external system will decide how they will proceed inserting data into SFDC.

Bulk query limitations:

Batch Limit:

You can submit up to 10,000 batches per rolling 24-hour period. You can’t create batches associated with a job that is more than 24 hours old.

So, within 24 hours you can load 10K * 10K = 100,000K records

Batch Size:

  • Batches for data loads can consist of a single CSV, XML, or JSON file that is no larger than 10 MB.
  • A batch can contain a maximum of 10,000 records.
  • A batch can contain a maximum of 10,000,000 characters for all the data in a batch.
  • A field can contain a maximum of 32,000 characters.
  • A record can contain a maximum of 5,000 fields
  • A record can contain a maximum of 400,000 characters for all its fields.
  • A batch must contain some content or an error occurs.

Refer more on Bulk API Limits

Refer this example Use Bulk Query on how to insert data from external system using BulkConnection object of Partner API.