[SalesForce] Loading Massive data into Data Extension using API

I want to load about a million records into a Data Extension from an external system. Wondering what would be the most efficient way to achieve that?

If I use SOAP/ REST API, how many records should I pass to the API call every time?

Any other suggestions?

Best Answer

Your best bet, due to the large audience is to export the records into a csv and then compress it and send it to the FTP associated with your SFMC account. You can then set up a program in automation studio that is triggered based on this file drop that decompresses it and imports it into your account. From everything I have read, this is the best/easiest way to handle massive bulk imports.

If you are determined to use API, you will likely want to do a bunch of asynchronous upserts using the REST API. I believe the limit is at around 10k per call though.


Similar questions/answers:

ExactTarget: Common data loading strategies?

Adding Bulk data to Data Extension Object in Exact target

How to bulk (import /export) to (data extensions / lists) with (soap / api / sdk)?

Related Topic