+ Start a Discussion
Yury BondarauYury Bondarau 

How may SOAP calls are required to import 1M records?


I have tried to import 2500 of records to my dev environment. It took 14 API calls.

I assume that we will need 1 call for login + 2500 records / 200 (size of chunk) = 14. It is okay. But what will heppen in case of loading of 1,000,000 records? What if insertion will throw an exception?

Is there any buffer taht I should consider in calculations of number of API calls? For instance  - refresh an authentication token in a middle of loading? Anything else?

Thanks in advance
Best Answer chosen by Yury Bondarau
Yury BondarauYury Bondarau
Hi Pramod,

Thank you for proposing Bulk API, but I do not plan to use async Bulk API.

Calculating of API calls is not a big deal: 1M / 200 = 5K API-calls. But practically this is WRONG answer. We need at least 1 API-call in addition for log in.

So the minimum number of API calls for 1M records is 5001 (in ideal world)

My question was addressed to somebody who have a real life experience in loading such amount of data, I need some statistic of what is the average percentage of unsuccessful API requests. For instance some requests can be loss due to connection timeout, some of them can be loss due to locks in the database etc. etc. So we will need to resend data and consume additional API calls.

All Answers

Aakaash NairAakaash Nair
Hi Yury,

This document provides all the information that you are looking for: https://developer.salesforce.com/page/Loading_Large_Data_Sets_with_the_Force.com_Bulk_API

Hope this helps!

Please mark this as a best answer if your query  is resolved so others also get help from this thread in future.

Thanks,
Aakaash
Pramod_SFDCPramod_SFDC
Hi,

Processing 1 million records will require 5000 API calls to Force.com. I hope this answers your query. However, as aakaash pointed out, the below mentioned link provide you more information on this.

>> https://developer.salesforce.com/page/Loading_Large_Data_Sets_with_the_Force.com_Bulk_API


Regards
Pramod
Yury BondarauYury Bondarau
Hi Pramod,

Thank you for proposing Bulk API, but I do not plan to use async Bulk API.

Calculating of API calls is not a big deal: 1M / 200 = 5K API-calls. But practically this is WRONG answer. We need at least 1 API-call in addition for log in.

So the minimum number of API calls for 1M records is 5001 (in ideal world)

My question was addressed to somebody who have a real life experience in loading such amount of data, I need some statistic of what is the average percentage of unsuccessful API requests. For instance some requests can be loss due to connection timeout, some of them can be loss due to locks in the database etc. etc. So we will need to resend data and consume additional API calls.
This was selected as the best answer