+ Start a Discussion
nkrishnaiah1.3902275446551082E12nkrishnaiah1.3902275446551082E12 

How to send large volume of data to external web service using callouts?

Hi, we have a requirement to integrate Salesforce with a ThirdParty Vendor using webservices.

We need to send around 300,000 Records from Salesforce to the ThirdParty vendor using their webservice.
We have a custom apex program using which we are able to call the third party web service successfully within a future method in a schedulable program. However, for a volume as high as 300k records, I am not sure how to avoid hitting the governer limits. My understanding is that, I can make 10 calls in an apex class and can call the future method 10 times, making 100 web service calls in total. This is not sufficient for 300k records. Supposing we can include multiple records in one call, we will still hit another problem. Since each call's size is restricted to 3 MB and our one SOAP call's xml size is around 20k, the maximum we can do is 150 records per call giving us the ability to send 150 X 100 = 15k records. Is there any method to avoid hitting governer limits in this scenario.

Please help me as I am new to Sales force development.
Best Answer chosen by nkrishnaiah1.3902275446551082E12
sfdcfoxsfdcfox
You'll need to create a batch apex class to send these records. Batch apex code allows you to split the load into chunks, where each chunk has its own limits. In short, you'll be looking for something like:

<pre>
global class SendData implements Database.batchable<SObject>, database.allowscallouts {
    global Database.QueryLocator start(Database.batchableContext bc) {
        return Database.getQueryLocator('select id, name from contact');
    }
    global void execute(Database.BatchableContext bc, Contact[] records) {
        // Perform your callouts here
    }
    global void finish(Database.BatchableContext bc) {
        // Notify of completion, etc.
    }
}
</pre>

Also, just to make sure you'll be fine, you can set the "scope" to 10 per iteration, using:

<pre>
Database.executeBatch(new SendData(), 10);
</pre>

All Answers

sfdcfoxsfdcfox
You'll need to create a batch apex class to send these records. Batch apex code allows you to split the load into chunks, where each chunk has its own limits. In short, you'll be looking for something like:

<pre>
global class SendData implements Database.batchable<SObject>, database.allowscallouts {
    global Database.QueryLocator start(Database.batchableContext bc) {
        return Database.getQueryLocator('select id, name from contact');
    }
    global void execute(Database.BatchableContext bc, Contact[] records) {
        // Perform your callouts here
    }
    global void finish(Database.BatchableContext bc) {
        // Notify of completion, etc.
    }
}
</pre>

Also, just to make sure you'll be fine, you can set the "scope" to 10 per iteration, using:

<pre>
Database.executeBatch(new SendData(), 10);
</pre>
This was selected as the best answer
krish99krish99
@sfdcfox

   can you provide the sample callout code how to call third party. which you mention in the comment. i am also looking for same.
nkrishnaiah1.3902275446551082E12nkrishnaiah1.3902275446551082E12
@sfdcfox
Thanks for your response.

I have been able to send bulk data to a third party using the logic you mentioned.


Is there a way to run these batches in parallel or implement Multi-Threading in someway

SFDC_BigDogSFDC_BigDog

@nkrishnaiah1.3902275446551082E12

If I am not wrong, the meaning of the Batch class is to run in parallel but with limited amount of data in each transaction that we define in our class. 

Correct me if I am wrong

Thank you,