+ Start a Discussion
Arjun y 7Arjun y 7 

How to hit multiple times to external server from salesforce

Hi All,

Requirement is we need to do update operation in salesforce from SAP. The job should runs every day.

In SAP, we have 30000 records. When we hit the SAP server, we have to receive entire 30000 records in one call using REST Api. I tried this and am getting this error "System.StringException: String length exceeds maximum: 6000000" Can i know is there any other approach to work on?
Rajiv Penagonda 12Rajiv Penagonda 12
What is your middleware? 30000 recs is a big number. It would be bad design to send all records at once. Multiple calls from SAP to Salesforce is the only good way out. Generally your middleware will have a batch logic that can, lets say for arguments sake, send 1000 records 30 times.
Arjun y 7Arjun y 7
Hi rajiv,

Thanks for the reply. In our requirement, from salesforce we need to hit the SAP server to fetch 30000 records. For this, they are giving pagination url to us. So, we need to hit the 30 times to that server to pull the data.Currently, i am working on it.
Arjun y 7Arjun y 7
Hi Rajiv,

I have tried with the rest approach. I am recieving heap size limit.

Please let me know if there any other approach to hit 30 times to the external server to pull the data?

Thanks
Arjun y 7Arjun y 7
Hi Rajiv,

We are not using any middleware for this integration. From salesforce, we need to hit the SAP server multiple times to fetch the data and need to do record updates in salesforce. For this, SAP gave odata pagination URL which gives 1000 records for one transaction.

Thanks
Rajiv Penagonda 12Rajiv Penagonda 12
Arjun, at the moment, the information you've provided is insufficient to visualize your implementation. The way I see it, because you need to run this job daily, you would need a scheculed class. This class when run would make multiple webservice calls within a loop. Because of Salesforce limitation, you will not be able to make 30 calls. 10 is the callout limit per transaction. What is the variance in record count? Is 30000 records worst case scenario?
Rajiv Penagonda 12Rajiv Penagonda 12
I see. I think your case can work if you implement Queueable with Schedulable. See sample code below:
 
global class SAPToSFDataLoadScheduler implements Schedulable {
	public static final String SCHEDULE_NAME = 'SAP To SF Data Load Schedule';

	global void execute(SchedulableContext sc) {
		ID jobID = System.enqueueJob(new SAPToSFDataLoader());
	}

   	global static void scheduleSelf() {
   		CronTrigger [] ct = [SELECT id FROM CronTrigger WHERE CronJobDetail.Name=:SCHEDULE_NAME];
   		
   		if(ct.size() == 1) {
   			try {
				System.abortJob(ct[0].id);
   			}
   			catch(Exception e) {}
   		}
   		
		/*
		 * Scheduled to run once every day at 12.01 AM
		 */
		System.schedule(SCHEDULE_NAME, '00 01 00 * * ?', new SAPToSFDataLoadScheduler());
   	}	
}
 
global without sharing class SAPToSFDataLoader implements Queueable {
	private Integer mPage = -1;

	public SAPToSFDataLoader() {

	}

	public SAPToSFDataLoader(Integer argPage) {
		this.mPage = argPage;
	}

	public void execute(QueueableContext context) {
		Boolean lHasNextPage = false;
		Integer lNextPage = -1;
		List<SObject> lRecordsFromSAP = new List<SObject>();

		/*
		 * This means it is running for the first time.
		 */
		if(mPage == -1) {
			// Do the first callout to SAP here and assign records to lRecordsFromSAP
		}
		else {
			// fetch the page at mPage from SAP and assign to lRecordsFromSAP
		}

		// Process the data available in lRecordsFromSAP
		// check if the next page to be processed is available and set lHasNextPage and lNextPage accordingly

		if(lHasNextPage) {
			ID jobID = System.enqueueJob(new SAPToSFDataLoader(lNextPage));
		}		
	}
}

Once you have filled the gaps in the class SAPToSFDataLoader with your application logic, just invoke
 
SAPToSFDataLoadScheduler.scheduleSelf();

from your Developer Console. The scueduler will be scheduled to run at 12.01 AM every day.

Here is what the above code does:
1. The schedule "SAPToSFDataLoadScheduler" is invoked at 12.01 AM
2. It queues "SAPToSFDataLoader" to run in default mode (no page index specified)
3. The "SAPToSFDataLoader" runs when system resources are available and depending on its mode (argPage), makes a callout to SAP
4. "SAPToSFDataLoader", fetches, processes and determines the next page for processing. If next page is available, it Queues a new instance of itself to process the next page from SAP.

This solution is scalable, in case if you have records that exceed 30000.

Hope this helps

Here are some useful links on Queueable
https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_queueing_jobs.htm
https://developer.salesforce.com/blogs/developer-relations/2015/05/queueable-apex-future.html
https://developer.salesforce.com/blogs/engineering/2014/10/new-apex-queueable-interface.html
https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_class_System_Queueable.htm

 
Arjun y 7Arjun y 7
Hi Rajiv,

Thank You for the above technique. I have one query regarding this.

 if(mPage == -1) {
            // Do the first callout to SAP here and assign records to lRecordsFromSAP
  }

If i made first call here, will it be pull all 30000 records at once?
Rajiv Penagonda 12Rajiv Penagonda 12
It depends on how your SAP odata pagination URL is designed. If it follows a pagination logic it will return you the 30000 records in chunks (depending on your SAP page size). You need to check this internally from your provider.
Arjun y 7Arjun y 7
Yes rajiv. They are sending via pagination technique only. Thank you.
 
Rajiv Penagonda 12Rajiv Penagonda 12
Arjun, did this work? Did you take a different approach? Do share your solution.
Arjun y 7Arjun y 7
Hi Rajiv,

To achieve this, i went for batch class methodology and it is working for me now.

Thanks.

 
Rajiv Penagonda 12Rajiv Penagonda 12
Alright, good :)
Arjun y 7Arjun y 7
Hi Rajiv,

I used below technique.It is working for me.

public with sharing class CalloutBatchApex implements Database.Batchable<Integer>, Database.AllowCallouts {
    public Iterable<String> start(Database.BatchableContext BC) {
        return new List<Integer> { 1, 2, 3 };
    }
 
    public void execute(Database.BatchableContext info, List<integer> iteration) {
        // Make the callout to the web service and import the records to Salesforce for the iteration.
    }
 
    public void finish(Database.BatchableContext info) {}
}