function readOnly(count){ }
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
+ Start a Discussion
vinod ram 9vinod ram 9 

REST API Callout - How to handle large response data

Any idea on possible options to handle large data which is received in the form of response when REST API callout made to external system.
SwethaSwetha (Salesforce Developers) 
HI Vinod,

In Apex, no chance. The maximum response size is limited to 6/12/36MB (depending on context). You'd have to do client-side processing (e.g. Visualforce via the AJAX Proxy) in order to handle this amount of data.

As per https://salesforce.stackexchange.com/questions/255209/how-to-be-able-to-process-a-large-json-response 

There's just no way to cram that much data into Salesforce as-is.
I'd imagine you'd need to introduce a layer in the middle to break that JSON up into more manageable pieces, and also make use of async processing (probably using Queueable).

As I'm sure you know, the transaction limit on heap size is 6MB (12MB Async), and that space needs to hold the incoming response, store it again (plus overhead) when you deserialize it, and have space left over to do whatever processing you need to do.

Another option would be to have the middle layer (Heroku, some VM on some cloud provider, etc...) do the processing for you, and then create/update/delete whatever it is that you need to do via accessing Salesforce's APIs

Also see https://salesforce.stackexchange.com/questions/334079/sending-large-json-data-in-response-body-apex

If this information helps, please mark the answer as best. Thank you