+ Start a Discussion

What is the least invasive and most elegant way to get ALL of the records from a table/object into a third party database?

The product I'm developing is going to act like acorporate backup solution at first but there are some data processing features that will be developed later that are the real focus.

Let's say I wanted to get a full copy of every single record in, say, an Account table (followed by every other table in the instance later on). I've dealt with clients that easily have over 1.5 million records in their Account history. What is the preferred way to do this without causing governor or other limit problems with as few api calls as necessary? Speed won't be a blg deal at first, I just want to do right for the customer's instance.

This could be done either way: externally from API requests from a python app on an external server or from a script written in Apex, but the end result is the same-- every single record in my SQL database ready to be queried in bulk with a happy customer whose instance hasn't been inconvenienced.

I believe I might have a few options. Batch apex might be one of them, but then there's also the ability to create batch export jobs from the API that I believe will export a full csv that you just have to build a parcer for. There might be other directions I'm not aware of.

Thank you very much for your thoughts!
AbhishekAbhishek (Salesforce Developers) 

Lightning Data and Data.com connect you with firmographic, industry, and region-specific data from third-party providers. You license Lightning Data packages directly from providers, and you use a convenient unified interface to integrate data from all third-party sources into Salesforce.

For further reference,


An external data source specifies how to access an external system. Salesforce Connect uses external data sources to access data that's stored outside your Salesforce organization. Files Connect uses external data sources to access third-party content systems. External data sources have associated external objects, which your users and the Lightning platform use to interact with the external data and content.


I hope you find the above information is helpful. If it does, please mark as Best Answer to help others too.