+ Start a Discussion
Jesse MilburnJesse Milburn 

Bulk API -- Your query request was running for too long

Hi, we are having an issue with one of our clients using our integration package. We are at our wits end so I am reaching out here to see if anyone has run into a similiar scenario.

When running a simple query for example:

Select Id From CustomObject

We do this on a regular basis to delete and refresh all of the objects as we currently do not have a unique field on the records we are uploading. The query only is getting around 1 million records max which seems in not a terribly large amount. We only have this issue with one of our clients and only on their production org. They are one of our larger clients, but we have a few in their ball park that run fine.

I was curious if anyone has seen this behavior before? And if so how was the issue resolved. I read some posts that the recycle bin needs deleted, and our client said they had done it prior to an integration run.

I have also read that our query needs to be more selective and add a where clause. But in this case we would technically 'tricking' the SOQL to still retrieve the entire record set. For example:

Select Id From CustomObject where isDeleted = false
Best Answer chosen by Jesse Milburn
Himanshu ParasharHimanshu Parashar
Hi Jesse,

I ran through this problem some times back for one of our client where we were deleting all records prior to insert new records, after some time sfdc was taking much time to execute a simple query for that object. Recycle bin was creating the issue as you mentioned as well, we hard deleted using emptyrecyclebin function but didn't solve our issue. we talked to SFDC support regarding this and they came up one statement that, SFDC hold the data fro 45 days before permanently delete from their side even you perform hard delete. So at the end we reached on following solutions.

1. Open a case with SFDC support so that they can perform permanently delete records where isDeleted=True. (They can only do this after opening a case. They say they will not always do that only 2-3 times)
2. Create a index on your object
3. Modify your business logic so that instead of deleting you will upsert the data (SFDC support proposed that though we were not agree with that.)
4. More selective query.

#1 should solve your issue for now and after that you need to work on #2, #3, #4


Thanks,
Himanshu
Salesforce Certified Developer | Administrator | Service Cloud Consultant

P.S. If my answer helps you to solve your problem please mark it as best answer. It will help other to find best answer.


 

All Answers

Himanshu ParasharHimanshu Parashar
Hi Jesse,

I ran through this problem some times back for one of our client where we were deleting all records prior to insert new records, after some time sfdc was taking much time to execute a simple query for that object. Recycle bin was creating the issue as you mentioned as well, we hard deleted using emptyrecyclebin function but didn't solve our issue. we talked to SFDC support regarding this and they came up one statement that, SFDC hold the data fro 45 days before permanently delete from their side even you perform hard delete. So at the end we reached on following solutions.

1. Open a case with SFDC support so that they can perform permanently delete records where isDeleted=True. (They can only do this after opening a case. They say they will not always do that only 2-3 times)
2. Create a index on your object
3. Modify your business logic so that instead of deleting you will upsert the data (SFDC support proposed that though we were not agree with that.)
4. More selective query.

#1 should solve your issue for now and after that you need to work on #2, #3, #4


Thanks,
Himanshu
Salesforce Certified Developer | Administrator | Service Cloud Consultant

P.S. If my answer helps you to solve your problem please mark it as best answer. It will help other to find best answer.


 
This was selected as the best answer
Jesse MilburnJesse Milburn
Thanks Himanshu,

After much searching we have pretty much done everything you suggested. For a longer term solution we are making architecture changes so that we do not delete and refresh records on a daily basis.

Thanks for the response,
Jesse