+ Start a Discussion
sunilkbansalsunilkbansal 

Process more than 10000 rows in VF/Apex

Hello Friends,

 

I know that there is a limit of 10000 rows for SOQL query.

But I have more than 10000 rows to process, how can I do that in pure VF/Apex approach.

 

Thanks,

Sunil

Srinivas_V2Srinivas_V2
you can use @future method in your classes
sunilkbansalsunilkbansal

Thanks Srinivas for reply.

Are you sure below method call will not fail if I have more than 10000 rows in Account. I belive it will fail.

 

 

@future public void allRecord(){ for(Account[] acctList: [Select Id from Account]){ //code } } public void testAllRecord(){ allRecord(); }


 

Srinivas_V2Srinivas_V2
yes might be. have u tried to use javascript. if you are using a vf page then it can be done
sunilkbansalsunilkbansal

I dont want to use javascript, Ajax API, same as in S-Control. I am looking for pure VF/Apex solution to process more than 10000 records. I hope you guys understand.

Any help is highly appreciated.

mattdarnoldmattdarnold

You should be able to use the @future annotation to process over 10,000 records; depending on what exactly you'll be processing. If you want to loop through this group, select some based a certain criteria, and then make insert of update DML calls, you should be ok. If you want to grab all 10,000 of those records and make DML operations off each one, I think you'll hit the 10,000 limit on total number of records processed by DML statements.

 

This code works to loop through over 10,000 records and to make some arbitrary operation. From the sample you posted it wasn't clear if you wanted to do DML operations or not.

 

public class scaleTest {

@future (callout=false)
public static void runTest() {
Integer count = 0;
try {
for(Integer i=0; i<10002; i++){
count = i;
}
} catch (Exception e) {
System.debug('General Exception: ' + e.getMessage());
}
}

static testMethod void unitTestRunTest () {
runTest();
}

}

 

However, when I attempted to run the same test with DML operations instead, I hit the 500 record limit for number of DML records processed as part of a test method in the code below. So, I assumed you'd be hitting the higher 10,000 record limit when attempting to run the method.

 

public class scaleTest {

@future (callout=false)
public static void runTest() {
List<Account> acctList = new List<Account>();
for(Integer i=0; i<10002; i++){
Account a = new Account (name='Test account', description=i.format());
acctList.add(a);

// Need to account for and handle size of acctList
if(acctList.size()==999){
try {
insert acctList;
acctList.clear();
System.debug('List size: ' + acctList.size());
} catch (DmlException d) {
System.debug('DML Exception:' + d.getMessage());
} catch (Exception e) {
System.debug('General Exception: ' + e.getMessage());
}
}
}

}

static testMethod void unitTestRunTest () {
runTest();
}

}

 

Let me know if this helps.

 

-- Matt

 

 

sunilkbansalsunilkbansal

I understand that I can manipulate more than 10000 records for DML operation by  inserting/updating a list of 1000 records.

But my question is about SQOL query returning more than 10000 rows. I hope below code helps in understaning the requirement.

@future
public void allRecord(){
List<Account> acctList = new List<Account>();
// suppose below query from Account returns more than 10000 rows even if we put some criteria, and I cannot leave any row without processing.
for(Account[] acctArray: [Select Id from Account])
{
for(Account acc: acctArray)
{
acctList.add(acc);
if(acctList.size() == 1000)
{
// do some DML on acctList
acctList.clear();
}
}
}
}

public void testAllRecord(){
allRecord();
}

 


 

Message Edited by sunilkbansal on 04-06-2009 10:46 PM
junaid_ajjunaid_aj

Can you please tell me how to use @future method

 

I am new to Apex

 

I have written a method which is calling a @future method

 

and that method i am calling from Page.

 

I am getting this error

@future call currently not allowed

 

public  void getDataAll(){

getData();

}

@future

 public static void getData(){

 

//calling DB to fetch more than 10000 records.

}

 

 

in page

 <apex:repeat value="{!DataAll}" var="ef">

</apex:repeat>

Anyhelp would be great.

 

Regards,

Mohammed Junaid

mattdarnoldmattdarnold
Hi,

You can't use an @future method as part of a VF controller. If you're trying to display a large number of records in a VF page you might try some kind of pagination, but displaying over 10k records to the user would likely be overwhelming.

@sunilkbansal, although you'd be able to manipulate larger numbers of records by chunking the operations into batches of 1000 to fit within a list, I think you're going to run into an overall cap on the total number of records processed as the result of DML statements - which is 10k. At least that's what I was experiencing in my tests.

-- Matt
sunilkbansalsunilkbansal

Yes Matt, you are right we cannot process 10000 as a result of DML statements.

But I am looking for ANY work around to select and process more than 10000 rows as part of an SOQL.

 

Just to manipulate the answer, out of those more than 10000 rows (lets say out of 15000 rows) I may process only 9000 rows for DML operation. But first I have to select those 15000 rows to find out which 9000 rows qualify for DML operation.

 

So the real question is how to select more than 10000 rows from an object (for any reason I have more than 10000 rows saved in a object) using pure Apex/VF approach. I know it is not possible in simple way, we need to do some work arounds. I am looking for suggestions on different workarounds.

 

Thanks,

Sunil

UmapenUmapen

Hello,

I am doing something similar. hitting too many SOQL query errors. 

Please look at my sample code

http://community.salesforce.com/sforce/board/message?board.id=Visualforce&thread.id=12104

I cannot process more than 100. How an I optimize my code to process more records?

 

How can I utilize @future call to batch my queries?

 

Thanks

Uma

 

sunilkbansalsunilkbansal

Can someone from salesforce Product Team can reply on this?

I am waiting for this since long.

 

Thanks,

Sunil

Suzan2009Suzan2009

I got same issue.

I need query out about 100k records. But the 10k limit can not let it work.

 

We need Salesforce.com Techniqual Team Support to build up our confidence. Please!!!!!!!!!!!!!   

nagalakshminagalakshmi

Hi,

 

I am also facing the same issue. How can i solve this. I have more than 10000 records in my list. Now i want to insert these values. But i am getting the 'Too many dml 10001 rows ' Please any one help me out. Its very urgetn for me.Thanks in advance.

 

Thanks,

Lakshmi

 

 

haripopuri1haripopuri1

Isn't this a Batch process?

Over 10000 records on VF page? Is it working, without ViewState error?

 

If it is a Apex Batch job, please refer this.

 

http://www.salesforce.com/us/developer/docs/apexcode/Content/apex_pages_standardsetcontroller.htm

 

It says it can hold upto 50 million records.

 

Thanks

Hari

 

cmd loadercmd loader

Hi Hari,

 

Actually the values which are in my list  are coming from visual force page dynamically. How can i pass this list values in to batch apex class. How can i use database.executebatch method. 

 

Actully i have wrote my batch apex class like this

 

global class batchinsert implements Database.Batchable<sObject>
{
global final List<storeproduct_category__C> fieldValues;
global final String Query;
global Database.QueryLocator start(Database.BatchableContext BC)
{
return Database.getQueryLocator(Query);

}
global void execute(Database.BatchableContext BC, List<storeproduct_category__C> scope)
{
/*List <storeproduct_category__c> lstAccount = new list<storeproduct_category__C>();
for(storeproduct_category__C s : scope)
{
storeproduct_category__C s1=new storeproduct_category__C();
s1.store__C=s.store__C;
s1.product_category__C=s.product_category__C;
lstAccount.add(s1);
insert lstAccount;
}*/

insert scope;


}
global void finish(Database.BatchableContext BC)
{
}

}

 

And i have called this batch apex class in my normal apex class as

 

Database.BatchableContext BC;
batchinsert batch=new batchinsert();
//Database.executeBatch(batch);
batch.execute(bc,updatestorebrand);

 

updatestorebrand is the list name which are having the values more than 10000. I tried a lot for solving this issue. But i am getting. Please help me out. Thanks in advance.

 

Thanks,

Lakshmi

haripopuri1haripopuri1

Hi,

 

Are you passing 10000 values from VF page to Apex Class dynamically? How did you get 10000 values on page? Who enters that much data?

 

Thanks

 

nagalakshminagalakshmi

Hi Hari,

 

I am having 100 values in my vf page. I need to insert these 100 child records in to every parent record with parent id. If i am having 102 parents. I am having the values in my list as 10200(100*102). How can i solve this issue. Please help me out.

 

Thanks,

Lakshmi

haripopuri1haripopuri1

Hi,

 

This looks like a multiselect picklist to me (if the 100 values from VF page are primitive type). If the 100 values are child records, you can store the IDs in a comma separated string in 1000 records.

 

Thanks

Hari