+ Start a Discussion
Vinay JVinay J 

Processing Large data volume in Controller

Hi Friends,

I have a requirement, where :-

1. Display some options in SelectOptions
2. Fetch historical data (for 5 years - Around 200k record).
3. Do calculations for each month during last 5 years, based on the options selected in step 1. Display the result finally.

The problem is, while fetching the historical data, there's no way I can use where clause in my query (apart from 5 year time frame). The selectoptions have such values due to which, I CAN'T have where clause in query.

I've built entire logic and have tested it for previous 2 years data. But when I change my where clasue to fetch 5 years data, I get error "heap size too large salesforce" with a 8 digit number!!

I've put the query to fetch historical data in constructor right now. I've also tried readonly mode but the same error pops up. Earlier I was getting view state error which I solved using transient wherever possible.

Please suggest me an alternative. It's ok if after selecting options and pressing the button, user have to wait for couple of minutes to see the results.
Andy on CloudAndy on Cloud
Actually, if you think about data growth, a more permanent solution is like what Google always does: pre-calculations.  First of all, it's historical data, it doesn't change. Second of all, I assumed, options select from 1 are not changing all the time in real-time either.  So if there could be a weekly job or monthly that pre-calculates or even perform records labeling to make query more selective when users select option 1, then performance will be at least close to real time.  I hope that answer your question, if it doesn't, perhaps I need more details on calculations that you do on option 1 selection or what kind of data you are storing.