function readOnly(count){ }
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
+ Start a Discussion
SScholtzSScholtz 

Heap Size Limits: Go over for a short period of time, so long as you come back down?

Hi there,

 

Question: Does Salesforce allow you to go over the heap size limit, so long as you don't stay over it for too long?  That seems to be our experience, but I'm not sure if that's "official" or if we're just getting lucky.

 

Background: We've been working on a Visual Force page which accepts a CSV from the user in order to parse it, run business logic and do some inserts.

 

Salesforce started complaining about Heap Limits after working just fine for a few weeks.  For one, the files had gotten larger, but as it turned out we were going over the Heap Limit even with the smaller files, Salesforce just wasn't alerting us.

 

In the code, we have a loop that goes through the Blog file (after converting it to a string) to do a row count.  Looking at how much heap we were using, we blew out the limits well before we got to the loop and had been doing so for a while, but Salesforce didn't start complaining until recently, and it only cacked out once we got part way into the loop.

 

We found another way to do the row counting that didn't require a loop (the new string.countMatches() function) and Salesforce stopped complaining about heap size, even though during some tests, we could go way way over. (nearly 4 times as much as the 6 mb's we're allowed)

 

The key was that we were always doing clean up, nulling out large variables and objects that were no longer needed very quickly after we were done with them.

 

It looks like whatever Salesforce does to watch for limit breaking either has a lot more give and take, or it only checks for limit breaking at certain junctures.  Anybody else have a similar experience?

bob_buzzardbob_buzzard

We've experienced exactly the same thing.  I've never been able to determine if its certain events that cause the check, or if its something that happens as some kind of background task.  I haven't seen 4 x a limit, but I guess if there's a window of opportunity it may be as much as you can get in that time!

 

We've also seen this on storage - we can go well over 100% for a short period of time, as long as we clear it down afterwards.

sfdcfoxsfdcfox

It seems to me that the heap governor limit is implemented in the garbage collection algorithm, since you can certainly allocate memory that would exceed the heap limits. The GC is probably run using some algorithm that is tuned to balance performance with responsiveness, probably in the neighborhood of every 5-10 lines of code executed. Also, heap size might only be estimated, from what I've observed (e.g. the allocation system "charges" a certain amount of heap per type of object, and it is up to the GC to determine the total heap used). Also, some type of heap, such as object references, might not count towards the total limit. I suspect they also use a string pool, as in Java, so identical strings might initially allocate more memory than they're actually using on the heap. It's also entirely possible that temporary variables report their size to the heap but are not counted by the GC for determining the limit (such as using for(string s:myLongString.split('\\n'))). Unfortunately, it is all speculation, as none of this information appears to be disclosed for general consumption.

 

EDIT: Actually, Limits.getHeapSize() actually states that it returns an "approximate" value...

 

EDIT: A little more insight:

 

for(string s:'1234567890'.repeat(15000).split('0')) {
  System.debug(Limits.getHeapSize());   
}

 

00:18:12.252 (26252718000)|USER_DEBUG|[2]|DEBUG|196057

 

00:18:12.290|LIMIT_USAGE_FOR_NS|(default)|
  Number of SOQL queries: 0 out of 100
  Number of query rows: 0 out of 50000
  Number of SOSL queries: 0 out of 20
  Number of DML statements: 0 out of 150
  Number of DML rows: 0 out of 10000
  Number of code statements: 15000 out of 200000
  Maximum heap size: 0 out of 6000000
  Number of callouts: 0 out of 10
  Number of Email Invocations: 0 out of 10
  Number of fields describes: 0 out of 100
  Number of record type describes: 0 out of 100
  Number of child relationships describes: 0 out of 100
  Number of picklist describes: 0 out of 100
  Number of future calls: 0 out of 10

Total usage was 0 bytes of heap, despite getHeapSize() returning an estimated size of almost 200kb. And it even counted 15,000 script statements, which seems to support a breakpoint theory, not just a normal "check every once in a while" type check.

sfdcfoxsfdcfox

Addendum:

 

I think the breakpoint theory is gone, too. I was able to run about 30 lines of code without ever once displaying any maximum heap at all, which suggests there's some other mechanism in play (maybe a minimum code size limit or something?). That, or Execute Anonymous just isn't playing fair at all. Regardless, it seems that the general idea is that you have a window of opportunity to violate that particular rule without actually getting caught. Interesting...