function readOnly(count){ }
Sign Up ›
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
Learn More

Browse by Topic
ShowAll Questionssorted byDate Posted
+ Start a Discussion
JBerryJBerry 

Dataloader command line = slow loading?

Hi,
I was wondering if this is typical or not.  When I use the command line to update data on salesforce, it runs extremely slow.

I am using the "process ../conf csvupdate" command line, with my process-conf.xml file located in my /conf directory. 

The process starts off quick, but when it gets to "Setting object reference types", this step alone takes 10 minutes on average.  When I run the same update in the GUI with the same field mappings file, it is instant with a speed of "27000 records per minute".

Any ideas? I'm not getting any errors or anything, its all successful, and nothing in my error logs.
Rick.BanisterRick.Banister
You didn't say what object you were loading, but in general, it could be because you're running into scalability limitations that could be memory related. View the Windows Task Manager performance monitor tab, and see what your virtual memory is like. If you exceed your physical memory, any Java application such as Data Loader will crawl. The SOAP buffers take a lot of memory. With our product, Relational Junction, we recommend a VM size of 1 GB for a 200 record buffer. If that's what yours is taking, buy more RAM or cut the buffersize down. If you have the required RAM, Relational Junction gives perfectly flat performance of typically 20 to 40 records per second.
JBerryJBerry
I am doing an Update from csv file into my Opportunities. It is typically 40 records updated, so it should not be this slow.  I have 1gb of RAM, and it does not seem to be taking up much of anything.

I changed my process batch size down to 5, and only imported 5 records.  It still took 10 minutes to process at the same spot.

Am I looking in the wrong place to set the buffer? I was under the impression that is the process batch size variable inside of my process-conf.xml file
mspohnmspohn
Hi,

Could you please send me your process-conf.xml file, so I can take a look?
Please make sure it doesn't include any confidential information before you send it.

Thanks,
Markus

Markus Spohn
Director Product Management, Integration
Salesforce.com