• NewSFUser
  • NEWBIE
  • 0 Points
  • Member since 2013

  • Chatter
    Feed
  • 0
    Best Answers
  • 0
    Likes Received
  • 0
    Likes Given
  • 0
    Questions
  • 1
    Replies

I am developing a Java application which will query tables which may hold over 1,000,000 records.  I have tried everything I could to be as efficient as possible but I am only able to achieve on avg. about 5,000 records a minute and a maximum of 10,000 at one point.  I have tried reverse engineering the data loader and my code seems to be very similar but still no luck.

 

Is threading a viable solution here? I have tried this but with very minimal results.  

 

I have been reading and have applied every thing possible it seems (compressing requests/responses, threads etc.) but I cannot achieve data loader like speeds.  

 

To note, it seems that the queryMore method seems to be the bottle neck.

 

Does anyone have any code samples or experiences they can share to steer me in the right direction?

 

Thanks 

  • February 10, 2010
  • Like
  • 0