function readOnly(count){ }
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
+ Start a Discussion
GoForceGoGoForceGo 

Apex Data Loader - setting dataAccess.name to include dates.

I want to use Apex Data Loader for integration through CSV files (ERP to SFDC).

There would be a one time initial dump of data and then incremental updates.


On incremental updates, I want to be able to include a date in the csv file. E.g accounts042409.txt for new accounts on 24th April 09.

 

Is there anyway of doing this?


The process-conf.xml seems to require the name of csv file. I am not sure if I can wildcard it...  I haven't tried it, but would accounts*.csv work? 

 

                <entry key="dataAccess.name" value="R:\SF_data_exchange\accounts.csv"/>

 

ShikibuShikibu

I use LexiLoader rather than DataLoader, but for incremental updates, you might want to take a look at the free tool Informatica On Demand Data Loader Service for Salesforce.

 

I've experimented with it, and was impressed.

GoForceGoGoForceGo

Thanks. It does look like it is more user friendly and usable.


Do you know what the catches might be in terms of "Free"? Things that come to mind are 1) Support cost 2) Are there any limitations in free version that don't exist in a not-free version.

 

 

ShikibuShikibu
I think that support is paid (tho there's a free forum), and that they have three products -- they would like you to purchase all three, and hope you will do so after experiencing this one.

I experimented with it a little just before DreamForce, when it was brand new. Read up on their website, have a chat with their sales department, and let us know how you make out.
GoForceGoGoForceGo

Thanks will do. I have an e-mail inquiry into Informatica.

 

BTW the account*.csv thing above doesn't work.

ShikibuShikibu

GoForceGo wrote:

 

BTW the account*.csv thing above doesn't work.


Not surprised. Hopefully, informatica's scheduler will do the trick for you, with a bright shiny ui to boot.

GoForceGoGoForceGo

 

Doesn't seem like it is possible to use external ids to perform a delete operation using Data Loader. 

 

I am already using Data Loader upsert to perform initial blulk data load and incremental insert/update.  I am automating the incremental insert/update using the batch mode of data loader.

 

How do I do automated deletes as they happen in external system?

 

P.S: Let's assume that I have to stick with Data Loader for this customer (we are an ISV, so I have to figure out Informatica arrangement before I can commit to using it on this account) 

ShikibuShikibu
use external ids to fetch salesforce ids, then use those to delete.
GoForceGoGoForceGo

How would I automate that using Data Loader? I would have to write a separate program to do it.

 

The only way I can think of automating this would be to first upsert the records I want to delete and mark the with a flag (checkbox field) deleted.

 

Then I would have to extract the data using Data Loader with the query where flag = deleted.

 

Then I would have use this extract to delete the records...doable, except that I have start keep a new field deleted on each object...

 

 

ShikibuShikibu

Sorry, we're getting too far into the details of your business for me to be helpful. I don't use DataLoader, so don't know much about it -- eg, does it have a command line interface, and how do you write shell scripts on Windows (I don't use Windows either).

 

These are the kinds of processes that Informatica is targeting.

 

The other option is going to be writing code, whether it is shell scripts wrapping the data loader, or an external process that accesses salesforce via the web services api.

GoForceGoGoForceGo

Data Loader does have a command line interface. I wrap it with a windows dos script to do other things (such as move files after I am done processing), and it seems to work fine for incremental updates.

 

I am at the last obstalce of deletes nows....I will have to try the script with the work around above, upsert with a delete flag, extract from salesforce with id and then delete.

 

I don't' want to write another program to delete if i can avoid it...

 

 

 

 

 

GoForceGoGoForceGo

I was able to use data loader to automate this. Pain, but doable. I hope there is a better way

 

1. Add a field to every object called "Active". This is probably the least desirable part of the solution. By default this value is True.

2. Run three processes in batch mode using Data Loader

    a. Using ExternalId, upsert the records that I want to delete with Active  = false

    b. Run an extract with SOQL query [select id from xxx where Active__c = false). Save it to deletexxx.csv

    c. Run a delete  operation with deletexxx.csv.

 

 

Keerthi P 5Keerthi P 5
Hi GoForceGo,

I need help for Data Loader Command line interface to extarct incremental loads.. It would really a help if I can get some inputs

Thanks
Kirthy