[SalesForce] Data Loader – Automated way to import data with parameters from Command Line

I found Salesforce's own documentation pretty useful on Data Loader and relationships. Good enough to just copy verbatim (aside from minor formatting).

Description of Steps

1. Add a Text field to your parent object. Make sure you set the length to 18 and mark it as an External Id.
2. Export your parent records with the Id field included.
3. When inserting parent records, make sure you map the Id to your newly created External Id field from Step 1.
4. Upsert the child records based on Id. You can then set what External Id field to use as a lookup for each parent object. Select the field you created in Step 1.
5. Optionally delete the field you created in Step 1.

Pros

• Can be implemented via Data Loader (free)

Cons

• Requires creation of a new Custom Field

Documentation referenced above:

1. In the destination Organization, on the Parent Object (in this case, Account), create a new field of type Text and mark it as an ‘External Id’ field. Be sure to make the length of the field at least 18 characters (the size of the Salesforce ID field). Let’s call this new field on Account – ‘Source Acct ID’.
2. Export the Parent Object (in this case, Account) data from the source Organization into a csv file using Data Loader. Be sure to include the ‘Id’ field when exporting this data.
3. Insert the data from Step 2 into the destination Organization using Data Loader. In the mapping screen of Data Loader, map the ‘ID’ field from the csv file to the new field created in Step 1 (in this case, ‘Source Acct ID’).
4. Export the Child Object (in this case, Opportunity) data from the source Organization into a csv file using Data Loader. Be sure to include the relationship field (in this case ‘AccountID’) when exporting this data.
5. Upsert the data from Step 4 into the destination Organization using Data Loader. In Step 2a of the Data Loader, select ‘ID’ as the field to use for matching Opportunities. In Step 2b of the Data Loader, select the field created in Step 1 (in this case, ‘Source Acct ID’) as the field to match the parent object. In the next mapping step of the Data Loader, map the ‘AccountId’ column of the csv file to the ‘Account:Source Acct ID’ field. This will allow the Data Loader to associate each Opportunity record with its respective parent Account record. Note that the ‘AccountId’ column of the CSV file should NOT be mapped to the standard ‘AccountId’ field (which is what the Data Loader does by default with the ‘Auto-Match Fields to Columns’ option). Instead, it has to be mapped to the ‘Account:Source Acct ID’ field.
6. After finishing the data import, the new field created in Step 1 (‘Source Acct ID’) may be deleted as it was only necessary for the data import process.

In the folder that contains the process-conf file, do you also have a file called log-conf.xml? Because if you don't, I know DataLoader starts logging everything, and creates a huge file...

Do you also have a config file in the folder? If not specified in the bean, that will also set parameters...

I have the command line setup on my machine for various projects - happy to to do a comparison test to check whether it is tool or machine specific - which version are you using?

AMENDED: I added the log-conf.xml and set the priorities to 'OFF'. Not sure why the setting in the process-conf doesnt override this, I am not sure.