How many records can be loaded into Salesforce using Apex Data Loader in one go?

Question

We have to insert thousands of record into Salesforce using Apex Data Loader. We use csv files to load data.

Answer 1

Well, there's no real limit in the Data Loader itself. From the Data Loader guide:

Use the Data Loader when: You need to load 50,000 to 5,000,000 records. If you need to load more than 5,000,000 records, we recommend you work with a Salesforce.com partner.

Really check out the PDF, especially for the (little) info about "bulk API" if the speed of normal loading is insufficient for you and you have already tweaked the number of records sent in one batch (by default 100 or 200 I think).

And if the bulk API looks too complex, you can always use very simple parallelization of the task - one username can have up to 4 sessions open, so you could invoke Data Loader 4 times with 1/4 of original CSV.

Answer 2

Data Loader definitely works fine with thousands of records. I have a setup that does this on a daily basis.

One good feature of the Data Loader is the ability to import data from, or export data to, a database. This way, you can avoid having to use CSV files (assuming that your data is available in a database).

By using a database as the source of your data, you can implement incremental loads. The Data Loader remembers the last time it ran, so you can write a database query that extracts all records added/changed since the last Data Loader run. That way, you won't need to load the complete dataset each time.

Also, Data Loader can perform an "Upsert", which combines an INSERT and UPDATE. That means you can load new records and existing records at the same time, with the existing records simply being updated. That's a feature not normally available in SQL situations.