tags:

views:

20

answers:

1

Whenever I import a text file with about 2 million rows and 2 columns into SQLite Administrator, it takes 3-4 hours to do so. Is it normal, or am I doing something wrong?

The way that I do it, is to take a tab delimited text file with rows, change the extension to .csv, and feed to SQLite Administrator. My PC specs are 2 GB RAM, Core 2 duo 1.86GHz. I also have about 10Gb free disk space when importing data.

A: 

Apparently, SQLite has performance issues in this field. Check this thread for more information.

You can try to do some performance tuning:

  1. SQLite Docs: Pragma
  2. SQLite Optimization FAQ
  3. SQLite Optimization
  4. SQLite Performance Tuning and Optimization on Embedded Systems
Hal
According to the thread that you linked to, it seems that problems only start after the size becomes more than some gigabytes. But in my case, the size of the file is less than 100 MB.
Sunny88
Not really: "When the file was about 7GB (sorry I can't be specific about row counts) insertions were taking far too long. I had estimated that my test to insert all my data would take 24 hours or so, but it did not complete even after 48 hours."
Hal
So it's plausible that your 100MB file takes that long
Hal
Also, only import takes so much time. Other things, like sorting, take a couple of minutes maximum(for my file).
Sunny88
Yup, because SQLite's issue is write speed, not read.
Hal