ansaurus

Question

Answer 1

+1 A:

If you run each INSERT statement in it's own transaction, SQLite can be very, very slow. But if you run it all in one transaction (or a logical set of transactions), then it can be very fast.

gahooa 2009-12-24 23:38:06

Could you explain in more detail - how i can flush all prepared data in one transaction. I use Ruby on Rails to generate some data randomly (for testing purposes).I only know the ActiveRecord:Base.save() method, but, as far, as i know, it must be called for each created object, but not for the whole model ... or ?

AntonAL 2009-12-24 23:54:57

Answer 2

A:

Seed_fu could help, as discussed in this question

zetetic 2009-12-25 00:05:55

Answer 3

+1 A:

Quite simply, inserting large amounts of records through manually saving AR objects one at a time is going to take years.

The best compromise between speed and "cleanness" (i.e. not a complete dodgy hack) for inserting large amounts of data is ar-extensions's (http://github.com/zdennis/ar-extensions) import method. It's not ideal, but it's better than any of the alternatives I could find, and the syntax is clean and doesn't require you to drop to raw sql (or anywhere close).

Example syntax:

items = Array.new
1.upto(200) do |n|
  items << Item.new :some_field => n
end
Item.import items, :validate => false

At least in mysql this will batch the records into a single INSERT statement with multiple sets of values. Pretty damn fast.

Michael 2009-12-25 00:25:52

ansaurus

tags:

views:

answers:

Ruby on Rails migrations are very slow

related questions