tags:

views:

420

answers:

2

In Ruby 1.8.6, I have an array of, say, 100,000 user ids, each of which is an int. I want to perform a block of code on these user ids but I want to do it in chunks. For example, I want to process them 100 at a time. How can I easily achieve this as simply as possible?

I could do something like the following, but probably there's an easier way:

a = Array.new
userids.each { |userid|
  a << userid
  if a.length == 100
    // Process chunk
    a = Array.new
  end
}
unless a.empty?
  // Process chunk
end
+14  A: 
require 'enumerator'
userids.each_slice(100) do |a|
  # do something with a
end
sepp2k
Note that you actually have to explicitly "require 'enumerator'" for this to work: the method is not available in classes that mix in Enumerable, which initially led me to think this answer was wrong. Then I learned better.
Mike Woodhouse
Yes, you have to require 'enumerator' in 1.8.6 for this to work (which is why I did). In 1.8.7+ enumerator has been moved to core and you no longer have to require it. However doing so will not cause an error, but simply return false. So for compability reasons you should always require 'enumerator' when using methods from enumerator, so that the code will work with all versions of ruby.
sepp2k
Thank you very much. :)
ChrisInEdmonton
+2  A: 

Rails has in_groups_of, which under the hood uses each_slice.

userids.in_groups_of(100){|group|
  //process group
}
wombleton
We don't use Rails. It doesn't scale sufficiently far for us; our databases are sharded across about 26 shards. Plus, we have a significant number of other database servers, probably another twenty or so, though these aren't sharded. Thanks for the suggestion, though, I'm sure that'll be useful for plenty of other people.
ChrisInEdmonton
Yeah, got that you weren't using rails which was why I linked through to the source so you could pull the method if you wanted it.
wombleton
And why I awarded you a +1. :) A good answer that didn't specifically work for me, but would for others.
ChrisInEdmonton