Ruby, MongoDB: How to share a Cursor between threads?

Question

The following does not work. The call to resources.next_document within the thread returns nil. The same call without threading works as expected.

Any MongoDB experts out there? :P

  resources = db[Resource::COLLECTION].find 

  number_of_threads.times do
    threads << Thread.new do
      while resource = resources.next_document
        puts 'one more doc'
      end
    end
  end  

Answer 1

Although the driver itself is threadsafe, individuals cursor aren't, so you can't reliably process the data in the way you're describing.

One possibility would be to have a single thread that iterates over the documents, handing them off to any number of worker threads for the actual processing.

Answer 2

This is the solution I ended up using:

Feedback welcome

pool = DocumentPool.new(db)
5.times do 
  Thread.new do
    while doc = pool.next_document
      #something cool
    end
  end
end


class DocumentPool   
  COLLECTION = 'some_collection'

  def initialize(db)
    @db = db                
    @first_doc = cursor.next_document      
  end

  def collection
    @db[COLLECTION]
  end

  def cursor
    @cursor ||= collection.find
  end   

  def shift
    doc = nil
    if @first_doc
      doc = @first_doc   
      @first_doc = nil  
    else
      doc = cursor.next_document    
    end
    doc
  end                               

  def count
    collection.count
  end
end