embed vs link on large mongoDB sets (ruby)

Question

Embedded vs link

I'm looking for the fastest way to search a Newsletter document for a connected Email. So far I have used MongoMapper with one document for Newsletter and another for Email. This is getting really slow with +100k Emails.

I was thinking maybe its faster to embed the emails in an array inside Newsletter since I'm really only interested in the email ('[email protected]') and not any logic around it.

1) Is it possible at all to embed as much as 100k-500k emails in one document? 2) Is Mongoid better/faster for this?

I'm adding the email if it is not already in the collection by asking

email = newsletter.emails.first(:email => '[email protected]')
unless email
    email = Email.new(:email => '[email protected]', :newsletter_id => self.id)
    email.save
end

And I think this is where it all starts to hurt.

Here is how they are connected Class Newsletter include MongoMapper::Document many :emails ... end

Class Email
   include MongoMapper::Document
   key :email, String
   key :newsletter_id, ObjectId
   belongs_to :newsletter
end

would love for any help on this :)

Answer 1

Did you ever get a solution for this? Did you try adding indexes to both collections?