Handling large ActiveRecord object set as binary data

Question

I have an issue with manipulating large number of record objects of type ActiveRecord model(Records are extracted from a complex operation but from the same table) and I want to pass that object set around between controller actions. So I have couple of solutions but is wage about which way to choose. Simpler but uglier solution is to save it in sessions. But as I know saving large object set in sessions clearly slows down application performance because of frequent serialization and de-serialization process(Serialization is time consuming I guess from Web services - SOAP and binary formatting). Second option which seems to be promising is saving all these ActiveRecord model objects as a blog in the database, and with one query we can retrieve all and can do the necessary operations with that dataset. But I don't know how to iterate through all objects after retrieving it from the database because it is still in binary object format. Calling inspect method shows all attributes and values but I am totally perplexed. Can anyone tell me how to save ActiveRecord model type objects as a single record binary obejct and retrieve it and go through the dataset. Do I need encoding like Base64 ?. Finally would memcached yield desired result?? or at least any links that can follow to solve this ?

And sorry for the lengthy question and thanks in advance.

Answer 1

The answer, and I hate saying this, is that probably the method is flawed. Is it going to take more time to query for this exact data again or to load up a BLOB row and unserialize it? My guess is that it takes roughly the same time without indexes but with indexes you may be able to get your entire resultset from the DB faster than even loaded the BLOB without even the unserialize call.

Now maybe I'm wrong about that, but certainly the amount of data coming back is about the same so a large chunk of the db time is exactly the same.

Memcached would certainly prove helpful here but only in the same way that the query cache (assuming you're using MySQL) would be helpful. Repeated queries for the same thing should be returned laser fast by either method assuming the cache in either case is large enough to fit the object.

But you are correct that storing in the session is a bad way and BLOB is probably also at best about the same as just doing the query again and letting the query cache return it.

But just to cover all the bases, how big is the total dataset vs how much is coming back and how long are you seeing the queries take? I'm guessing this post is really about performance but is clouded by implementational details.