This feels like a really bad idea on implementation and I'm not sure of the speed savings given the potential cost in memory.
Using Dalli and Memecache to store a set of data that gets pulled on every page request that needs to be paginated. Its a lot of data so I'd really rather not keep hitting the database if possible, but I also don't know how expensive this operation is for memory, both in memcache as well as just in system memory. To paginate, I do basically the following:
- Call to memcache to see if given data exists
- If found, get data and go to 4.
- If not found, get from database and cache
- Dup object because object is frozen. ?? <-- seems like a really bad idea...
- Return paginated data for display.
WillPaginate has to perform actions on the dataset coming out of the DB but it throws a "can't modify frozen object" error.
How much bad is this? dup-ing a large object (which is cached to save time in calls to the db) seems like it will end up chewing up a lot of extra memory and run the GC a lot more than needs be. Anyone suggest some ways around this?