how to handle large dataset like sproutcore

Question

Hello all, I really don't have any substantial code to show here, actually, that's kinda why I am writing: I looked at the SproutCore demo, especially the Collection demo, on http://demo.sproutcore.com/sample_controls/, and am amazed by its loading 200,000 records to the page so easily. I tried using Rails to provide 200,000 records and in a completely blank HTML page with <% @projects.each do |p| %> <%= p.title %> <% end %> that freezes the browser for seconds on my m1530 laptop with 4gb ram and t7700 256gb ssd.

Yet the sproutcore demo does not freeze and takes less than 3 seconds to load.

What do you think the one technique they are using to enable this is?

Thanks!

Answer 1

More on sproutcore here.. http://ostatic.com/blog/sproutcore-raises-the-bar-for-client-side-programming

If on the other hand you are looking for concurrency then node.js is the way to go.

Answer 2

I imagine the demo provided isn't being generated dynamically - it's static data.

Very few systems would be able to iterate a collection of live data that size. There are a number of techniques including streaming the dataset (using batch iteration through the records) through to caching and ajax partial loading strategies.