Good strategies for REST -> XML -> Core Data -> UITableView?

Question

What are good practices for asynchronously pulling large amounts of XML from a RESTful service into a Core Data store, and from this store, populating a UITableView on the fly?

I'm thinking of using libxml2's xmlParseChunk() function to parse chunks of incoming XML and translate a node and its children into the relevant managed objects, as nodes come in.

At the same time that these XML nodes are turned into managed objects, I want to generate UITableView rows, in turn. Say, 50 rows at a time. Is this realistic?

In your experience, what do you do to accomplish this task, to maintain performance and handle, potentially, thousands of rows? Are there different, simpler approaches that work as well or better?

Answer 1

Joe Hewitt (Facebook app developer) has release much of his code as open-source. It is called Three20. There is a class there that is great for fetching internet data and populating it into a table, without the need for the data beforehand. The classes used for this are called TTTableViewController and TTTableViewDataSource.

From here, it would not be much of a stretch to store as CoreData, just subclass the classes as you see fit with the supplied hooks.

If you are worried about too much data, 50 at a time does sound reasonable. These classes have a built in "More" button to help you out.

From the Three20 readme:

Internet-aware table view controllers
TTTableViewController and TTTableViewDataSource help you to build tables which load their content from the Internet. Rather than just assuming you have all the data ready to go, like UITableView does by default, TTTableViewController lets you communicate when your data is loading, and when there is an error or nothing to display. It also helps you to add a "More" button to load the next page of data, and optionally supports reloading the data by shaking the device.

Answer 2

Sure, this is a pretty standard thing. The easiest solution is to do the loading in a background thread on one MOC, and have the UI running on the main thread with its own MOC. Whenever you get a chunk of data you want to have appear (say 50 entries), you have the background MOC save:.

Assuming you have the foreground MOC rigged to merge changes (via mergeChangesFromContextDidSaveNotification:) then whenever you save the background MOC the foreground MOC will get all of those changes. Assuming you are using NSFetchedResultsController it has delegate methods to cope with changes in its MOC, and if you are using Apple's sample code then you probably already have everything setup correctly.

In general CoreData is going to be faster than anything you roll yourself unless you really know what you are doing and are willing to spend a ton of time tuning for your specific case. The biggest thing you can do is make sure that slow things (like XML processing and synchronous flash I/O caused by save:) are not on the main thread blocking user interaction.