Recommendations for persisting data on Android?

Question

There is a web service that provides some data that my app makes use of. This data is fairly large and only changes VERY infrequently so I thought it would be nice if the app could cache it on the SD Card and only update it as needed.

Currently I'm grabbing the data (an XML file) and parsing it into an object tree using SAX. This process takes (at most) 2-3 seconds over my WIFI. However, serializing the resulting objects to the SDCard takes significantly longer (a minute or more) and deserializing it still takes longer than just download/parsing in the first place.

Does anyone have any recommendations for improving this or alternate ideas for persisting this data (other than just saving the XML file and reparsing every time)?

UPDATE: This is more than a trivial collection of records. The object-graph is actually ridiculously complex and storing it into a database would result in dozens of tables with only a single record in each one.

Answer 1

Why don't you use database? See Android Data Storage Guide

Answer 2

I've just been writing an android application for the last week which basically does this. It fetches some (large) XML file online, and then displays part of the data in various views.

We do it by fetching and parsing the XML using SAX, and (while parsing) writing it all to a SQLite database. And then we are just querying the database each time we need to display some view of the dataset.

Works like a charm, and is fast enough for displaying a lot of data on a google map overlay, where we are querying the database on every single call to the draw method of our map overlay.

So I would definitely suggest going for a SQLite database, if the data in the XML document is easily represented in a database.

Answer 3

Android serialization is notoriously slow. I highly suggest switching to using XML or JSON (or whatever) and writing the file out that way. Since you've already got an XML parser, it may make the most sense just to cache the original XML file you downloaded and reparse it as necessary.

I have switched from Serializable to JSON file storage in an app before and the speed increase was incredible, at least one order of magnitude.

(I may be misunderstanding your question - I assume you are using Serializable for writing to the disc. If you are reproducing the XML, then I'm not sure why it is so much slower on the SD card. Also, I agree that the SQLite database makes the most sense typically, but as you've already stated it does not fit the needs of your application.)

Answer 4

Also unless your data is at least 100s of Kb, I would suggest just storing it in your private data storage instead of on the SD card. Keep in mind that you can't rely on the SD card being available.

Answer 5

Hi!

If the web service can give you just a specified number of results(something like: requestData between index 1 and 10 or give me first 25 results) try to use that (put a simple "Load more results" button or implement an auto-loading mechanism). If the web service not provide this feature then try to save your xml on sdcard and when you need the data try to parse just a specified number of results. Hope this help!