Best way to store application data when data stored and data format could change in future versions?

Question

I'm making an Android Java app game (although this question applies to all languages really) and hope to release the first version soon. I'm nervous about how I save data in my game. My problem is that, if in a later update, I decide to store more data or store the same data in a different way, I need to be careful I don't lose or corrupt data for users that upgrade (i.e. I want users to be able to use data created by an old version in the new version, like their high scores from before).

For example, say I want to save high scores in version 1 and I use a class like:

class Score { String name; int score; }

I then store the scores in an ArrayList and then, to save/load the scores, I serialize the ArrayList object.

In version 1.1, maybe I decide I want to store the date with each score. I could then update the Score object to include a date field and use default values for date if an old object does not include a date.

In version 1.2, maybe I decide that I want to store scores in an TreeSet instead and in version 1.3 perhaps I want to load/store scores online as well.

Anyway, my point is, are there any general tips for making my life easy here? I'm particularly concerned about situations from the above, where one person upgrades from version 1.1 to 1.2 and one person upgrades from 1.0 to 1.2. Testing all scenarios for data corruption sounds like a headache.

Is it really just a case of thinking really hard to pick something sensible and scalable to start with?

I'm thinking it might be easy to use a HashMap from String->Object as a general purpose storage object e.g. storage.put("HighScoreName1", "Bob"); storage.put("HighScorePoints1", 15);. Getting the information out is a little messier than if I'd have used a custom class, but it seems easy to add extra fields and so on without much work. Iterating over lists stored in this way isn't great though.

Answer 1

There is no definite answer, but the term you are looking for is versioning. Best serialization implementations (and file formats) can have that working both ways (old versions of the application can read files of new versions, and vice-versa).

Answer 2

If you are using SQLiteOpenHelper, have a look at the onUpgrade method.

Called when the database needs to be upgraded. The implementation should use this method to drop tables, add tables, or do anything else it needs to upgrade to the new schema version.

Answer 3

Write a version number to your output stream before you write the serialized content. I also suggest writing a magic string sequence to the front of the stream (just something that you can say is yours - maybe 5 or 6 characters). So you'll write MAGIC, then you'll write your version number. Then you'll write your serialized content.

Reading from disk is fairly straightforward - read and confirm magic. Read version. Deserialize, then pass the deserialized content on to a handler based on magic.

A couple of words of caution here, though: If possible, keep the classes you use in the serialized stream simple (String, Integer, etc...). If you use your own classes, you must be insanely careful that you never have to refactor in a way that would change the package or classname.

As tempting as it may be to just write your business objects to storage using serialization, over the long run, this is almost always a mistake. Instead, I strongly recommend developing a Configuration object. Your app initializes itself form the Configuration object. When it's time to save, the app constructs a Configuration object. Then you never, ever, ever change a Configuration class once it ships. Just create a new one (sub-classing is fine).

By separating the reading and writing of configuration objects, you can do the following pretty easily:

1. Read old configuration
2. Initialize app with old config Later...
3. App creates new configuration
4. Write new config to file

presto - instant file format update.

Obviously, this sort of thing is tough (a lot of work, anyway) to do with big, complex data graphs - but this is Android you are talking about, so your persistent state is probably not that complex.

Note that saving as XML, or even to a SQL database is just a different form of the above strategy.