How would you store tabular constant Data in your C# project?

Question

I am currently writing a plugin to a CAD style software. The plugin does calculations based on data read from the CAD model and a lot of table lookups (think printed tables in a calculation guide). I inherited this plugin and the current solution defines a class Constant which has a bunch of static struct members and two-dimensional arrays. These arrays are then indexed by enum values at runtime to find the appropriate data.

I'm not too happy with the sollution, as the representation in the Constant class is kind of hard to read - the enum values used when retrieving data are of course not visible when editing the data (allthough that only ever happens manually and very seldomly).

I'd prefer not to bundle a DB (and engine) with a small plugin, but would like similar semantics, for instance using LINQ to select values where some fields match etc.

What is your preferred solution to this problem?

• do you use a bunch of XML files and parse them at runtime?
• do you use a templating engine (t4?) to generate classes from XML files at compile time?
• do you store XML versions of datasets in the resources (read 'em in at runtime, LINQ to dataset...)
• would you just keep the Constants class, maybe add some documentation to the members (don't get me started about legacy code with no comments whatsoever...)

Answer 1

Unless performance is a constraint, I'd go with XML-dataset-in-resources approach. It's fairly obvious (so whoever picks up the code from you in the future won't be befuzzled by its look), it's easy both to edit and to access, and there's no need to reinvent the wheel in form of custom parser of tabular XML data.

Answer 2

I'm doing that at the moment for a performance-critical application. The way I do it is by serializing the data in a flat file (as part of my release process), then (at launch) deserializing it into a class model, allowing LINQ-to-Objects querying.

In most scenarios xml would be reasonable, but by preference I'm using my own binary serializer - mainly for speed.

Since the constant data doesn't change, it is usually fine to cache it away in an instance you keep handy and re-use many times (rather than deserialize it per use).

To clarify: the data to use is stored in a standard database (they are good at that type of thing, and lots of tools exist for import / edit / query / etc). As part of my release process, I load the data into the object model (using the same classes) from the database, and serialize it:

// during release
MyDataModel data = new MyDataModel(); // wraps multiple data lists
data.Load(); // from database tables, using ORM
data.Save("data.bin"); // serialization

then I ship data.bin with the app (well, actually it is stored separately, but that is an aside...); and at runtime:

MyDataModel data = new MyDataModel();
data.Load("data.bin"); // deserialization
data.Freeze(); // make immutable

Note that in this case, the data is "popsicle immutable" - i.e. to preserve the "constant" nature (but while letting it edit the data during load), I have a cascading Freeze method that sets a flag on the items; once this flag is set, all edits (to items or to lists) throw an exception. This has zero performance impact or the running code: since you expect to treat it as constant data, it only does reads!

Answer 3

You can generate the Constant class.
This way you can store your data in a more convenient method, such as MDB (MS Access) or MDF (SQL Express) file. Editing these files is user friendly.

I suggest you have an additional project that generates your main project's Constant class with all the needed data as constants. The extra project can be a Console Application. Use a pre-build event in the main project to execute this console application, and then you have your generated class.

Answer 4

I'd second the suggestions to use XML (be it resources base or not) but like to elaborate a little on the questions of parsing/editing/handling this scenario.

---

First a short note concerning the question related to Pavels answer whether visual editing XML via a table editor in Visual Studio (VS) is possible: it used to be in VS 2003 in fact (see e.g. 'Figure 6.28. XML document: data view' here on how this looked), in VS 2005 the feature got demoted somewhat (see e.g. XML Appearance in Visual Studio 2005 versus Visual Studio 2003) and in VS 2008 finally removed (see e.g. XML Data Editor missing in Visual Studio 2008), which is pretty unfortunate, as it had its uses and might have helped you too here. The same (likely related) fate applies to the XML schema designer, which at least will get a replacement in VS 2010, don't know about a new XML data view yet.

---

Back to your original question: given your requirements you could supply a dedicated XML schema for your XML files. Once in place, any decent XML editor will be able to supply Intellisense at least (i.e. enum values are visible while editing) and warn you about erroneous data entries too. If editing XML sources this way still feels too 'raw' you might get a more comfortable 'non source' editing via something like XML Notepad 2007.

Yet the most important benefit of having a XML schema is that this allows automated use via tools, in particular you can generate both, plain old .NET classes and/or LINQ enabled datasets from it via Microsofts XML Schema Definition Tool (Xsd.exe). (For well known structured data, i.e. backed by XML schema, this is much preferable than using a tedious custom t4 based solution indeed.)

Granted, getting schemas in the first place and getting them right in practice might require some efforts, but these pay off big time for any serious XML related development, imho.

And for the task at hand you might actually be better of already and be able to infer the XML schema from the existing class Constant, see the '/t' switch for Xsd.exe. Likely the initial result might be incomplete or not entirely adequate, but with a bit of up front refactoring and/or manual tweaking you should be able to derive a decent schema pretty quickly.

As a result you you should be able to work with your codebase pretty similar like you do now (presumably), i.e. you'll have strongly typed enumerations etc., or you you could switch using the XML data via LINQ enabled datasets as you please (in fact you could do both in parallel).

Answer 5

If deployment is a concern, you could always embed an XML file inside your assembly. This allows you that same level of constantness, as users will not be able to muck around with the data.

Any file can be embedded as a resource using VS, just add the file to the project and select: Build Action => Embedded Resource from the properties.

Then on startup you could call:

// name is the name of the embedded resource
Assembly.GetExecutingAssembly().GetManifestResourceStream(name);

To get a stream to the embedded data.

The advantage of this approach is that you can keep your constant data in whatever format you want, which would make maintenance much easier. On startup you can do all the indexing and organisation so the data has a proper API.