Should I unit test XML schema?

Question

I am in the very early stages of writing XML schema for my work's enterprise application. The XML to be validated represents an application - similar to Winforms - forms, grids, menus etc but without layout.

The prime purpose of the XSD is not so much to validate the XML but to add design-time discoverability to the XML file, so that one would get IntelliSense for available elements and attributes.

As I am writing up the schema I have found myself doing elements of TDD and validating a document against the schema, changing either elements/attributes in the document or in the schema to make validation fail to make sure I am writing the schema correctly.

This brings me to the question of whether or not I should be unit-testing the schema, just to throw a few permutations of the XML at it and make sure it behaves how it's supposed to.

It would certainly make sense to me as my XSD-fu is abysmal and I want to be even more sure the XSD, which in effect IS a specification by itself, is correct.

Answer 1

If you're not good with XSD, then I recommend you build the XSD using TDD. That is, create a failing unit test, involving validating some XML you mean to have work, and an XSD that won't permit it to validate. Then, update the XSD to permit that XML to validate. Then, refactor the test and the XSD, repeating the tests.

Answer 2

I guess there is no reason not to unit test XML Schema. If it is code (which it is), then TDD will favour testing it.

The other question will be how to go about it? A simple approach will be to create two groups of sample xml files, one containing all the files that are not valid according to your design and the other containing files that are valid. Then simply parse each group with your xml parser and assert the results.

But the actual challenge will lie in creating the sample xml files. Especially if you are also evolving your design simultaneously.

Answer 3

Generally I find it very difficult to test XSD schemas:

• For XSD often the modeling of your domain is the key, often I didn't have problems with the XSD constructs itself but I analyzed the domain wrong.
• The generated model isn't based on the XSD itself, but also on the binding configuration (e.g. JAXB for Java). So in the end you are testing too much.
• Such tests depending on lots of things tend to break often, especially when you refactor the XSD.

In the end to improve XSD quality I prefer:

• Have early reviews of XSDs (by colleages or QA). Having actually people look at them (both XSD and XML instances) discovered flaws whcih would never been found out by automated testing.
• Do Integration testing on the generated xml instances. These integration tests can be automated.