Documenting a File Format

Question

We're developing an application that receives a lot of it's input from a custom file format we've defined. The input files are produced by an application being developed by a third party so naturally we need to be able to communicate effectively what is and isn't acceptable input into our program.

The file has a header portion of 10 lines each line representing a different piece of meta information followed by a newline and the "footer". The footer contains the guts of the information. The header is fairly straight forward but when we come to the footer each line has several fields that start at a predefined character offset.

I was wondering if anyone else has been in a similar situation and could offer up some hints on how you might document these types of file formats. The documentation of this file spec would most likely be displayed on a Wiki however I would also like it to be suitable for publication in a hard-copy manual.

Answer 1

The second paragraph of your question would be a good introduction to your interface document.

Then I would add two sections, one for the header, the other for the footer.

The header section contains a description of what is expected on each line.

The foorter section is a table where each line describes one field and the columns are:

• field name,
• offset,
• length,
• purpose

Answer 2

I've helped write an XML Schema in the past which is roughly similiar to this.

My advice would be to be absolutely explicit in documenting exactly how the file is constructed, so no ambiguity exists.

So things which you need to consider are:

• Spacing
• End of line markers
• End of file markers
• Allowed characters in each area of the file.
• Allowed value ranges in each area.
• Any encoding which is required, and what method/algorithm to use.

Ideally you can write a validator program to give to the third party, to validate files. Thus minimizing the chance of something going wrong.

Answer 3

The answers from mouviciel and Bravax are helpful. One deals with display the other with some technical aspects which your document should supply about your file format.

I've been at both ends of this problem (creating/using file format specification). Here's a list of things I wished I had when I was reading documentation.

• Having a visual representation of the file format.
• Having an example of the file format. Including examples of files that document creators might think are legitimate but aren't would also be helpful.
• If your documentation is electronic, having it hyperlinked to relevant parts is helpful.
• Providing a justification for the file format. This helps people understand why they are doing something, as opposed to just blindly following a spec.

On the technical side, create flexibility within the file format for later additions. Having a header is a great idea. I would even create a version number header line along with a "length of header" attribute so that if you need to add other meta attributes in the future you can!