How Do You Get Time to Document Existing Systems?

Question

We have a large system that processes logs from a very large web site. By large I mean a site that has more than 300,000 updates per day. Yes. Per day!

I've just finished an update to this system to correct a bug. Part of the problem doing the work was a lack of documentation for the system. Most questions were met with "Ask Andre, he'll know." or "Andre will know that." That is, our truck number is dangerously low, i.e. one!

This leads to such pointy-haired discussions as:

PHB: Why did the stats work take so long?

Me: Because there's hardly any documentation.

PHB: What are you doing now?

Me: Updating our TWiki to document the parts of stats system that I worked on.

PHB: There's no time for that. Start working on the code for the new SLA measurements.

Me: But, but.

I'm sure others have been in such an infinite loop.

I want to convince the "powers that be" of the good business logic of spending some time documenting the system?

Any suggestions as to what I should highlight? Especially while the "bits that I know about" are still fresh in my mind.

Edit: I'm not talking about being given time to document the complete system. I'm talking about documenting those parts of the system that you have touched. In this way you would slowly build a complete set of documentation for the whole system.

cheers,

Rob

Answer 1

You will never get time to do the documentation and most likely will fail to convince anyone that it is a crucial part of development.

You may try to go without splitting your development into 'coding', 'unit-testing' and 'documentation'. Treat these as one task and report your time accordingly.

So if someone ask you 'what are you doing' you could say you're working on reports. Documentation (in the same way as the comments and unit testing) are integral for the quality of the product.

If your boss have troubles with the speed it takes you to do the job properly it's more a problem of the planning on the company level and not yours.

P.S. Obviously don't try to document things outside of the part you'r working on. If you're changing the reports you could document reports, don't spend time documenting the UI for an unrelated screen of the security framework.

Answer 2

I make the documentation sort and sweet. You don't have to write novels with pictures and flow-diagrams and charts and tables to do effective documentation, especially if its intended to assist maintenance instead of showing something to the PHB.

First thing you need is a good document repository. Store your requirements in it, organised by project. So you get a requirement, change request, bugfix etc, you put it in there. You can also store your developed binaries in the same place, and a changelog (which takes 5 minutes to complete). Testing also places test scripts there, etc. The important thing about this is that it also de-couples you from the test or delivery teams, you will have to put all the information about how to install and configure your changes in the pot for them. You get used to this very quickly and it isn't that onerous - you'll need to say 'I changed this config value, and you need to put this file in this directory, or add this config file to the system'. Easy.

Now, when its all over, you will have a per-project repository with everything you ever needed to know about each change. The why, the what, the how, all will be there ready for the auditors to view.

It may not be as good as a full suite of documentation, but its good enough for most purposes. If you're not writing docs, then this will be a major benefit to you over what you do have. If you hire a technical author, he can take these scraps and turn them into a proper document, and his writing and presentation skills will be better than what you or I could come up with.

PS. it only works if every change, no matter how minor, is tracked in this way. A single character change to a file will still need to go through the process, you will still need a bugreport and a (very short) request made. On the other hand, even miniscule bugfixes are still bugfixes and need to be handled through a process anyway.

PPS we use a combination of mantis and dotproject for this, both open source and easy to use.

Answer 3

In my opinion I wouldn't set aside time to document the system as chances are your documentation won't cover the areas, or detail you need next time you come to make a change.

To cover all areas all areas in sufficient detail would probably take a long time, which will be very hard to justify, as firstly your time is valuble as there are other areas you need to be working on, and secondly because it will slow down delivery of other needed functionality quite radically, as there aren't many developers on the project.

Instead of asking for certain amount of time, set aside for documenting, I would document areas as you are involved in them, I also wouldn't document at the end of delivery, but document while you are planning, developing or fixing a particular area.

Doing it this way will highlight the area you need to develop, possibly highlight problems with your solution, and aid in your thinking in developing a solution.

Thus over time your documentation will improve, and your boss probably won't have an issue with it, as you'll be doing it spread across a number of months. (Chances are he won't even notice.)

Answer 4

One way that has worked for me is to do documentation similar to the way tests are written in TDD. You document the process up front before you do the work based on how you think it will happen. Then you try and follow your own procedures. When you realize your process is wrong or incomplete you correct it immediately before continuing the process.

When you have finished the task, your documentation is already done and you can be confident that it is correct because you followed the steps to complete the task. It is also very interesting what you can discover about the process when you try document it before you do it.

Answer 5

You are fighting against business rules: does the system function well enough to produce the expected profit? Yes - don't waste time on it; No - fix it.

In your case it seems that the lack of documentation for the system does not (YET) have enough of a detrimental impact on the profit margin. This clearly indicate your options: show management the benefit(profit), not in maybe's and if's but the real benefit, e.g.: the savings in time (= expenses) should there be proper documentation.