What do gurus say about Requirements Traceability Matrix?

Question

Our organization is at CMMI Level 2, and as part of the requirements of the level, we have to maintain an RTM which more or less contains the following entries for each requirement:

• Requirement Description
• Reference Section Functional Specification Document
• Reference Section Design Document
• Reference Section Test Cases Document

Now, this might be an overkill for a small project. But more importantly, this could be a nightmare to maintain when the requirements/ features keep changing, and documents have to be constantly updated.

What do the gurus say about this? Should one avoid such level of documentation or are there any tools to manage when a "change" out dates so many artifacts? And by using the term 'gurus', I am not talking of coding champs; rather people like Steve McConnel or others who have worked on commercial projects of medium to large scale.

Quotes/ book references/ articles will suit me.

EDIT: It's not just requirements that change. Design Document can change; well, even test cases may change.

Answer 1

If your requirements are constantly changing then you have more issues than just keeping your RTMX up to date -- how can you hope to develop your code?

CMM level 2 also talks about establishing configuration management; specifically a Configuration Control Board which will help reduce the churn on your requirements by managing change properly.

Answer 2

The RTM is an instrument that should help you dealing in an organised way with requirements changes, not an administrative burden that has no or insufficient benefits. If it is a burden rather than something that helps to organise your work, then obviously you have to change your practices.

Answer 3

It might be worth checking the requirements analysis section on SQAF, I'm not into CMMI myself but I know that many others on this forum are. As Stewart says, requirements tend to be dynamic in many cases, and static modelling techniques can fail to provide a reasonable return for the effort required in maintaining them.

Answer 4

The Requirements Traceability Matrix gives you a number of things:

1) The ability to know which version you should be developing with. In the document, you should know that version 2 of the Functional Spec applies to version 2.3 of the Design, which in turn applies to the version 4.5 of the test cases. This is very useful so that the developer and tester know what to do.

2) It gives you a list of things to update for each requirement, if it changes.

Its worth saying that if there are a lot of changes to requirements, then a document that helps you trace those changes is very useful, if it is kept up to date.

Answer 5

Any decent sized project to be really successful over the long term needs to perform the steps you described, but not necessarily in the manner you've described that you must do them. Look at what's happening in the market and there's a huge trend of moving towards Wiki's as well as test driven development. These systems, more and more, tie everything together as a natural part of every project members daily working practice. My personal experience is that having a table/matrix in a document somewhere means that someone has to maintain it, one mistake and the trail starts to become obscured. You end up with process descriptions which are a list of changes, which inevitably become impossible to piece back together.

Nirvana is where a request/requirement is logged, documents are actually linked to via a system, from which they are checked in and out of. There are approval processes, in the system along with audit trails. Tests are written and automated as much as possible, running of the tests is tracked and logged, even if manual. There are processes for test escapes to make there way back into the process from the requirements level. Code is built against requirements, tests and escapes (bugs). The source code management system is tied to this central repository and no change to the code is allowed, unless it can be linked back.

There is full end to end traceability through the systems. The systems are structured so that people interacting with it will find it easier to do their jobs and have less stuff-they-don't-like to worry about.

At many companies I've used Atlassian Confluence, Jira, Subversion (with hooks to Jira), fisheye and cruise control to provide these facilities. These systems are far from perfect but much much better than the riduculous manual process of someone responsible for maintain a table in every one of 2000 documents. Even Microsoft project server/sharepoint (if implemented properly) can be used to do this today and provide a single site where projects/docs/developers all come together for about $15K. Is that cost justifiable, even on a small project?

Answer 6

"Now, this might be an overkill for a small project. But more importantly, this could be a nightmare to maintain when there are changing features and documents are constantly updated."

Yes, this can be very hard to maintain. The way it happend on a couple of very large projects where I worked to this kind of quality level (ISO9000 etc) was to be extremely harsh on adding new requirements to the current work plan. This means that the requirements stay stable and it is not too hard to keep all the documents, code, acceptance test in line. It is still a full time job for several admin staff and a daily overhead for every engineer (but I am talking here about 1M+ LOC projects).

New requirements are only admited after rather lengthy consideration. The process is basically there to make the customer think very hard about adding a new requirement to the current release. Adding requirements to future versions is fine, but to add one to the current phase is costly. There is a fixed fee payable just to submit a requirement and get it read by an architect and presented to a change control board. Once this is done a quote is given to the customer (and if the new req is complex the quote may be just for further investigation and not even a commitment to implement the feature). It is then up to the customer to decide whether or not to proceed, and if they do then the team undertakes to do the work and adds the req to the matrix, fixes the docs, adds new test and so on. The schedule may also need to be adjusted at this point.

This approach might not sound commercialy feasible if your organisation needs to be open to new requirements. If this is the case then you can still have tracking and a managable level of overhead if you go for frequent iterations. That way your current version stays stable as you work on it and you can keep the matrix and docs up to date; if your customers have new requirements they can add them to the "wish list" for the next version, but you keep them out of the current version. When it comes to the next version you take on all the new ideas then, and spend time creating a new matrix only after the previous version has shipped.

What do the gurus say about this?

Here are some references, these are just personal favourites. I haven't read anything specificaly on CMM so can't point you there I'm afraid.

Anything by McConnell is worth reading ( I think you mentioned him, but worth repeating I feel), but I think Professional Software Development might be most relevant to you. It is a collection of essays, some of which are online at the above link, ch. 13 "Business Case for Better Software Practices" seems to be the kind of thing you are after.

Peopleware and Brooks' TMM-M are 2 of the classics, they don't really touch on the specific issues you mention and are too early for CMM, but if you haven't read them I would try to make time as they provide great background.

I also seem to recal NASA published a lot on software quality (to this day the Space Shuttle is regarded as an example of probably they highest quality).

One final thought, have you read any of the ACM or IEEE journals?

Answer 7

http://www.gilb.com/Requirements

Lots of free books to download. I can't do requirements without Gilb style tagging, these days.

Answer 8

Actually, what you are describing is not that bad. You probably already have the requirements written in a functional document already. If you use a requirements tool like Doors then you get those 2 for free.

Assuming that you have to sell off your requirements to a customer then mapping the requirements to Test Cases is necessary. Otherwise, how do you prove the system does what it is required to do and each requirement is verified?

Mapping requirements to Design sections (and worse...code files) has always been a mystery too me. Only on very rare occasions does it add value (such as timing constraints) but the mere fact that you can prove the requirements are met by the Test Cases makes this mapping nothing more than a lot of overhead. One can argue that it helps the designers, but IMO the overhead costs much more than it saves and most of the mappings at this level end up being very arbirtrary anyways.

Out of the 4 sections you described, the only one that requires an inordinate amount of time to maintain is the mapping to design sections.

Thus, I know it is easier said than done, but try to get your company process changed so you aren't required to do the design section mapping. If you can't get it changed at the company level, then you can change the Software Development Plan for the project to specify that you won't be performing that particular mapping.

Answer 9

Hi,

I hope following notes will help you:

1. Systematic and document intensive handling of requirements is commercially useful if project is considered big and/or project is executed in very formal environment and/or project status must be known at very granular level and/or changes must be controlled finely.
2. Tools should be used for handling such scenario. Check RequisitePro from IBM Rational or other similar tools. From IBM's rational web pages, you should find few good white papers and articles. ReqPro supports creating and maintaining RTMs. Note that ReqPro is capable to handle any requirements model, not just use case model as popularly known. Btw, I am not affiliated with IBM. Essentially, you should look at tools if you want to maintain RTMs.

Answer 10

Hi, we have just release a product that provides traceability matrix analysis by linking JIRA to test cases (unit and manual/functional). Trace Analyst is a project management tool that provides traceability information about a project. It also produces burn-down charts about the project's progress by comparing estimated test cases against passing test cases. You can find out more

Answer 11

One tool I have used in the past is Enterprise Architect (http://www.sparxsystems.com.au/) and it's great at providing tracibility between requirements and design. I second others who have stated that if upkeeping the RTM becomes a chore, then you really need a new process. RTM should be a tool and if its just administrative then IMHO it's a waste of valuable time.