Using SCRUM in a company that imposes strict governance on the development process?

Question

Our company has to adhere to strict SOX/Cobit protocols. This means that there's a huge paper trail behind everything we do, and noone can start coding until the proper documentation has been signed off. That means you have to wait for the Change Spec, Software Spec, and Architecture Documents (at least) to be signed off before you can start coding. The same process follows any enhancements (which is a natural part of software development). Clearly this doesn't gel well with the idea behind SCRUM.

Currently we follow the SDLC as defined in the Rational Unified Process (RUP). It also encompasses an iterative process (Inception/Elaboration/Construction/Transition/Repeat), but we want to try to replace that process with SCRUM.

Is it feasible for a company with strict governance to switch to SCRUM? Does anyone have a case study that shows how it's been implemented successfully in a similar environment?

Answer 1

I've been successful in projects in that environment by doing things a bit more covertly. In projects I was managing, in environments like this, I usually try and snag an intern to use full time. The intern's job was to fill out all the beaurocratic paperwork and garbage, giving the illusion we were following one process, while in the background keeping my developers working and moving forward on the project. Internally, I used a strict scrum based methodology. My projects ended up with a 100% success rate, while the corporate average for successful projects was somewhere around 20 - 30%.

Use my messed up opinions at your own risk.

As far as SOx compliance BS, haven't they thrown that away yet?

Answer 2

I could not see why you believe that the documentation and formal process don't work well with the Agile.

If you know how to do RUP properly it is even easier for you to do SCRUM, as you already have enough discipline to be agile without being cowboy.

Nobody said that you cannot do proper documentation when you do SCRUM. These documents DO add some value - they just are not exactly necessary when you follow Agile process and so most team don't do them.

If you consider the documents to be your deliverables, add them to the iteration scope, write important things to them so they are useful to you team - they are an overhead but they also have value especially on a long-term project, or when you have a big and/or distributed team.

RUP iterations do not impose a Waterfall model - they are just guidelines for the emphasis on certain disciplines within them but a proper iteration in RUP has all the activities mixed in. E.g. Consider the Elaboration phase to be a normal Agile phase with some spikes and you will not be able to tell the difference.

The way it worked for us with the documents was to use Wiki for information like this and export its content to the Word files which could be later signed off.

The documents could pass the review but they don't have to be maintained and the team go to Wiki for their information.

Answer 3

I have worked for a company teetering on the edge of SOX madness while still maintaining a fairly agile development process. Having knowledgeable managers who were willing to work around the paperwork helped a lot. We did several things to convince the SOX auditors that our electronic processes were not just replacements for but better than their paperwork.

• We locked down access to the code repository using a special feature of SSH so that only the administrator could get at the raw files. This prevented any developers from doing any hanky panky repository manipulation to hide a change. While we weren't worried about that happening, it made the SOX auditors happy and they accepted the version control system as a secure change log rather than making us submit (no joke) diffs on paper.

• We delegated the majority of the paperwork to the QA department who were used to doing repetitive tasks by hand. :)

• The security in our online task manager was increased. Every change had to have a task associated with it. Tasks in our task manager could not be worked on until "signed off" within the system by one of a handful of managers. In practice, this meant IM'ing a manager to tick a box in a web form.

• We moved the authority to push changes from staging to production to the QA department. Again, in practice this meant telling someone in QA to run a script. As a side effect it helped make our production push process much faster and simpler.

• By using development, staging and production branches we could work freely on the development and staging branches, since they didn't have access to production databases, only requiring all the paperwork to be in place for changes which reached production.

It's worth noting that while there was some things we did under the table to get our jobs done, for the most part we made the folks in charge of SOX compliance know how much of an impact it was having on getting work done and worked with them to balance their SOX paranoia and our efficiency. We found out what they really wanted and offered more efficient alternatives (like the secured version control rather than paper diffs). Rather than working around a broken system, and creating a battleground between the developers and QA, we worked to improve the system.