Staying Agile in a waterfall

Question

We have a large enterprise application where projects are scoped designed and finally coded using a formalized waterfall process. I am often given code changes for non related initiatives just because they are in the same section of code. All initiatives must be turned over at the same time. Development team also has little say over scope or delivery time line. We are not able to talk to the users we must go through a group of requirement gathers who don't know the business.

Does any one have any advice on how to implement even the smallest of agile techniques to such a entrenched environment.

Answer 1

At least you could start writing unit tests, or even - as much as circumstances allow - Test Driven Development yourself (possibly spreading the ideas among your fellow co-developers too). You can change a lot without management even noticing anything ;-)

Of course, in an average or better place, people in the management are not completely stupid. Over time, when you have managed to raise awareness about these issues among the development team, you (as the team, collectively) can talk to upper management too, and convince them to take steps toward the right direction. Start small, get concrete results, and build on them - and build leverage by finding allies both in the dev team and (as much as possible) among management and users.

Very often certain processes are followed only because "we always used to do it like this". If you can show people that there are better ways - and prove it with convincing arguments - you have a good chance of succeeding. Note that management and users need quite different arguments than developers. You can try making rough cost-benefit calculations (or google - I am pretty sure there are lots of stuff on the net about these). I also remember there is good material about this in Kent Beck's first XP book. You can also collect bug statistics which over time (hopefully) show that features developed the agile way have noticeably less defects in later (integration test or production) phases. (For this you need a defect tracking system, if you don't already have one.)

Another useful tool is asking questions. If you something - a document, a way of doing things - you don't understand, dare to ask questions:

• Why are we doing this like we do?
• Is there a better way?
• Do we actually need to this thing at all?
• Who needs this document?
• What information does she actually need out of it?
• Does it contain the right info for her?
• Is it up to date?
• Who updates it?

Often people just take things as "given", but when you start asking for causes, you may find lots of interesting things... and ideas for improvement.

A very useful agile tool is retrospectives. After each iteration (whatever it is called in the actual process) the team gets together and brainstorms about

• what went wrong in this iteration, and how to make sure it doesn't happen again (or at least improve it to some extent)
• what went nicely and how to ensure it will continue like that

This may be fairly easy to get accepted by management, and is a good way to start positive changes. The most important thing is to wake up and activate people - to make everyone realize that the success or failure of the project is (at least to some extent) in their own hands, that they can do something to improve the situation.

Answer 2

Agile is all about breaking down those waterfall walls. BDUF; simultaneous, multi-component release; lack of communication between the developers and the business process owners; planned iterations - these all stand in your way in the waterfall process.

At my position, we have broken down a lot of these walls - and it started with getting direct access to the customer. When that happened, the customer got a better product. That led to happier customers. That drove away things like BDUF, etc.

We still don't quite have true iterations/sprints, continuous integration, etc., but we're getting there. (Old habits, and all that.)

Answer 3

You as the development team could still coordinate internally using agile methods (test driven development, pair programming, story cards, CI, common language, etc.)

Agility in my mind is about being able to have confidence in changes to the software and preventing large misinvestments in features which nobody needs three steps down the waterfall route. Testing and refactoring and avoiding over-engineering are key here.

Answer 4

Depending on your domain, automated testing and continuous integration should be doable.

Also, consider coming up with your own, highly granular burndown list (inchstones) for your currently assigned tasks. It should help your work estimates be more predictable and make it easier for you to explain any schedule slippage and unplanned tasks.

In general, keep track of some metrics in your system. If you can show some Agile technique adds value (shortened cycle time, lowered defect rate, etc.), it should be easy to sell your leadership on that technique.

...depending on the manager, you may want to avoid actually using the word "Agile" though, especially if you're just looking for a small victory on using one technique.

Answer 5

To me the problem doesn't seem to be that you're using Waterfall rather than Agile. It's that your waterfall implementation has big problems. Most obviously:

Requirement gathers who don't know the business

Waterfall can and does work well if done properly. I think it sounds like some of the people involved and how they do things are wrong, not the conceptual process.

Answer 6

In my experience, large enterprises are concerned with RISK, PREDICTABILITY, and MEASURABLE RESULTS. You'll have an easier (though maybe not easy) time introducing Agile if show how it aligns with those metrics better than existing practices.

1. Make it possible to ship often, even if you don't do it yet: Leverage CI tools and automated build scripts to build and package your application. This way you're primed to capitalize on any opportunity to incrementally release new code that might come up.

2. Measure your productivity now so that you have a baseline: The more you can measure the better.

   1. Average # of programmer hours per "feature".
   2. Average length of time between code checkin and the discovery of defects against that code.
   3. Average length of time between defect discovery and defect resolution in production.
   4. Average amount of time needed to identify, resolve, and deploy defect fixes.
   5. etc.

3. Project the changes in these metrics under an Agile process: For instance, in most cases the sooner we find a bug the easier/cheaper it is to fix, so benefits from TDD and rapid releases to QA should be easy to quantify.

4. Start small: You may have a waterfall schedule handed to you, but you can still break that down into iterations, so do so. Get your engineering practices in place, then start trying to adjust process. See if you can try Agile on a small auxiliary project as a proof of concept.

5. Find a sponsor: Try and convince someone higher in the pecking order than yourself of the merits of Agile. Engage their help in framing the "Agile vs. Waterfall" arguments in terms familiar to the decision makers.

6. Be patient... it may take time to see results.

7. ... or don't. If you're deeply interested in Agile and are getting zero support, find a new job. Yes it's rewarding to effect change from the belly of the beast, but it's also rewarding to work with people that share your ideas about building software.