How much flexibility in business requirements is too much?

Question

I feel like I'm beating my head against a brick wall sometimes.

I was just talking to someone in my company about the need to encourage our [company's internal] client to put thought into their requirements up front and make a bigger effort to ensure that they're fixed before providing them to my development team to start on.

The argument against was essentially, "They're paying the bills so they should be able to change their mind as much as they want" and "They're paying us so we should just do what we're told".

While I recognise and agree that the client should be able to change their requirements (especially when using a Lean or Agile methodology), I also feel that there should be a point at which some (any!) fixed, approved and signed-off requirements are supplied to my team. I'm therefore trying to implement a simplified Lean Software Development process that requires the client to have fixed a set of requirements (not necessarily everything; just enough to keep my team occupied for a 3-week development & testing iteration) before work can start.

This allows me to:

• Provide a more accurate estimate
• Plan my team's resourcing (I'm the Team Lead)
• Create Work Items in TFS (or Stories if I'm using an Agile flavour)
• Write unit tests ahead of development rather than after the development is in a state where it's not going to change anymore
• Provide reports at the end of an iteration of - for example - actual effort cost vs estimated

Is the argument of "They're paying, so just do what you're told and don't argue" a reasonable one? What are the counter-arguments if not?

If the client (or internal business reps in my case) holds the view that:

• they don't care about how many times my team has to do their work over again
• they don't care if expected delivery dates need to be moved out
• they aren't concerned about the wastage incurred with all the do-overs

Do I have a reasonable basis for concern?

Answer 1

Yes, "they're paying" is a reasonable argument.

It's also your best (and only) counterargument. Increasing revenue and/or decreasing expenses should be close to the heart of any business person.

If you can make a monetary case in your own favor you'll have a better chance. You've got to be able to quantify some of the implications of their decisions.

You'll have to hope that whatever agile practices you put into place will have the desired positive effect on the monetary health of your company.

Good luck.

Answer 2

Lean is more about discipline and avoiding waste - it's not really a process. It sounds to me like the issue might be more a communication one. In other words - you don't value the decisions handed to you enough to consider them valid. Your reaction probably breeds a bit of hostility in your "client" and it turns into an ego challenge.

First thing that jumps to mind is that your "clients" aren't dumb. It may seem confounding to you, but keep in mind that unless you blow this door down you need to find a new job at a new company where the client can put things in terms you respect. Believe it or not - I'm not trying to be harsh here, I'm just suggesting that the communication isn't flowing, and it sounds to me like you're saying "I'm not getting what I need".

It's great that you're trying to improve your process, but no process will change this:

"I feel like I'm beating my head against a brick wall sometimes." and "he need to encourage our [company's internal] client to put thought into their requirements up front and make a bigger effort to ensure that they're fixed before providing them to my development team to start on."

... says to me that 1) You're frustrated and 2) you don't respect what they're doing.

I've worked their before - I hate it. And I moved on - it was my only solution. However if you like the job and the people you work with, process won't fix anything - personal skills will. This is where you break from a "dev" to something more - a pro (which you already are - I just mean a bigger one).

Get to know the business side more. Try out DDD practices and the ubiq. lang thing so you know more about what might be coming, and what might change. In this economic climate you can't have a long straight road - you'll need to adapt and change as your business does.

Good luck!

Answer 3

You need to learn a bit about "agile", and to get used to the fact that requirements change. They're meant to change, and you're meant to produce code that meets the changed requirements.

You are not ever meant to be "done". That's an old idea that stems from the false idea that it's possible to know the requirements ahead of time, accurately. The best it has ever been possible to do is to know all the requirements as of a particular point in time. Of course, by the time those requirements have been implemented, the original requirements will have changed, as the business changes, or as the world changes.

"Done" is an illusion.

Answer 4

What it sounds like you need is to adopt evolutionary prototyping. However, to help counter some of what you're feeling when they make yet another change request you should say "Yes but it will delay the project by X days/weeks/months and increase costs Y amount." If they're happy to pay, be happy to work. "They're paying" is a valid reason.

Answer 5

You can work on that issue on two fronts.

One is internal. Switching to the approach which expects many changes in requirements all over the time should reduce your (and your team's) frustration a bit. Now you're trying to fill the work 3 weeks ahead. Some teams work with Scrum using 1-week iterations which make planning more frequent (weekly) but also reduces chance that something will change during iteration. You can even go further and try Kanban, where you change priorities all over the time and still from engineer point of view that's more of an order than a chaos.

Another thing is working with business people. It they really don't care they're burning money it can be hard, but even then you can try to set up some techniques which isolate their ever-changing decision from the team (e.g. making change management pretty formal or engaging them to organize priorities of whole backlog every time you plan work for another period of time).

Answer 6

I thought within Agile, especially the Scrum method, that the content an iteration or sprint was decided upon at the start, and then the team would deliver that content in the allowed time (your example of 3 weeks is a good one), THEN you review and adjust for changes in the next sprint.

No one likes to use the words "fixed" or "signed-off" to describe how a sprint works, because it sounds too old school waterfall, but agreement on sprint content (such as a set of of User Stories from the Product Backlog) is essentially the same thing for each sprint, just done more often than in non-agile projects.

If you can't get even that kind of agreement over a few weeks, then your client is indeed burning money for no good reason except a tax write-off.

Answer 7

"They're paying, so just do what you're told and don't argue" is reasonable but leads to one wanting mindless work as, "To avoid arguing, you're going to have to do all my thinking. Are you OK with that?" See what kind of response that gets. It is escalating things in a sense but sometimes a bad "What if" game needs to be played out completely.

There is likely some middle ground where one can say, "This is what we want now," and that is what gets done and then improvements are suggested. Changing wants is part of the process as the only way to find out what you don't know is usually the hard way.

A counter-argument is that the team should have a sense of pride in their work and if what gets delivered is considered crap over and over again, at some point this will burn out most people. Do you mind having high turnover? Some places are ok with burning people out and others not so much. How would you like to run in circles or on a big hamster wheel for a day or two? Doesn't that sound like fun? ;)

You have a reasonable basis for concern, but the key is to have some flexibility that things will change so that while the first deliverable may work as requested, that is no longer valid and so something else has to be done instead. I'm reminded of what a former team lead used to tell us: "90% of your code will probably be throw out. It's just the way this works, accept it." As sad as that may sound in terms of a lot of work with little good results, it does seem to be rather true most of the time.

One little point to add about that pride is that Dale Carnegie's book, "How to Win Friends and Influence People," has as one of its principles under "Be a Leader: How to Change People Without Giving Offense or Arousing Resentment:"

Give them a fine reputation to live up to.

There are a few stories in the book illustrating that.

Answer 8

One of the things that has helped us with this problem is at the beginning of the iteration we all sit together and go over what we are going to work on. Developers, client, QA, etc. We talk it over and estimate every piece that we are working on. This tends get the client thinking and reduces the number of changes that come later on.

Other than that, keep the iterations short and except to make changes after every iteration. This is the idea of Agile/Lean Development.

Answer 9

One thing that I have found reduces the amount of change in the requirements is that every time they change or add a new requirement, you change the deadline and the cost. When you make changes without a pushback as to how that changes the deadlines and cost, then they will want the world, when they realize there is a price attached to all those changes, then they will generally be much more reasonable. If the intital requirements are questionable, do not provide a time estimate or hour estimate until you think the major questions you have are answered.

Answer 10

EDIT - Since you say they don't care about pushing delivery forward, then perhaps you must recognize you are not in a development project at all: you are doing research. There is no hard deadline there and research has different goals. Generally the goal is not the working software, but merely an insight, or knowledge gained.

So, if you measure "insight/knowledge gained" instead of "production ready software developed" you may be doing great and enjoy yourself tremendously.

If you don't enjoy research, no sweat: just propose to take the very uncertain stuff out of your software development project and set up a prototyping research project for it, with explicit goals.

Answer 11

Thanks for the great question! Since your dev process iterations are already 3 weeks long the solution would probably not involve improving risk management (for instance, pushing requirements that are likely to change into the end of the development cycle) or expectations management (making your team aware that changes are inevitable). Instead I’d suggest a few things:

• Putting up higher formal barriers for any requirements change: forms, approvals, impact analysis, change management process.

• Creating an up to date road-map together with the business users to give the development direction and focus. This will also give a way for various users to resolve their priorities and reach an agreement on how the limited resources are spent.

• Finding the courage to say “no, not in this release, and probably not in the next either” and stand your ground.

The latter, the courage not to give in and agree under an intense pressure to the changes that will put the project and the software product into jeopardy in the long run is one of the toughest parts of the software development project manager job. More so because often you won’t have any hard data, the impact of the change might seem uncertain and often negligible. Unlike products in construction or mechanical engineering trades, software is inherently malleable and making a little change might seem as no big deal to everyone and it might not be a big deal, but then another one, and yet another one on top and soon you lose track of the original design.

Developers are often critical of their management who allow the requirements change through late in the project. However, the stakeholder pressure is intense and often PM feels, even if it’s just implied, as if it’s their job or career is at stake, on one side of the scales and just a vague gut feeling on another.

Not saying that there shouldn’t be any change allowed at all, there is a seamy side, sometimes change is crucial for success. And it is all about knowing when to give in and when to firmly stand your ground.

One of key management tasks is to design the development process and designing the software development process is akin to designing the software itself: it’s all about finding good trade-offs between conflicting requirements and, at the same time, staying within constraints:

• Programmers prefer having as much information about the task at hand and as early as possible. The requirements are needed to create and validate software design. And the late changes are disliked for a reason: they force programmers to re-evaluate big chunks of the design and go through the validation process again. This requires a lot extra time and care on their part.

• Development managers prefer to plan based on stable requirements that are well known up front. Most planning techniques assume a great degree of certainty, at least in the short to medium term.

• Business users, would love to have zero lead time for implementation of changes required, they’d like and often would have a genuine need for flexibility to change the requirements as late as possible. After all this creates a huge advantage over competitors. However, regardless business wants to be able to maintain or even increase the rate of software change without increasing the resource; the business also assumes that any software solution delivered is going to be robust.

But all three groups are often oblivious of each other needs:

• Programmers don’t have the knowledge of the business urgency, window of opportunity, completive advantage or compliance threat the change will address.

• Development managers time and again seriously underestimate the effort required for the re-design; or just lack the courage to stop business users from shooting themselves into a foot.

• Business users are just unaware of the lead time required to deliver a piece of software, the development process is completely opaque from where they are, it just seems that things start taking longer and longer to happen and the end product is a complete mess.

Some tips:

1. Always do your best to have a clear and up to date overview of priorities and outstanding work, failing that you’ll lose any requirements negotiation before it even starts — you’ll have no facts to argue your position or re-negotiate the schedule, re-shuffle features and will have no other choice but simply succumb to the pressure and promise the impossible. Hence have:

   • An up to date schedule.

   • Clear descriptions (or specs) and estimates (even if it’s an order of magnitude) for all outstanding tasks.

   • All outstanding tasks in writing, stored in one place.

2. On the outset of a project identify the requirements that might change:

   • Schedule them as late as possible

   • Set dev team expectations; explain the way the business thinks and any existing constraints

   • Aim at providing a more generic functionality, seriously weight excluding from the software anything that is bound to change and can be easily handled in other ways.

   • Look out for political requirements, i.e. requirements that do not have any other sound explanation.

3. Establish a clear change managements process:

   • Formal process will make changing requirements more expensive, hence many whims just won’t make into your list.

   • Have the courage to say “no”, when you think the requirement will endanger the project.

   • Avoid going over the top with the formal process, it’s just a tool; don’t risk completely losing touch with the end users and stakeholders.

Answer 12

Regarding:

• They're paying us so we should just do what we're told

That's all fine to start with. But longer term if a client isn't persuaded to have some kind of process, they're going to see quality and productivity suffer and almost certainly blame you for it.

A software company's job is not to mindlessly create what a client asks for - that's what cheap Asian outsource teams provide - but to help the client figure out what they need, and in some cases tell them what they want.

An architect doesn't expect his client to tell him exactly where every window goes, it's his job to take their vision and produce something useful and real.