How to educate a development manager about the difficulties of software design?

Question

I have had a few development managers who don't seem to understand or appreciate the difficulties of software design and implementation.

Such managers believe that processes and methodologies completely solve the problem and I have a tough time explaining to them that it is not so and that you cannot read a book on the latest process fad and hope to get results by applying them as is.

The latest frustration I have is to convince my manager to (a) Give me requirements not piece-meal but a larger set as far as possible. (b) Give my team lead time to think about how to design, thrash out a few alternatives, work out an implementation sketch, to plan out the tasks etc.

The frustrations are compounded because of Agile methodology and the interpretation of it that says not to do up-front design (as against BIG up-front design in Waterfall), product owner can change requirements at any time and so son.

So far I haven't had much success and have to put up with the resulting frustrations. Can you give me some arguments that can convince such managers?

EDIT-1: Retrospectives are done, though not always at the end of every sprint, and the problems are brought up. But as I mentioned, my manager doesn't appreciate the need for design lead time and the frustrations with piece-meal requirements.

EDIT-2 I don't have a problem with changing requirements. I understand that it will be so, but imagine this: You want a small feature to begin with and then you keep adding more around it. After a few iterations, the design cannot handle it anymore and a redesign (not refactoring) is required. This could have been solved better with an upfront design in the first place, had the related features been investigated together. Its not BDUF, its the natural way of doing it (what I call software engineering common sense).

My manager doesn't understand why I ask for time to redesign (a few times I just call it refactoring so that it fits the Agile way of doing it, but it really is redesign) and not developing and demoing new features.

Answer 1

Every time requirements are changed (or increased) so should

• the estimate to complete and,
• the assessment of risk

Start giving updated estimates (even if you have to guess) and lists of risks every time you get an updated or new requirement. This will help your manager make the connection.

Try to do this in a spirit of helpfulness--"for planning purposes"--so that you aren't perceived as obstructive or lacking "can-do attitude." Remember that estimates can (in theory) come down, and risks can be reduced.

Answer 2

Buy your manager this book. That's what I did, and it worked great :)

Answer 3

Business requirements are going to change no matter where you work. It's not your fault, it's not your boss's fault, it's not anybody's fault. The entire point of taking the requirements on piecemeal is to encourage you to think about the problem at hand, not some other problem that you might or might not need to solve. It's quite liberating once you get into the rhythm of it.

Think of upfront design as premature optimization. You may not need it, and even if you know you need it, you'll know more about your design two weeks from now than you know about it today. It'll help you solve your engineering problem with the best possible knowledge about the state of your code.

That having been said, edg is absolutely right. When you add more requirements, the estimate changes. This isn't the fault of the developers or anyone else; more work means more work no matter how you square it. If your boss doesn't realize that adding requirements will result in a larger estimate for the project you need to explain to him that Agile isn't a magic bullet that allows you to add more features without paying anything for them.

Answer 4

Agile Simple Design doesn't mean don't do ANY design/architecture up front. It means do the minimal design up front, so that you will not pay a horrible price for reasonable change requests.

Scott Ambler talks about Change Cases - http://www.agilemodeling.com/artifacts/changeCase.htm James Coplien talks about Agile Architecture - http://www.infoq.com/presentations/Agile-Architecture-Is-Not-Fragile-Architecture-James-Coplien-Kevlin-Henney http://blog.jaoo.dk/2009/03/04/handling-architecture-in-the-agile-world/

The art/craft in all of this is in how to slice the architecture in a way that allows:

• relatively fast convergence on overall architecture/infrastructure - on the order of days per months of estimated development time.
• developing "just enough" architecture/infrastructure per each feature/requirement
• doing the right balance of preparations for the future compared to focus on the features of today.

Its important that your Product Owner is aware of all of this balancing act as well, and you work collaboratively. He should understand that if you disregard all thinking for the future, each change will be very costly. There is a price to be paid for flexibility.

Its btw very similar to investment in QA and test automation. You pay something now, that will pay off only after X times you test the code. if the code never changes it was a waste of effort. but everyone knows that most code changes...

Answer 5

First of all this issue seems quite sensitive, so all I wrote below is just my personal opinion, and not necessarily a wise thing to do.

In my opinion you cannot make software if you do not know what problem it should solve. If requirements come in small parts that are too small to oversee the problem, then I would just fire questions about the parts that seem to be missing. Like: "okay so the software should do X, but does that also mean Y or otherwise maybe Z? Because if it is Y then ... but if it is Z then ..." Of course if the manager is in the middle of extracting the requirements then he cannot answer, but at least he knows that there are still open issues that influence development.

About no lead time for design: design and development are an iterative process that could go hand in hand. It is just how you name the thing. If the manager wants to see some code at the end of the day, okay then I would just use the first half of the day to design and the second half of the day to make some code based on that design. If the manager does not want to see the design, fine with me then I'll just show the code.