How to adapt agile to different companies? An MBA thesis

Question

My master's thesis is to look at how to apply agile.

There is an awful lot of corporate selling of agile - lots of management consultants selling their brand as 'best'.

I'm not interested whether XP, Scrum, Crystal Clear, Agile-CMMI, Six Sigma or any other brand/variant is best. I'm interested in what real, active developers (i.e. you guys) actually apply as agile.

What I've investigated is how to tailor agile to different organisational requirements.

From research into how different organisations apply agile I've developed the following guidelines - a recipe for what agile variations should be applied in what situations:

• Larger and more distributed or more flexible teams need stricter coding and testing standards, small teams can (and should) use less.
• Process documentation should be minimal, real-time and current.
• Detailed statistical control indicators are an unnecessary overhead: early release of incomplete software is a better indication of progress.
• Ideally developers should be close to the customer with no specialised intermediate roles. Additional roles should only be used if customers are specialised in a way that stops developers from also being users.
• Iterations should be flexible unless it benefits coordination of releases with other departments or other processes.
• Developers should be able to easily and regularly communicate but meetings should be infrequent (monthly and weekly, rather than daily).
• Pair programming should only be used for training and investigational tasks.
• These guidelines are a starting point only: continuous improvement should be used to further tailor the agile variant to the exact circumstances.

These factors change when applied in an organisation with existing traditional (i.e. BDUF or waterfall) models, where agile teams must either coexist with or be adapted from teams using non-agile methods:

• Process documentation with sign off and structured steps will help other teams track the project.
• Statistical indicators (like velocity) can help reassure non-agile teams that the process is under control.
• Fixed iterations will help co-ordination across teams.

These additional guidelines will help agile co-exist with traditional models, but they provide additional overhead and restrictions.

What I want to know is what you - the people who write software, not agile consultants - think of this framework.

What do you think is accurate? What do you think is wrong? What would you change? What have I missed?

Most importantly: why?

---

I've added a bounty to this to offer an extra incentive to answer what is a rather long question. The bounty will go to whoever gets the most votes from the SO community - I realise that there's no single right answer, but I'm interested in what's closest to the community's consensus.

Answer 1

Oh, boy. Good luck.

I like your approach, especially getting input from us grunts rather than the gold-watch powerpoint set.

I would say that any such approach needs to be adaptable to the specific circumstances. Nobody should do something only because somebody else said they should. Thinking for oneself should always be at the forefront, IMHO.

It's been a while since I worked with Sloan School MBAs, but I got the clear sense that they had been taught that programmers were to be "managed", rather than "cultivated". That we were something of a disagreeable commodity, rather than full team members. I hope you have a better experience than that.

Answer 2

The last two points I have a small problem with. Daily stand up meetings are very beneficial. It lets us go over what we worked on the previous day, and what we intend to work on until the next meeting (tomorrow). It is important to note that daily standup meetings should be kept short and to the point. We got into a situation where some people enjoyed bringing up all sorts of questions about the project, so we decided that no questions can be asked, only productivity statements. If there are questions, a further meeting is scheduled with the person in charge of that component.

Also, pair programming should not be used JUST for training and investigational tasks. Pair programming has many benefits, such as knowledge sharing/transfer and code ownership. Two minds on one problem can bring out many interesting points of views and can help isolate potential design flaws. In my opinion, pair programming is underrated and most selfish programmers do not like pair programming. It is a benefit to the project and the team, not to one's self. Good software is written by collaborative teams, not one nerd.

Answer 3

• Larger and more distributed or more flexible teams need stricter coding and testing standards, small teams can (and should) use less.
• Process documentation should be minimal, real-time and current.
• Detailed statistical control indicators are an unnecessary overhead: early release of incomplete software is a better indication of progress.
• Ideally developers should be close to the customer with no specialised intermediate roles. Additional roles should only be used if customers are specialised in a way that stops developers from also being users.
• Iterations should be flexible unless it benefits coordination of releases with other departments or other processes.
• Developers should be able to easily and regularly communicate but meetings should be infrequent (monthly and weekly, rather than daily).
• Pair programming should only be used for training and investigational tasks.
• These guidelines are a starting point only: continuous improvement should be used to further tailor the agile variant to the exact circumstances.

Coding / Testing standards IMO need to be implemented regardless of the size and distribution of the team. Having coding / testing standards leads to more managable / stable code.

I agree with documentation. By using some Clean Code practices such as meaningfull comments and intent revealing naming conventions in your code then you remove some of the need for documentation of the code itself. Documentation should be at the business level and I prefer it to be in the form of Acceptance Tests.

While having developers close to the customer improves with the iteration process, you need to protect the developer from the business circomventing the process by coming directly to the developer for additions/changes/scope creep. The Business still needs to follow the process through backlog grooming/prioritization etc.

By using release iterations along with development iterations you can maintain a flexible schedule of your iterations that works for the team. Currently we work on 1 week sprint iterations with a release sprint every 3 to 4 weeks.

The type of the meeting should dictate the frequency of the meeting. Daily standups need to be daily. They provide accountablity and transparency to the team which is vital in having a successful team. Retrospectives need to happen at the end of each iteration and that frequency is dictated by the size of your iteration. Other meetings such as code reviews, demo's should never be dictated on time but rather on need and completion.

Pair programming should never be nailed down to specific types. We do pair programming with our QA Testers, our BA team so that both sides have a better understanding of the UAT's and Stories. We also do pair programming for knowledge share, prototyping, investigational tasks etc. In our environment pair programming has become second nature.

Continuous improvement in Agile will become second hand nature as you learn to modify the practices of Agile to your business needs. So long as you don't stray from the Manifesto your Agile should be successful.

Answer 4

We practice agile in a large company with highly integrated software and pre-determined windows when software can be deploy to production on shared hardware environments.

We developed a set of agile practices composed of SCRUM (project management) practices and XP (engineering) practices. Many of the systems we deploy with make use of traditional waterfall processes

Regarding your framework I will reply to each item:

"Larger and more distributed or more flexible teams need stricter coding and testing standards, small teams can (and should) use less."

If by coding and testing standards you mean engineering practices (XP) such as continuous integration, pair programming, test driven development, automated functional and performance testing etc. we employee all the same practices regardless of the size of the team or project. As a result, we often release software with zero defects found during User Acceptance Test. Releasing high quality software (low defects) that remains malleable to enable continuous development at a sustainable pace drives the need for engineering practices, not the size of the organization.

"Process documentation should be minimal, real-time and current."

If by process documentation you mean the agile process documentation, all of our fits on a wall. We show the manifesto, the mapping to the 12 principles and finally the 21 practices we employee. The fact that they fit on the wall and they are highly visible enables the team to understand them and more importantly actually put them into practice.

"Detailed statistical control indicators are an unnecessary overhead: early release of incomplete software is a better indication of progress."

Percent completes on traditional work work artifacts carry little value. Demoing working software to our product owner/ business partner every two week is the best indicator of progress. Tracking actual velocity (units of story cards completed) and defect rates and displaying the data on big visible charts is very helpful. When introducing certain practices to a team it is useful to show progress. For example, the number of unit test written before the code when learning TDD.

"Ideally developers should be close to the customer with no specialised intermediate roles. Additional roles should only be used if customers are specialised in a way that stops developers from also being users."

Users, if possible, should work with the team in the open workspace. If it is not possible for the user to be with the team then a user proxy is the next best person to be on the line. No one can give better feedback than the people that will actually use the software to do their jobs.

"Iterations should be flexible unless it benefits coordination of releases with other departments or other processes."

More importantly, release dates should be alined across teams. Iterations are best if kept consistent in duration (we use every 2 weeks). Consistent length of iteration is consistent with time boxing and the calculation of velocity used to commit to story cards for the next iteration.

"Developers should be able to easily and regularly communicate but meetings should be infrequent (monthly and weekly, rather than daily)."

The team should participate in the daily stand up meeting, and every iteration the iteration planning meeting, the show & tell meeting, planning poker meeting and the retrospective meeting. Every release the team participates in the release planning meeting. Given that the user/ business partner works with the line, daily conversation can occur about the work being completed.

"Pair programming should only be used for training and investigational tasks."

The first reason to use Pair Programming is to eliminate defects. I have written a number of response regarding pair programming. To summarize, pairs less likely to become blocked, less likely to take email or web vacations, provides mechanism to transfer business domain, application domain and engineering practice skills, eliminates silos of knowledge (key in larger organizations) etc.

"These guidelines are a starting point only: continuous improvement should be used to further tailor the agile variant to the exact circumstances."

Retrospectives, planned each iteration or when an event occurs that demands a retrospective be called, and a mindset of zero defects drives continuous improvement. Employee the XP engineering practices enables the team to keep the code healthy enabling new features to be readily added indefinitely. We treat deliverables from waterfall projects as constraints. To manage the constraints we request work from these teams well in advance and use engineering techniques like mocking to test the story cards that will integrate with these deliverables.

"These factors change when applied in an organisation with existing traditional (i.e. BDUF or waterfall) models, where agile teams must either coexist with or be adapted from teams using non-agile methods:"

We have agile teams that live in a waterfall world. We invite the waterfall projects to our release and iteration planning meetings (and retrospectives). The success we have and continue to have continue to bring new converts to the agile practices within the company.

"Process documentation with sign off and structured steps will help other teams track the project."

Disagree. The story card wall, iteration plan and release plan are all that is needed by those on the team or external to the team.

"Statistical indicators (like velocity) can help reassure non-agile teams that the process is under control."

Agree. Agile teams will make visible velocity and defect rates. Budgets will be within 1 or 2 percent and releases will be on schedule since they are time boxed.

"Fixed iterations will help co-ordination across teams."

Needed in highly integrated and large environments. Also helps the team build iteration and release plans based on velocity.

Answer 5

As part of your PhD thesis, you should also consider how Agile is used across teams that are placed in different parts on the globe. So, if you have a product team on the east coast, development teams in India and Russia, QA team in Singapore and so on and so forth. This to me is a whole different ball game and needs completely different patterns.

For example, some of the patterns are:

1. Based on the time zone, pair the early risers in one part of the world with late nighters of the other part or vice-versa. This is not exactly pair programming but you can call this whatever pattern you want.

2. Emphasis must be to make the code as readable as possible as opposed to writable. What this means is we may have to put emphasis on uniform design patterns and fluent like naming.

3. Create teams in such a way that you have one of them and one of us. Which means you pair the Russsian developer with the Indian developer and they work together on a module.

Inventing your own patterns over time that make agile work is really fun and takes a lot of patience and hardwork.

Hope this new angle helps you in some way.

Answer 6

1. "Process documentation should be minimal, real-time and current."

What do yo mean by "minimal"? Is it in the sense of say sum of the number of pages or in the sense of coverage (v.gr. standards doc, configuration management manual, design doc).

When you assume a project where there is no turnover in the team - and in long projects this is not a reasonable assumption, you need an efficient mechanism for knowledge transfer, and the most efficient mechanism for knowledge transfer is documentation.

I have seen projects where documentation is kept minimal (in the sense of number of pages) because it is too cumbersome to develop the documentation. But if you can re-use documentation from one project to another with no changes or minimal changes (and this is reasonable for documents like programming standards, configuration management manual) there is no documentation development burden.

Process documentation should cover all of the software development process or you will risk to have processes with steps that are executed in an inconsisten way. Agile teams have agile software processes. And by software development processes I mean clear mechanisms for managing code, controlling versions, controlling reviews, checking in code, fixing bugs, etc.

Answer 7

• Larger and more distributed or more flexible teams need stricter coding and testing standards, small teams can (and should) use less.

Coding and testing standards should exist regardless of the team size. They improve maintainability of the code, and make it easier to bring new resources up to speed.

• Process documentation should be minimal, real-time and current.

Agreed.

• Detailed statistical control indicators are an unnecessary overhead: early release of incomplete software is a better indication of progress.

There aren't many statistical control indicators for software development that are consistently valuable. I would be looking for variance to time and budget, defect rate in test and in production, and variance to estimates.

• Ideally developers should be close to the customer with no specialised intermediate roles. Additional roles should only be used if customers are specialised in a way that stops developers from also being users.

So often this is impractical or impossible. Customers have so much to do in their own jobs that they rarely have the time to be extremely hands on with development staff. Business Analysts have the skills necessary to consolidate business questions and get clear answers much more time effectively. If your release cycles are short enough, customers will have plenty of opportunity for feedback.

• Iterations should be flexible unless it benefits coordination of releases with other departments or other processes.

There should be some flexibility, but not much. One of the benefits of an agile approach is that it forces the team to limit scope to the most valuable requirements. Adding flexibility to iteration timing introduces increased risk of scope creep.

• Developers should be able to easily and regularly communicate but meetings should be infrequent (monthly and weekly, rather than daily).

Daily meetings are effective with larger teams. They keep people on track and increase collaboration. They help prevent team members from getting stuck on a problem without asking for help. They also help maintain control around the iteration, which is very useful considering the lack of other controls available.

• Pair programming should only be used for training and investigational tasks.

I don't have much to say about this one.

• These guidelines are a starting point only: continuous improvement should be used to further tailor the agile variant to the exact circumstances.**

EXACTLY! Agile is meant to flex to the organization's needs. Constant tweaking is essential to perfecting the process.

Good luck to you on your thesis.

Answer 8

• Larger and more distributed or more flexible teams need stricter coding and testing standards, small teams can (and should) use less.

Continuous testing is invaluable regardless of team size. What should vary is the amount of testing based on project maturity and component criticality.

For new products periodic demonstrations are great for morale and are adequate for delivering progress feel to management.

For mature and shipping products at minimum periodic acceptance testing of major features will track a product's readiness for release.

For major system interfaces unit testing is illustrative and helps avoid surprises. When working with contracted sub-projects unit tests are better than documentation and critical to avoid finger-pointing delays.

Test-first development is a worthy experiment for the heavy lifting pieces of code. If your most valuable asset is your technology then it should be extremely thoroughly tested.

• Detailed statistical control indicators are an unnecessary overhead: early release of incomplete software is a better indication of progress.

Statistical control indicators are vital to early identification of project overrun, but they are best left in the hands of the "Agile Coach" or "Scrum Master" either permanently or until the entire team becomes disciplined at producing accurate accounting of progress and delays.

When effectively applied, agile development practices should produce stories and tasks which are small enough that they are completed in hours to days. Effective management of the "planning game" and feedback resulting in individual task overrun can minimize the overhead of estimation and collecting progress data.

Finally, a historical record of customer (and/or management) decisions and an accounting of where time (days) was actually spent in reaction to these decisions will lead the way to improving both engineering and business process. A "swim lane" or "gantt" chart of blocked development and time spent will show how "but we're agile" is no excuse for rash business decisions.

• Ideally developers should be close to the customer with no specialised intermediate roles. Additional roles should only be used if customers are specialised in a way that stops developers from also being users.

For once I agree! Separate dev and management meetings work well to separate concerns.

• Iterations should be flexible unless it benefits coordination of releases with other departments or other processes.

Agile iterations are a valuable development rhythm for establishing a continuously releasable product. The ends of iterations are when feature decisions can be implemented and source control changes can be made to branch off features which won't make the target.

However this practice may be difficult for teams to implement when iterations to not meet up with business needs. Iterations should be scheduled around business needs like demos and promotions.

• Developers should be able to easily and regularly communicate but meetings should be infrequent (monthly and weekly, rather than daily).

Monday / Thursday standup meetings are a great idea. However it needs a hard 15-30 minute cap. Split the meeting as needed to reduce the time. Separate concerns to avoid wasting developer time.

An office dog can help keep people attentive during meetings. A few barks is better than people nodding off or texting.

• Pair programming should only be used for training and investigational tasks.

Pair programming, unit testing, merciless refactoring, and test driven practices are some of the most difficult agile skills for teams for adopt. The benefits of pair programming are to increase awareness and knowledge and prevent work from being done in isolation where bad decisions and assumptions can linger and cascade.