Estimating Integration times, is it really 3x?

Question

Hi

I read in a mythical man month that integration takes 3 times the amount of time it took to develop the individual components.

What has you guys experienced?

Answer 1

That was referring to the bad old days when they built all the software components separately and then tried to put them all together. Smart people don't work like that anymore - they integrate continuously.

Answer 2

Yes and no. Integration effort is probably still 3x, but now it's amortized over the whole development process (e.g. early integration, integration tests (esp. in TDD), etc.)

We still have to do the work but it doesn't catch us by surprise anymore.

Answer 3

Integration time depends on a few of factors: the size of the project, communication between teams, and your integration philosophy.

Small projects take less time to integrate. Large, very large, or huge projects will take more time to integrate. I've been on small projects where integration was minimal. I've been on huge projects spread across multiple component teams where integration took a very long time.

Time to integrate also depends on how well you project communication across teams. If your teams are not communicating it can take 3x or more time to integrate and work out all the related bugs.

Continuous integration helps with the perception that integration takes less time. With CI the time to integrate is amortized over the life of the project. But again if you have a poor relationship with the other component teams to total time for all integrations will take non-zero time.

CI is definitely better than the alternative. Waiting until late in the development cycle to integrate is bound to cause you much pain. The earlier you begin the integration the more comfortable each team becomes with the process.

Answer 4

Kind of a side note, there is an interesting talk given by Juval Lowey regarding this. If you have many tiny components, then you increase effort to integrate. If you have only a few large components, you decrease integration, but you have more complex components to maintain. So your effort to integrate is dependent on the architecture and where it balances the number of components to their complexity. A good balance is key, because if you drift too one side or the other, the effort required increases exponentially. If you have 20 components and add a 21st, it's not just 5% (1/20) more complex, because you have to consider the interactions of the potentially other 20 components. So adding one component adds the potential for 20 ways to interact with the existing components. Of course a good design will limit this interaction to as few components as possible, but that was basically why Juval felt it was exponential.

Answer 5

Neil and Markus' answers are spot on. In Windows, we integrate continuously. Our source code control system (we call it "Source Depot") is hierarchical with "WinMAIN" being at the top, and a tree of branches underneath it that general correspond to the organizational structure.

In general, changes flow up and down this tree in a process of "forward integrations" and "reverse integrations". This happens very regularly - almost every day for the top level branches under winmain, and a bit less often for the lower level branches. Each major branch builds fully every day in at least four flavors. The top level branches build in six flavors. A flavor is something like x86/fre, or x64/chk. We also get some daily pseudo localized builds as well.

By "build" I mean build a fully installable Windows client product from source. This happens several hundred times per day.

This works well for us - there are two important goals here:

1. maintaining good code flow (we call it velocity) up and down the tree. The idea is that any branch is never too different from WinMAIN.
2. Catching integration errors as early as possible.

Markus is very correct that this amortizes integration costs over the life of the project. For us, this makes the costs a LOT lower than they would be if we deferred costs more toward the ends of cycles. This is how things used to work. Not to air too much dirty laundry, but when I started in Windows about 5 1/2 years ago, getting a build out took a very, very long time. They now happen every day, like clock work. The Windows Engineering Tools and Release group (WETR) gets all the credit for this.

As many others have suggested in various forums - regular integration and full daily automated builds are essential for most projects.

Note, daily and regular automated test are a whole other topic (its a massive effort as you can imagine).

Answer 6

I would agree, if not higher. Though it really depends on the integration touch points.

I was involved on a project to carry out integration of a number of modules between Siebel and SAP. While both of these products have integration modules available, all the problems on the project (and there were many) were involved in the integration.

It wasn't helped by the fact that the majority of SAP that we were using was in German, and the messages being transfered were in different XML encoding formats (UTF8 / UTF16).

Once we'd got to grips with the intricacies of what SAP wanted to send and recieve, the whole project moved along much quicker.

Key things for a successfull integration project:

• Good documentation (in English!) on the integration modules
• Good documentation on the message formats
• Good project management

The project management bit is important as they supply the pizza, and do show some understanding when you have been working 30 hours straight to get an account name from one textbox on one machine to appear in another textbox on another machine.

Our project lasted over a year. The rest of the configuration of Siebel that we did, which was alot was only a couple of months,

So Integration - 10 months+, rest of the config 2 months.

Answer 7

If you have an integration phase towards the end of your project on you project plan you are doomed and x3 is not too bad.

You should rather go for continuous integration, where you have, say every 2 weeks, a release with some integration before it.