How possible is it to estimate time for programming projects?

Question

It seems like it is nearly impossible to get close because you could run into any number of issues and things not first anticipated. How close can we be expected to reasonably estimate? Our PM wants to be able to have things like Gant charts and such mapping out weeks at a time and such... So we say we can get these bugs done, and this is how long each will take, and the goal will be Friday, but things get thrown off and pushed into the next week, like every time! How are we suppose to guess the right time?

Answer 1

Our host Joel recommends evidence-based scheduling, which includes methods to account for inaccurate estimation, interruptions and distractions, and all the other usual suspects.

The biggest bang items:

• The person doing a given piece of work has final say on its estimate. No managers, leads or committees are allowed to overrule estimates, only re-assign work to someone else.
• The more you break tasks down, the more reliable the final estimate will be. The effect is twofold: first, over- and under-estimate errors will tend to cancel each other out more, and second, to perform the breakdown you end up thinking about the work in more detail, improving the overall accuracy.

Answer 2

It'll take 6 to 8 weeks.

Spend a couple of days planning, and break the project down to 2 or 3 day units, and then factor in time spent doing non-development activities. But anything longer than a few weeks, it's going to be a estimate based on other projects of similar difficulty.

Answer 3

This this is the best method I have ever used: http://en.wikipedia.org/wiki/Wideband_Delphi

It is still not perfect, but if done well at least gets you in the ballpark.

Answer 4

When unexpected things come up we react by changing the architecture, the design, or the code, etc. The estimate itself should evolve along with those things. It should not be a value determined at the beginning of a project and never updated when everything else can be updated and evolve during the project.

Answer 5

experience counts. You know what happened last time, and the time before that, and you know how much you can do of a particular type of task - so the estimate is really easy. Yuo pick a number out of the air based on an educated guess. As you get more experience at estimating and developing, you'll find you get better at it.

You'll never be perfect enough to satisfy a project manager and his gantt charts (sigh, such things should have fuzzy edges enabled by default) but you are estimating the time, not giving a guaranteed time.

Answer 6

For me, it is just something that comes with experience. When I was a wee lad as a programmer, it seemed like I was always underestimating the length of a project. As you said, things would come up, and the project would take longer. Now that I've got "flight time" I'm a lot better at anticipating what those "things" are. Unexpected stuff will still happen, but in general the time frames are much more accurate these days.

Answer 7

One thing is that estimation is possible only if you're working with something familiar, using familiar technologies. If you're working on something genuinely new, the time estimations certainly won't hold. Nevertheless, making a plan (breaking the project to pieces and ordering them somehow) is quite beneficial by itself, even if the time estimates are really just guesses. Your PM probably knows this; the main thing is to make some sort of plan, not that the estimates are correct.

Answer 8

I would strongly recommend "Agile estimating and planing" by Mike Cohn. It describes relative estimation based on points which is easier to do by human nature. This book also describes poker planning which is really fun.

Answer 9

I really like the book Software Estimation: Demystifying the Black Art by Steve McConnell. A couple of really important points that he focuses on:

1. Are you looking for an estimate or a target? An estimate means "How long will it take to implement these features?" A target means "How many features can I implement by this date?" They are two different things, and it's important to make sure everyone involved knows which thing you're all talking about, and not to confuse the two of them.

2. Beware of giving an estimate with too much precision, especially earlier on. Don't give single-point estimates; if your best guess at the beginning of a project is 4 weeks, estimate something like "2 to 8 weeks". It's impossible to be more precise until you work through the phases of the project (requirements development, design, etc.) that eliminate some of the uncertainty.

Answer 10

Working as a freelancer this is a constant problem to me. If I get it badly wrong I loose money.

The problem with estimating for programming is precisely that which several people have identified here - it's the unknowns and the gotchas that can blow and estimate off into never-never land.

So increasingly I what I use is a two-fold approach

A. If the problem is in a known domain on software I am totally familiar with I do a straightforward mental breakdown and give a quote. Often with clients they'll ask for a ballpark upfront when discussing the project and with these I'm usually happy to agree - I ere on the side of caution, but this is just experience building on previous projects.

B. If the problem is an unknown domain. In these case I'll try an identify what are the crux issues upfront and I will code a test case around them to 'scope' the problem. Sometimes one can persuade the client to pay for this as a 'feasibility' study but even if you can't putting in the time unpaid and scoping the problem is cheaper than quoting the client X then finding it needs nX time to solve.

Identifying which problems fall into A or B, and identifying what are the crux items in B, is down to experience, but I think the general rule is universally applicable: always isolate the crux issues up front and deal with these first. Once they are know then more conventional techniques can be used with some confidence.

Answer 11

Here's two estimating methods I've used in the past. They work, but I don't recommend them.

1. Estimate how long you think it should take, and then double it. At the outset of each project, I made it clear that changes would increase the amount of time. I would also remind of this every time a change was requested. It's very much a fly-by-the-seat of your pants method of estimation, but it was impossible to establish anything realistic because the company didn't believe in methodologies or meetings.

2. It'll be done when it's done. This usually only works for spare time freebie projects, but I managed to convince a former manager of it when he got wise to the double-your-guess method. I think the only reason it worked is because the expected benefit of the project was enormous and need for the solution was deemed critical. Their only alternative was to buy off-the-shelf software, which they refused to do 99% of the time.

Answer 12

I would recommend to split task as much as possible before performing any evaluation, this exercise only will force you to have a deep understanding of what you have to do, and how you plan to do it. Once this is done, you increase your chances of comparing the subtask to something you have already done and have something closer to reality. If the specification of what you have to do is unclear, you will also find out at this moment, which is a good thing.

Tasks over 40h can hardly be evaluated with precision, if your project manager is good, he'll know that! It's ok to be wrong, experience is the only real thing that will make a real difference in your evaluations (and in the end, will only help you be closer to a correct answer, not perfect).

I also had a very nice experience with poker planning, the more my team uses it, the better their evaluations (plus it's fun!)