Does evidence based scheduling work right with heterogenous estimations?

Question

Observing one year of estimations during a project I found out some strange things that make me wonder if evidence based scheduling would work right here?

• individual programmers seem to have favorite numbers (e.g. 2,4,8,16,30 hours)
• the big tasks seem to be underestimated by a fix value (about 2) but the standard deviation is low here
• the small tasks (1 or 2 hours) are absolutely wide distributed. In average they have the same average underestimation factor of 2, but the standard deviation is high:
  • some 5 minute spelling issues are estimated with 1 hour
  • other bugfixes are estimated with 1 hour too, but take a day

So, is it really a good idea to let the programmers break down the 30 hours task down to 4 or 2 hours steps during estimations? Won't this raise the standard deviation? (Ok, let them break it down - but perhaps after the estimations?!)

Answer 1

I think it is a good idea. When people break tasks down - they figure out the specifics of the task, You may get small deviations here and there, this way or the other, they may compensate or not...but you get a feeling of what is happening. If you have a huge task of 30 hours - can take all 100. This is the worst that could happen. Manage the risk - split down. You already figured out these small deviation - you know what to do with them.

So make sure developers also know what they do and say :)

Answer 2

• Yes, your observations are exatly the sort of problems EBS is designed to solve.
• Yes, it's important to break bigger tasks down. Shoot for 1-2 day tasks, more or less.
  • If you have things estimated at under 2 hrs, see if it makes sense to group them. (It might not -- that's ok!)
  • If you have tasks that are estimated at 3+ days, see if there might be a way to break them up into pieces. There should be. If the estimator says there is not, make them defend that assertion. If it turns out that the task really just takes 3 days, fine, but the more of these you have, the more you should be looking hard in the mirror and seeing if folks aren't gaming the system.
  • Count 4 & 5 day estimates as 2x and 4x as bad as 3 day ones. Anyone who says something is going to take longer than 5 days and it can't be broken down, tell them you want them to spend 4 hrs thinking about the problem, and how it can be broken down. Remember, that's a task, btw.
• As you and your team practice this, you will get better at estimating.
• ...You will also start to recognize patterns of failure, and solutions will present themselves.
• The point of Evidence based scheduling is to use Evidence as the basis for your schedule, not a collection of wild-assed guesses. It's A Good Thing...!

Answer 3

"So, is it really a good idea to let the programmers break down the 30 hours task down to 4 or 2 hours steps during estimations? Won't this raise the standard deviation? (Ok, let them break it down - but perhaps after the estimations?!)"

I certainly don't get this question at all.

What it sounds like you're saying (you may not be saying this, but it sure sounds like it)

1. The programmers can't estimate at all -- the numbers are always rounded to "magic" values and off by 2x.

2. I can't trust them to both define the work and estimate the time it takes to do the work.

3. Only I know the correct estimate for the time required to do the task. It's not a round 1/2 day multiple. It's an exact number of minutes.

Here's my follow-up questions:

What are you saying? What can't you do? What problem are you having? Why do you think the programmers estimate badly? Why can't they be trusted to estimate?

From your statements, nothing is broken. You're able to plan and execute to that plan. I'd say you were totally successful and doing a great job at it.

Answer 4

Ok, I have the answer. Yes it is right AND the observations I made (see question) are absolutely understandable. To be sure I made a small Excel simulation to ensure myself of what I was guessing.

If you add multiple small task with a high standard deviation to bigger tasks, they will have a lower deviation, because the small task partially compensate the uncertainty.

So the answer is: Yes, it will work, if you break down your tasks, so that they are about the same length. It's because the simulation will do the compensation for bigger tasks automatically. I do not need to worry about a higher standard deviation in the smaller tasks.

But I am sure you must not mix up low estimated tasks with high estimated tasks - because they simply do not have the same variance.

Hence, it's always better to break them down. :)

---

The Excel simulation I made:

• create 50 rows with these columns:
  • first - a fixed value 2 (the very homogeneous estimation)
  • 20 columns with some random function (e.g. "=rand()*rand()*20")
• make sums fore each column
• add "=VARIANCE(..)" for each random coulumn
• and add a variance calculation for the sums

The variance for each column in my simulation was about 2-3 and the variance of the sums below 1.