How does one deal with Hofstadter's law?

Question

When estimating tasks, how does one break from the grip of Hofstadter's law?

Answer 1

Hofstadter's Law is not meant to be taken seriously --- if it were true to the letter, every task would take an infinite amount of time if you took Hofstadter's Law into account.

Answer 2

If you can politically: Estimate in small chunks, work in small iterations, and focus attention on what caused the deviation from the estimate to make the next estimate better.

One of the major causes of bad estimates in my experience is the lack of experience actually using the architecture planned for the project. By adjusting the estimates as things become more concrete and clear the estimates get better over time.

The other major cause of bad estimates is bogus estimates. Estimates kept artificially low to win a bid. The only way a consulting firm can break that cycle is give good estimates and win enough projects and deliver on the estimates to earn a reputation that they hit their estimates. Enough clients will respect that to make a reasonable business out of it, but building that up will be hard.

Answer 3

Base you estimates on past performance, not on best case scenarios. This does require you keep track of time spent on your projects. I don't care if you "know" that it will only take "6 weeks" to finish, if it took you 3 months to complete a similar project last time, it will probably take you 3 months the next time.

Answer 4

+1 for @Yishai - one of the benefits of an agile methodology like scrum is that people actually get feedback on the accuracy of their estimates.

You're never going to get better at something if you never know if you're wrong...

Answer 5

I like this method:

1. Make an honest estimate of the effort required for the task.
2. Apply a multiplier to the estimate. At least 1.5 probably 2.0. With time comparing actual effort to estimated effort you will be able to calculate the true multiplier.

Collecting the estimated and actual efforts are key to improving your estimates.

Answer 6

Agile estimation always uses "ideal hours" which implicitly takes into account Hofstadter's law. So you don't need to fudge.

If you're answering as an employee ...

"Gee, boss, In a perfect world it would take X days. Let's add a cushion to it and I'll do all I can to get it to you in that amount of time. If the estimate changes I'll let you know immediately."

That is music to a boss's ears!

If you're answering as the business owner ...

You only give estimates to your customers when backed into a corner. Then you use ideal days with clear disclaimers and be ready to adjust because you're aware of Hofstadter's law.

Answer 7

1. Estimate how long time something should take to code.
2. Multiply by pi.
3. Be amazed by how often that is closer to how long it actually takes.

(This is also not to be taken as a scientific method, but it is another way of expressing how hard it is to correctly estimate time. I really use it sometimes, though...)

:)

Edit:
A method that is a bit more scientific: Specify a time for the absolute minimum and maximum time for a task, for example that it will definitely take between 5 and 30 hours. (Divide into subtasks to possibly narrow the time span somewhat.) You get a very wide time span, but at least it's more reliable than a guesstimate.

Answer 8

While "Hofstadter's Law" is a bit tongue-in-cheek, there are a couple of practices that can help you, in particular for first-pass/large item estimation:

• Estimate in relative sizes. Meaning you don't say that an item takes X time, you say that an item A is twice as big as item B, and that item B is about 4 time as large as item C.

• Gather data from previous estimating rounds and use it as a base line. So that when you are estimating a project, and notice that item A is about as big as item B from a previous iteration/project, and you know that item B has taken 2 days, you know that item A will most likely take about as long

• Use "wisdom-of-the-crowds" to get higher quality estimates. I've used Planning Poker in a couple of projects and the outcomes are rather good.

If you want to know more about this you can start by watch Mike Cohn's presentation (Part 1 and Part 2) and/or read his book. While it's not the end-all,be-all of estimation, he does present some good practices and best of all, the reasoning behind the practices.

Answer 9

Estimating is an art, as you know and there is a sub art that is the art of estimating contingency. :) In order to properly estimate contingency (generally a % of a total estimate), one must understand risks and mitigations. Basically, you multiply the risk of something happening with the damage it can do to come up with a risk factor. Then, you sum up all your risk factors and estimate your total risk. Contingency should range from 15% for very low risk projects (I never go below 15% contingency) to 50% for very high risk (it has been my experience that very few customers will except a higher than 50% contingency estimate).

Answer 10

See Evidence-Based Scheduling. There is already a SO discussion of some of its pitfalls here.

Answer 11

Hofstadter's law is just another implication of how notorious self-referencing is !..... the subtle humour has far reaching effects. In hindsight this law confirms that every law/principle/axiom which is structured by logic, is incomplete (Godel like), thus even taking such laws into concern, the logic may never be complete. The sense of infinity is again a play from Zeno's paradox (turtle vs. Achilles).... infinite time for Achilles to complete the race ....etc .... these are illustrations of the omnipotent evil of self referencing which contaminates all affine logical structure.

Answer 12

I used dice. Openly. In front of my manager. Typically I use 3 standard six-sided dice.

Boss: "How long is this going to take?"

Me: (rolls) "About 11 days."

Boss: "No, seriously."

Me: "Oh, seriously." (rolls) "About 7 days."

I also used to have a poster on my wall that said "Deadlines Amuse Me". Take from that what you will.