How do "spikes" figure in the schedule / estimation game?

Question

Might be subjective and/or discussion.. but here goes.

I've been asked to estimate a feature for the next big thing at work. I break it down.. use story points come up with a estimate. The feature however calls for interfacing with GoDiagrams a third party diagramming component in addition to various other company initiatives.. (a whole set of 2008_Limited_Edition frameworks/services:). I've been tracking myself using a burn-up chart and I find that I'm unable to sustain my pace primarily due to "spikes".. Definition

I estimate for 2 points a week and then I find myself working weekends (well trying to.. end up neither here nor there) because I can't figure out where to hook in so that I can preview user-actions, show a context menu, etc. In the end I spend time making spikes that throw my schedule off-track... and decreases its value.. doesn't give the right picture.

Spikes are needed to drive nails through the planks of ignorance. But how are they factored into the estimation equation? Doing all required spikes before the feature seems wrong.. (might turn out to be YAGNI) Doing it in between disrupts my flow. Right now it's during pre-iteration planning.. but this is pushing the touchline out on a weekly basis.

Answer 1

I guess you are constantly underestimating

• what you do already know about the 3rd party component
• how long it takes you to create usable/helpful spikes for unknown areas

1. Get better at estimating those two things.

So, it's all about experience. No matter what methodology you use, they will help you to use your experience better, not replace it.

2. Try not to get lose track when working on those spikes.

They should be short, time boxed sessions. They are not about playing around with all the possible features listed on the marketing slides. Give them focus, two or three option to explore. Expect them to deliver one concrete result.

Update(Gishu): To summarize

• Spikes need to be explicit tasks defined in the iteration planning step.
• If spikes exceed the timebox period, stop working on it. Shelve the associated task. Complete the other tasks in the current iteration bucket. Return to the shelved task or add a more elaborate/broken down spike to the next iteration along with the associated task. Tag a more conservative estimate to the generation 1 spike the next time.

Answer 2

@pointernil.. It's more of no estimation coupled with a Indy-Jones Head-First approach to tackling a story. I estimate stories by their content.. currently I don't take into account the time required to find the right incantation for the control library to play nice. That sometimes takes more time than my application logic.. So to rephrase the Original question, should spikes be separate tasks in the iteration plan, added on a JIT basis before you start working on a particular story?

My Spikes are extremely focussed.. I just can't wait to get back to the "real" problems. e.g. 'How do I show a context menu from this control?' I may be guilty of not reading the entire 150+ page manual or code samples.. but then time is scarce. The first solution that solves the problem gets the nod and I move on. But when you're unable to find that elusive event or NIH pattern of notification used by the component, spikes can be time-consuming. How do I timebox something that is unknown? e.g. My timebox has elapsed and I still have no clue for plugging-in my custom context menu. How do I proceed? Keep hacking away?

Maybe this comes in the "Buffering Uncertainity" scheme of things.. I'll look if I find something useful in Mike Cohn's book.

Answer 3

I agree with pointernil. The only issue is that your estimates are incorrect. Which is no big drama, unless you've just blown out a 3 million dollar project of course :-)

If it happens once, its a learning experience. If it happens again and the result is better, then you've got another learning experience under your belt. If you are constantly underestimating and your percentages are getting worse, you need to wisen up a bit. No methodology will get you out of this.

Spikes just need to be given the time that they need. The one thing I've seen happen repeatedly in my experience is that people expect to be able to nail a technology within a couple of hours, or a day. That just doesn't happen in real life. The simplest issue, even a bug caused by a typo, can have a developer pulling their hair our for huge chunks of time. Be honest about how competent yourself or your staff really are, and put it in the budget.

Answer 4

If you run out of time in your timeboxed spike, you should still stop and complete your other committed work. You should then add another spike to your next iteration to complete the necessary work you need to complete in order to accurately estimate the task resulting from the spike.

If there is a concern over spiking things for too long and this becoming a problem - this is one reason I like 1 week iterations. :-)