Do you find that programmers generally get half way through a project and then slow to a crawl and how to resolve this?

Question

I have found this both in myself and managers have mentioned it to me regarding other programmers: about half way through a project, the progress of a programmer will slow to a crawl and it's hard to get them to finish a project. (Half way could be anywhere from 10% to 90%, it's just getting to the "home stretch".)

I'm thinking it's mostly related to the first part of the project is more interesting: creating database, creating some libraries and generally figuring out the best way to do things and even testing new ideas, libraries, etc. Whereas the last part of the project is bug fixes, making small annoying tweaks or even making changes that you know will cause a problem later.

I'm wondering if others have found this in either others they have worked with, have managed or themselves? And what steps you have taken to resolve this?

Answer 1

This has happened to me on personal projects many times. The best way I've found to get around it is to take a little time before I start coding and actually plan out what needs to be done instead of firing up a text editor and coding. I used to think of a problem/project and dive right into it without thinking the whole thing through, then I'd get discouraged or annoyed halfway through when I encountered a use-case or condition that I hadn't considered before.

Better, more thorough planning will help spread the "fun" out a bit more evenly over the whole project, instead of just going buckwild crazy for the first 30-40% of it and then running out of gas.

My $.02, obviously :)

Answer 2

What has worked well on my team was a combination of two things:

(1) A good bug/case tracking system

(2) Daily scrum meetings where developers commit to a task by the next meeting and report a Yes/No answer in front of the group as to whether they met yesterday's commitment.

It tends to keep things on task and has the side benefit of exposing rabbit trails.

Answer 3

In my experience, it's mostly a question of managing expectations.

At first, a project will seem like it's making progress very fast, because visible stuff is happening. Then, the finishing seems to take forever, because it's less visible (but just as important). The same thing happens when you're creating just about anything too, whether you're building a house or sewing a dress.

One thing I try to do, personally, is to have the outside appearance of a project match the state of its guts. This makes it easier for an outside observer to gauge the level of completion of my project. For instance, using ugly fonts for early prototypes is a somewhat silly but very efficient way to show that this is just a proof of concept.

Bugs should also be resolved as they occur, and not just at the end (same with documentation, testing, etc...)

It also helps to keep time estimates for a milestone, and to reflect back on how accurate you were. Did you underestimate debugging? What took longer than expected in the last mile? That usually help craft a more realistic schedule in the future.

Of course, if you're dealing with procrastination because you dislike some parts of a project, then it's mostly a question of kicking yourself in the butt, setting goals and just getting it over with.

Answer 4

At the start of a project, anybody is fresh and motivated. So things go great. Then if a project is further on its way, small annoyances start to enter the team. Maybe the customer or the manager is not satisfied, or maybe some programmers can't get along very well. Or maybe too much features are added and there is not enough time to do anything right. So yes, it is not strange that motivation drops at the end of the project.

And you must not forgot that 80% of the progress is gained in 20% of the time. The remaining 20% are the hard bits that need 80% of the time.

Answer 5

I have found it to be at the 80% point. At that point a number of things happen:

1] You have proven that you CAN build the project and there is little else left to prove.

2] Also, in conjunction with this, the remaining 20% is just plain boring work, or is something that involves stuff which you have never done before. Which brings us to...

3] Roadblock. Often it happens that the project is stalled because of failure to implement SOME feature which is difficult to implement. The programmer is tired and mentally fatigued with the work, and often the unpleasant work seems to be pushed till the end.

One way to do it is that if you KNOW that you are going to have to do something you arent very good at, implement atleast a prototype of that first, so you can solve the problem while you are still fresh.

The threat of being laid off also works as an excellent motivator, IMHO.

Answer 6

This is simply the 80/20 rule, in that in any project the last 20% will take 50% of the time, and personal observation suggests that this is a recursive property ;-)

Joking aside, I think you're approaching the issue from the wrong angle. There is a strong tendency among all analysts/developers to underestimate the time required to implement and test the detailed business rules of a project. We are all familiar with the issue of showing a client a screen mockup of a system - perhaps with basic click through - and the client believing that the project is then practically done whereas in reality it's barely been started. I believe there is a similar issue with programmers in that once they have the screens, database, and basic code structure defined they believe they are practically done, but in reality this is probably just the half-way point.

The solution? Experience really. You have to learn to include to allow for time to implement the detailed logic that wasn't apparent in your top-down project specification and the iterative low-level debugging process. The old engineering saw of taking your initial best estimate and doubling it really isn't such a bad approach - although learning how to stand up to your boss and defend that is a whole different ballgame.

Answer 7

I think many problems that a programmer can tackle are not able to be finished because of a lack of vision. You start out with an idea that's vague and nebulous so the direction is fuzzy. Once you get into the divide and conquer parts of a project you can go no further without knowing with certainty the direction your application must take. Also many of the managers or drivers of the project may not understand how much work is really involved and the typical programmer doesn't have the ability or desire to try and force someone else to understand the limitations they face.

Answer 8

There are many, many reasons this could happen, but one I've often seen is a lack of good archetecture. People make a lot of "progress" early on, but then find themselves fighting against decisions made early on that are boxing them into uncomfortable corners. Good archetecture design up front can help with this considerably.

It should also be noted that (in my opinion) that the best programmers are not always the best archetects, so just telling the same people that are having troubles to "plan ahead" often won't work. They're probably already doing what they think they need to. Having an archetect work with them to design the project can help considerably.

However, good archetects can be very difficult to find. (I am most defintely not an archetect...but I have seen good ones in action, and have found that they have made my coding projects much smoother.)

Answer 9

Some programmers are great starters, they jump in quickly and get the project going right away. Some programmers are great finishers, they have trouble initially, but they have the power to slog through and get it finished.

If you have access to the resources, teaming a starter with a finisher is always a great combo. One dives in, the other drags it through to the end.

If you're a starter, and you stuck in the mud, it's best just to concentrate on getting the next piece completely done, before moving on. That is, the main problem you're having is indecision and too many open pieces. Pick a direction regardless of whether it's right or wrong (just stay consistent), and one-by-one close off all of the pieces, even if its ugly. Scale back everything, and head for a release sooner, not later.

Oddly, if you like starting better, shorter iterations are more natural and comfortable.

Paul.

Answer 10

1. Depends on what you mean by "half way." How are you measuring progress to begin with?

2. No matter what, the devil is in the details. Try to mitigate this by

   • planning -- do your diagrams (but don't go crazy), and
   • testing as you go.

But by many measures, you can reasonably expect the pace to slow as you proceed through a project.

Answer 11

One thing I have found helping me keeping up the productivity is TDD. If you have a good set of tests, they should tell you if you have implemented all the features that your fuctional spesification describes. You could look at them as your project manager constantly complaining until you have implemented all features. Also they will help you through the difficult phase of implementing tedious infrastructure code (logging, cache, validation) to make sure this does not break you business code.

Unfortunately this is quite limited to your business logic and even then it can be quite difficult writing the correct tests. Usually my productivity falls when its time to implement the user interface, but people are of course different here so having different people doing the things they like in the different parts of project is a key part for keeping up project productivity.

Answer 12

Its an illusion, usually. Later on in a project, you need to make sure your code can survive regression tests against everything else already developed, among other things. It just takes longer to make feature advancements as the project gets bigger.

This can be managed somewhat by a well organized plan of attack on your project, but it cannot ever be entirely avoided.

Answer 13

Very much agree with @mostlyharmless' point 1:

1] You have proven that you CAN build the project and there is little else left to prove.

One solution to see a project to completion is to appeal to your obsessive compulsive side:

• Coverage tools to coax you to right more tests
• That nagging feeling that there must be an edge case out there that needs testing. You must be able to say with confidence and with pride "That will. not. break!". (Unlike the yeah it 'shouldn't' break attitude a lot of developers have).
• If you use maven-release-plugin, desire to release, which usually involves a battery of tests passing. And you get the satisfaction of seeing a release tag being created for you in svn.

Answer 14

It's natural and it occurs in other kinds of projects and areas of life too.

Consider moving out of a home. Packing all the big things and the fun things first is easy. But all the little things at the end take an enormous amount of time again, although you thought you were almost done.

Another analogy: Running a marathon. People who have done it say that the last 6 miles are the hardest. In a way, the first 20 miles are the first half of the marathon, and the last 6 miles are the perceived second half.

Or consider learning a language (e.g. French). The learning curve is really steep in the beginning, you learn a lot of new words and start being able to make simple conversations, which is a lot, compared to not being able to speak the language at all before. But progress goes much slower after a while. It's just not that easy to improve something that you can already do reasonably well, but not yet good enough.

Answer 15

I've found this occurs most frequently where the project plan is not defined. Usually, for me, these are on smaller projects where I think "I don't need to plan, I can whip this out in no time."

On projects that I know will be more intensive, I'm more likely do the following, all of which keep me on task:

1. Define success (almost a requirements gathering, but at a higher level, enough so that the final customer easily understands what we are shooting for) and get them to sign off on it.
2. Do a work breakdowns structure to map out what tasks need to be accomplished to reach success and get them to sign off on it.
3. Use the WBS to determine a good estimate at how long the project will actually take, and get them to sign off on it.
4. Hold weekly meetings for the sole purpose of ensuring that we are on track. (At this point, I am keeping stakeholders on task, and they are keeping me on task.)

All of these allow us to set a target date that is realistic (but flexible.. We can change the target date if it appears we are not going to meet it).

Having this deadline keeps me focused, as I know I have to be done by a certain date or have to explain to the group why I didn't hit my deadline.

Also, the combination of the "Success document" and the time line helps alleviate scope creep, which can cause a project to drag on forever. I find that if I keep the customers focused on the question "Is this feature necessary for us to be successful with this product?" and the question of "Will this affect the project time line?" I can count on them to weigh the balance of additional feature requests against these factors and come up with the right answer based on an actual business need.

Answer 16

The case that sticks out most in my mind was large team that worked on a big project for a long time, continuously acting like they were almost done. Eventually they were drowning in half-baked quick-fix solutions, and progress, of course, slowed to a crawl.

The didn't have automated testing, their build was fragile (required not just lots of manual steps on a fresh machine, but also lots of manual steps for each subsequent build), they didn't optimize their architecture to make common changes easy (instead they just tried to do them as little as possible). The last one I think hurt them the most, because they'd come up with the most imaginative work-arounds to avoid changing the 'fragile' configuration items, which made the whole system even more rube-goldbergian.

The solution that I suggested (that they ignored) was to refactor vigorously. Of course they didn't, because they were 'almost done'. That was about 1.5 years ago, I think they were about 6 months away from completion at the time. They are still going strong, I think they are about 6 months away from completion.

Answer 17

I think it is partly a question of complexity. At the beginning of a project, there is no previous body of work to build from. Everything you implement shows some kind of progress. As you proceed, your new ideas have to integrate with what you have already done. This can mean changes to problems you have already solved which shows little progress to outside observers. The closer you get to the end, the more stuff you have to modify to get the new stuff to work.

Answer 18

Schedule slip, quality deterioration, and irrelevant software are three common risk factor in any software development.

There could be many causes to schedule slips, but lack of requirements and cow boy coding (developer codes what he/she feels right) could usually do the trick.

Case 1.
If the requirements are not bound to domain expert's knowledge, the perceived progress would be made initially as the developers make things up as they go. Later when the software actually start to work, the users and experts realize that it's not doing what they wanted.

Case 2.
Feature creep. In the middle of the project when things start to work, either developers or the stakeholders start to add features that was not initially planned.

Case 3.
Modifiability and maintainability of the system/code was neglected. When the system/code is relatively simple and small things move along, but when it reaches some point rate of bug growth exceed rate of actual development.

All cases can co-exist simultaneously. There are certain things you can do to alleviate the risk factors, but they first need to be acknowledged as possible risk.

First, the requirements. Both the developers and stakeholder (business domain experts, users, and/or user surrogate) must agree on what are being developed before the coding starts. Of course, it doesn't have to be complicated or be done all at once up front, but it should be a document that could be reviewed to correct misunderstands. At the least the document should specify the expected input and output that could be used for the domain experts to determine whether the system succeeded a task or failed. The document should not mention anything about "how" the system is implemented.

Second, to avoid feature creep, let the stakeholder pick the must have tasks for a given iteration.

Finally, invest time and resource and refactoring and unit tests to maintain high modifiability.

These are some suggestions, and not a silver bullet.

Answer 19

Plan it out. Do the hard and boring stuff first, the fun stuff last.

Answer 20

A lot of answers contains good thoughts, but i think the main problem is that you dont't want to finish the project. You just want to code some interesting problems.

I don't want to write a speech about the purpose of life but coding for coding is definitely not a good purpose to chase.

Answer 21

What's happening is management is likely seeing some early-stage UI and, since they aren't engineers (otherwise, they would have become engineers and not MBAs), they don't understand that "eye candy" != functional product. So your managers are either lousy at being project analysts or they're lousy at managing their employees. In either case, they are the likely culprit.

Don't beat yourself up over it. As many have said before me on this thread, this is just the way that projects work: Hard bits, things that require new skills, and otherwise boring components are going to get thrown on a pile for a later. Just be thankful you spent all that time before-hand planning the project out so that your 80%-done doesn't turn to 10% when you realize you have to refactor the entire project to support X.

Answer 22

Sometimes this is what happens when the developers have done everything structurally to implement the mechanical maintenance of database tables according to the prescribed relational design, and built all the OOP classes as described in the URL or other documentation, and they starting bumping into the edge cases where domain knowledge starts to get slippery.

I've found it's a lot harder to get the non-technical stakeholders to accurately describe their work processes than early design phases assume. More commonly than not, they don't appreciate how incompletely they understand their own processes, and how much of the way things really work isn't as they assumed. This is especially a potential issue when the domain experts are management.

So the real Requirements Discovery phase starts, and software skills yield to listening skills and questioning skills for a while.

The only effective way to alleviate this that I've been able to find is to take seriously the Agile practice of rapid-fire short term deliverable working software.

Answer 23

You need an experienced closer.

It takes experience to learn how to close a product, how to realize when enough is enough, when to know how to wrap things up and finish a project.

I have found, that by breaking a project into smaller pieces, and ensuring that each piece is closed first, before moving on, is the best way to get over the big bang project wrap-up.

Answer 24

What's happening, I think, is that until you're "mostly done" you don't fully understand the problem you're trying to solve. Only then do you realize just how much you have left, and it's always more than you expected when you began.

Try taking smaller steps. Build a smaller product and ship it; repeat. That last 20% will change from an extended deathmarch to an afternoon's work.

Answer 25

Maybe people want to finish their project but tent to forget that it needs to run perfectly and perfectness is reeeaaally expensive.

Answer 26

I've voted up Beska and Jason's answers, and would add mine to address the question.

One of the key issues is that early on there are no constraints to the solution of the problem being addressed. The further you get into a project, the more design that has been done, and the more that has been written creates a structure that is now constrained as far as how it can change and adapt. Bugs, issues, definitional changes, etc., all force changes to occur, and how well that system can change and adapt is what determines how long it takes to finish that last part of the project. Some hard problems arise when the solution to a bug requires causing instability to work already performed.

Addressing this issue is one of the hard parts about management and architecture, and is why the right people making those judgements are important.

Answer 27

The end is more difficult than the beginning. Programmers know this, managers need to understand it.

You need a good triage process to sacrifice features at the end, or you'll end up with a bunch of programmers sitting around bored or mad or goofing off while the programmer who is blocking everyone else panics.