How to approach something complex?

Question

You know that particular part of your code that is essential for the project but will probably take a lot of time to get it done? Do you ever get the feeling that you'd rather work on something else (probably less important) or not code at all instead of working on that part? That beast you try so hard to avoid and use every lazy trick you know to delay its inevitable implementation?

Now, I'm probably just being lazy, but I've always had to deal with code like that. Write something I don't feel like writing (and it's worse if you're doing it for fun and not getting paid!). A large system that will take a lot of time to get it into a stage where you get back any useful results or indication of it working. How do you start coding something like that? Most people would probably suggest divide and conquer and similar architectural techniques, but this isn't about how you do it; it's about how you get yourself started on doing it. What's the very first steps you'd take?

Answer 1

I agree with you that many large, important parts of a software are not fun to write. I usually start my development day with some smaller things, like adding a feature here, or fixing a bug there. When it's time, I start with the large part, but when I just can't see the thing any more, I do something different. That's all fine if you still get everything done on time. And remember that it may make things easier if you talk with other people about that large beast before you're doing it, while you're doing and after you're done. This will not only free your mind, but you'll also get other people's opinion from a less subjective point of view. Planning such things together also helps.

Answer 2

Funny, I'm the other way around. When I start tackling a problem, I go for the big ones first. The core of the problem is usually what interests me.

If I'm doing a project for myself, I usually couldn't be bothered to implement all the fuzzy bits around the edges, so they never get done. If I'm doing something for real, I eventually get to all the fuzzy bits, but it's not my favourite part.

Answer 3

I think there are two issues here.

First is actually getting started. As you say, that can be pretty tricky. Personally I just start on any bit, just to get something on paper/screen. It will probably be wrong and need editing but, in general, it's easier to criticise than create, even on your own work.

Then there's the actual process of solving hard problems. There a great book called "Conceptual Blockbusting" that discusses various ways of approaching problems. I learned a lot about how I approach problem solving and my blind-spots using that book. Highly recommended.

Answer 4

I'll tell a story of a case in which this happened to me.

I wanted to implement a new frametype decision algorithm for x264 that used forward dynamic programming (the Viterbi algorithm). But it was going to be complicated, messy, ugly, and so forth. And I really didn't want to do it. I tried to pawn off the project onto Google Summer of Code, but out of some sort of terrible bad luck, the one student that we had that simply bailed on his project... was the student that chose that project.

So after two months of complaining about it and dodging it, I finally got to work on the algorithm. And here's how I did it.

First, I talked to the other developer, who apparently already had some ideas on how to do it. We talked it over and he explained it to me until I fully understood the process from an algorithmic standpoint. This is the first step of any such project: understand the algorithm behind it so well that you can pseudocode the entire thing.

Then, I talked to another coworker of mine. We went up to a whiteboard and I sketched it out until he understood it too. By explaining it to someone else, I gained understanding myself. This is the second step: explain the algorithm to someone else so well that they can pseudocode it. This is an emulation of the programming process, since programming is a form of "explaining" an algorithm to the computer.

Then, I wrote a simple Java prototype that used arbitrary fake values for the cost function and was solely being used to test the Viterbi search. I finished it, and checked it against an exhaustive search--it matched perfectly. My dynamic programming was correct. This is the third step: write the simplest possible form of the algorithm in the simplest possible environment.

Then I ported it to C, x264's native language. It worked again. This is the fourth step: port that simple form of the algorithm to the full environment.

Then, finally, I replaced the fake cost function with the real one. After some bughunting and fixing, it worked. This is the final step: integrate the algorithm completely with the environment.

This process took barely a week, but from the perspective of me at the start of the project, it was completely daunting and I couldn't get myself to even get started--yet by breaking it down into such a step by step process, I was able to not only get it done, but get it done much faster than I expected.

And the benefits went far beyond x264; I now understand Viterbi so thoroughly that I now can explain it to others... and those others can benefit greatly from it. For example, one of the ffmpeg developers is using an adaptation of my algorithm and code to optimally solve a somewhat different problem: optimal header placement in audio files.

Answer 5

Generally I love the large, complex part. They are the parts that actually extend a challenge and force me to carefully consider what I'm doing. It's all the small, tedious bits that I dislike. However, when it comes to doing anything I've been putting off I find one simple piece of advice important:

JUST DO IT!!!

Seriously, once it's started it's much easier to finish. I always find I put things off until I start them, then suddenly I find that, now that I've started, it's not as bad as I had imagined, and look, it's almost done already!

Answer 6

Divide and conquer is not just about structuring code, it also works as an approach to make a project conceptually manageable. If I have a hard time getting started on a project it almost always because it's to big and scary. By dividing into conceptually manageable pieces, it becomes less scary.

I also believe in "tracer bullets" as described by the pragmatic programmers. Reduce the project to the absolutely simplest possible "proof of concept" of the core parts, e.g. without UI, special cases, error handling and so on. Perhaps its just a few core routines with associated unit-tests. With this you have conquered the scary parts, and can build from the core.

Basically the trick to getting started (for me at least) is: Don't start on the whole project. Start on one small (preferably core) part and build from there. If I still have a hard time getting started, It's because the small part I decided on is still to big, so I have to divide and reduce it further.

Answer 7

I try to establish a metaphor for what the system is trying to do. I always feel more comfortable when I can describe the behaviour in terms of a metaphor.

I then approach it from a test-driven development point of view, i.e. start describing what the system needs to do by setting up the tests that will verify correct behaviour.

HTH.

cheers,

Rob

Answer 8

The most difficult part of the project is going from having nothing done to the first line. Just putting anything down on paper gets this process started and it's amazing how quickly the rest can flow from here.

I'm a fan of the "divide and conquer"-type approach myself.

When there's a particular large task in a system hanging over me, I leave the computer, take a pen and paper, and break the task out into all of it's logical components and work-flow.

Then take each of these tasks, and break the down into the most basic functions / calls required.

I can then put in stub methods that I reckon I'll need. and flesh them out one by one. At this point each of these "sub-tasks" is no larger than the smaller development tasks orbiting the same project, so feel like a much less-onerous task hanging over me.

Answer 9

I usually tackle this kind of problems at home using a pen and a piece of paper.. I imagine the algorithm and/or logical flow and then stub (on the paper!) the classes and method stubs and when I get in front of a/the computer I can do it much easier... Probably it's just me..