How to manage a large project?

Question

I've done most of my coding on smaller projects for myself and classes, and this summer ended up working on a fairly large project.

For reference, I'm working on a web app using Zend Framework, and I'm working with one other person.

I started working on it in May - at the time, there was a lot of code already written (pretty sloppily) that I didn't have much control over. When I started, not knowing any better, I was just either editing files directly on the server with vim or using eclipse and ftp-ing the files up. Over the course of working on this, I've started using eclipse for editing, and SVN for source control and to deploy the files to the server.

My question is what are other good ways to both help my productivity with this, and also with managing the deployment better, as it gets closer to real people using it?

Edit: Realized two related things I left out.

One is that I've been eradicating bad/dangerous code as I end up touching on something that uses it. Is it generally better to devote time specifically to that, or is the way I'm doing it generally efficient enough?

Also, given that I have relatively limited time, is unit testing worth the effort or even appropriate?

Edit #2: I was reminded I left out the project management part. I've been having problems deciding what to hit first. I feel like some days I spend as much time keeping tabs on what still needs to be done than actually doing it. Is this common, how do others deal with it?

Changes I'm going to make: The big one I decided was to actually keep track of everything that needs to be done. I have a well-written spec of what's going to be there in the end, but a lot of times I'd lose little things and say over and over they need to be done. Now I'll keep track of them. Also, going to look more into a testing and deploying system.

Answer 1

Unit testing is crucial: it's the only way to get some assurance that your "eraticating bad code as you meet it" isn't going to break things badly. When you smell or spot bad code, build tests around it, THEN proceed to refactor -- really the only sane way to deal with legacy code!

You're already doing other key aspects, such as putting the code under version control; if you were managing/leading a large or distributed team I'd strongly recommend DVCS (hg is my choice, but git and bazaar are very popular too), but as you appear to be working on your own, svn's just fine.

Continuous build is the next recommended practice -- but, again, if you're basically on your own and just make sure to run your test suite obsessively, that comes to much the same thing. Next: good releng practices -- if and when a bug's reported on your code, you MUST be able to reconstruct EXACTLY the set of sources, configs, etc, that caused that bug.

What IDE or editor you like -- eclipse, vim, emacs, zend stuff, w/ever -- is really secondary. You seem to be developing the right instincts -- good for you!!!-)

Answer 2

This question is tagged as project-management, but I don't see any PM-type questions here :) If I were to put on my PM hat for a moment, I'd say that you can't manage without a plan. And a plan is a reasonably detailed description of all the activities you need to undertake.

But this is more methodology than anything else, so my two cents are thus;

1. Be absolutely sure that your bad code is actually bad. If it works, are you certain its bad code or does it just look odd?
2. Unit testing's value is inversely related to the bad code risk you're willing to accept. For example, if I work for a major telecom and we're changing our billing system, you damn well know we're going to test the change to death. But if the application is minor and you're willing to accept fixing an issue post-production, then you can get away with less testing.

Answer 3

The PM question... managing your activity list (or inbox) can become incredibly unproductive if you don't watch it tightly. The best item to be working on at any point in time is going to depend on;

1. What you can work on - Are all the prerequisites complete? You're not trying to code something before you've planned it, are you?
2. What do you really want to work on? - If you have something that you can work on, and you're in the mindset for it, go for it!
3. How much time you have available - If you have ten minutes, don't even bother trying to start a two-hour task.

Answer 4

Code:

1. Source Control - Done, good.

2. Unit Testing

3. Clean up bad code as you need to not just because. You should be working on producing a product and not just clean code. If you are moving to a code area where you need to do something then clean.

4. You mentioned a partner, consider pair programming....

Management:

1. Develop a Project Plan

A. Include Tasks / Feature List

B. Create a time estimate for them - If the tasks estimate is say beyond a few days worth of effort then you likely need to break the tasks down more. Large time estimates can be a sign that you haven't broken the problem down enough to really be estimating it well. Often when this is done, the estimate gets better.

C. Talk to your partner about how you want to manage this workload together.

D. Set a goal for what you want to accomplish for each week and go all out for it.

E. Perform post-mortem discussions on what went well the last week, what didn't, how could we do better, how could we keep doing what we are doing well.

And above all else, have fun!

Answer 5

It is ok, that issue tracking eating up more and more time. If you do not manage issues (features/bugs/etc) you will not be able to manage the project out. Since PM is a full time job, I'd expect that it will take bigger and bigger part of your time as project grows. It's ok. Embrace it as it is. Do not cut costs (i.e. time invested) on management (you do not have a dozen of assistants, do you?). Thus, coding progress will be slower and slower. It is ok. You need to invest time to the management, otherwise the quality of the project will drop so miserable, you won't be able to manage out.

Testing is crucial! But not unit testing, but acceptance testing. Unit testing is very important for the quality source code. But acceptance testing is important for quality of software. Invest time in a good testing framework and automate tests.

The biggest issue is requirements management. The question is: after all these changes, feature requests, bugfix and etc. how in the world should this application work? Accurate issue tracking and acceptance testing (with a good record keeping) make this hell a bit more comfortable.

Deployment should be a detailed procedure with each step defined and carefully described. Deployment procedure should be revised and refined after each version is deployed. No playing with a live site. Just follow the procedure, step-by-step. (You should have "Backup the website" as a first step).

But speaking generally, you need a special education (school or self-education) to do the software project management. It is very business-related and all about risk mitigation. Are there other people working with you or even under you control? Then there is another axis of complexity: relations between the team members. That's when you need to learn psychology, be brave and very patient.

Answer 6

Dangerous Code, I'd suggest should be eradicated if there is a reasonable chance that danger will bite you in the arse it will be worth the time to track it down and fix it.

Sloppy Code, Fix it as you find it (as it sounds like you need to focus on producing a product to a deadline right now), if any of this turns out to be dangerous code then treat it as above.

Unit Testing, depends, if you've a lot of experience with it you could likely be productive with it. If not it could be a drag on your time. Only you can decide if the trade off is worth it based on your schedule. I'd suggest some simple automated testing might be worth the investment, just to check that the absolute basics of you application work. This can be achieved at a pretty high level of abstraction, you could use selenium or something similar. This just avoids any really embarrassing mistakes being released when the pressure is on.

If you're going to have automated tests it'd be good to have an automated build process to run them, ie a continuous integration server. If you've other steps in your build such as compilation etc this can make sure these things are working too. Again though, your schedule might determine if you set this up now, or later. I had a large bunch of very basic tests running from my IDE when I remembered, sometime later we got the time to setup Hudson for our continuous builds and haven't looked back.

Prioritising Work, It's ok to not always know which is the important task to hit next. You need a list of everything outstanding, where I work it all goes in bugzilla. After this you need broad priorities on those things, If you're working towards a v1 of you product I'd suggest you make the prirorties v1, v1.1 and Road Map. After this you need to decide the next one or few things to work on. Depending on how often you want to return to the list it'll be one thing, or a bunch. This may mean sitting down with your partner and going over the list and working out what to do and how to do it. One problem I notice sometimes when I don't know what to do next is that I actually don't know how I'm going to do any of the next jobs, realising that often makes it pretty clear what is important next and that I just need to figure out how the hell to do it.

Managing work, If you've got a single place for new enhancements etc to go, ie bugzilla or user voice, make sure people use it and don't expect you to put things in there for them. Make a point of weekly (or whatever frequency works) checking over new entries and giving them a priority. The point being you can process these in batch freeing you up from the interruptions of dealing with them as they come in.

Better Deployment, You say you're using svn for deployment. I presume you just do a checkout straight to your production system. You don't say if you're using tags or branches at the moment. When you do deploy I'd suggest branching at that point, ie if it's v1 of your product create a v1_maintenance branch. Any bug fixes etc go on that branch and are merged (regularly and frequently, believe me it can get painful otherwise) into the trunk. This allows you to carry on full steam on the trunk whilst still being able to support the released version. You may want to look at ant or similar one day for deployment, but again if you've got something that works right now your effort is probably better spent elsewhere