How can I organize all my code, data, scripts, tasks etc?

Question

What tools or approaches would you recommend to a 'one-man team' to keep organized?

I'm doing research that involves a lot of coding, writing hundreds of throw-away perl scripts, C++ binaries that get used until I find some better approach, large amounts of data that gets preprocessed in different ways, where some new preprocessing makes the old way obsolete -- until I find out that the old way was actually better, and so on. My work is inherently a moving target, as I have to try many things out, and often none of it is perfect.

It's not a completely chaotic situation, but it's also far from perfect. Are there general approaches that you would recommend in such a situation? I do use SVN for my code, although not for the different versions of the data because that gets too big. It's hard to keep track of all the scripts and binaries, so I always comment them, write down how I ran them etc. But I'm curious if you have some additional ideas.

(I work on a linux system.)

Answer 1

Keep all your code in your Version Control System, and create build/run scripts for each. Your data you are going to have to carefully file away (since you don't want to put it in SVN).

The other thing I would add would be a wiki so you can make notes quickly about each test/script/application.

Answer 2

why not checkout some opensource projects to see how they organize their code bases?

even though you are a one-man team. it'd be smart to organize your project so it would be easy to add more programmers.

also if you're worried about filesize for data files you might want to checkout git. the index size is usually a lot smaller than that of svn.

Answer 3

I'm using a wiki (TiddlyWiki in my case, as that runs absolutely anywhere - all it needs is a browser with JavaScript) as my "engineers notebook". Almost anything goes in there - lists of questions (and later on their answers), procedures (steps by step instructions), notes of what I put where (might work for your "data"), phone numbers (easy to find with a full text search), anything goes.

As my tasks are not that code-heavy, I'm even using it to store code-snippets (mostly SQL statements for me). Using a "real" versioning system is better if you want to keep track of different versions. Other than "use it" I can't offer specific advice for this area.

However, what has been important for me in GTD fashion: Use a simple system for almost everything. That way, the time to search for something and to decide what to put where is cut down.

Answer 4

Version Control is a must, as others said. Keeping descriptive labels of milestones on Version Control is very very helpful I think.

Also as IronGoofy said, I keep my snippets in my Personal Wiki TiddlyWiki, I uploaded it to my website, so I can access it anywhere, anytime.

As an additional option you may think a Time Tracker application. There are many free Time Trackers. I use ASP.NET's Time Tracker Starter Kit. You can keep evolution of your software, bugfixes, milestones.