Development tactics : phoenix development cycles

Question

Hi world,

I was wondering how you guys actually develop large applications when you are your own boss. For myself I have been learning the hard way the necessity for patience and hope. I have been working on implementing an application (in the form of a series of scripts linked to a database) that clusters wikipedia articles using a combination of knowledge of wikilinks and article text/content. I have been at it for two years now; no results yet.

I cant seem to get any results for I am continuously reengineering my scipts and db due to changes in either the essence (pseudo-pseudo code, the theoretical algorithm) or form (script, threads, db tables, the practical algorithm) of the algorithm. Basically, I find myself continuously learning from the mistakes I discover while implementing; the devil being in the details, so are the answers it seems.

Anyway, every time I reengineer a script or a table or something, I need to scrap all my documentation and script. I am now able to do this without fear, but it has made me hate programming (I hate the details).

I feel that reengineering is the way to go since I am thinking long-term and I wish to learn fast, but I am wondering if you guys have a similar programming experience or if you never really need or choose to have a better script come out of the death of the last one (like a phoenix).

The hardest part for me is scraping my documentation for I spend more time documenting than coding; I use documentation as a means to discuss issues and consider solutions; I use it to formulate implementable solutions. While if it would be but for me, I would not mind scraping it, yet I always write it as if it were to be published the next week for while developing a script, I also seek to develop myself; I also try, like those of you who participate to this website, to share my knowledge or wisdom with others.

Anyways, I have been developing at full speed these last 2 months, reengineering countless essays, scripts, tables, etc; my patience is running low for I seek results.

Any tactics, any help, any experiences or anecdotes you wish to share?

Thanks for your consideration!

Answer 1

Since you are working solo, try to get feedback from someone else that knows enough to understand what you are trying to do. You will be surprised by things other people can come up with.

When working for pay it is somewhat easier because you have to get acceptance from your client, it is not as open ended as in your current situation.

Answer 2

keep a development log; this is the place to discuss details with yourself, work things out on paper, sketch documentation, make notes about current and future issues, remind yourself what you've tried and what you have left to try, and so on. It's also a good idea to always note what the 'next thing to do' is, so you don't waste time remembering/reviewing when you have time to work on it. Date each entry.

i have used this approach on a number of long-running, sometimes frustrating, one-man projects of durations varying from a few months to several years, and it helps keep me focused and it helps keep me from trying something more than once.

It also eliminated the internal pressure to document everything, which might be of great benefit to you.

think of it this way: notes to yourself about what you've done are useful to you, but detailed documentation intended for third parties for code that gets scrapped is a waste of your time.

Maybe this will help your OCD documentation neurons to shut up so you can solve the problem completely, then document the working solution - instead of continually re-documenting the latest prototype.

Answer 3

I think the hardest part of software engineering (not programming per se) is finding the right balance between pragmatism in getting it done NOW and getting it done RIgHT.

There is a tendency among (good) programmers to always want to pull things apart and put them back together is a neater/cooler/cleaner/... way. This is a good thing ... except when it means everything ends up continually in pieces and the elusive goal of a working (and really neat) system drifts further away.

For myself, I have to force myself to accept that something are not perfect and settle for getting some results in a short time period. The best way of doing this I find is to set a series of short term milestones that lead to the eventual goal. The stick to meeting the milestones even if some other part of the project looks more tempting for a while.

Answer 4

A good motto in programming is 'be willing and happy to throw everything away at least once', usually more. I'm currently writing a complete shell from scratch for a new operating system and I'm about to completely scrap part of the design, I don't like how I'm handling built in commands vs loadable modular commands.

It sounds like you dove in, head first .. excited to get some code written. From what I gather, you did spend some time with psuedo code (I even use a big dry erase board) to determine the structure of your code .. but the code was not able to survive radical changes to said structure.

There may be some good reasons why you're working completely isolated. If its possible, as others have suggested, bring someone else in who understands the end result that you hope to accomplish and hammer out a new design. Keep in mind that you application has proven to be quite volatile, the new design should not make you throw everything out because a few things changed.

I think you're also falling victim to what is known as premature optimization. Try to get something put together that works, even if its horribly inefficient and clunky, then really spend some time looking at how things can be improved. This step is almost a precursor to a new design that could survive radical changes in the future. If you can't bring someone else in, a working model of your current mistakes is almost as good as another collaborator.

Answer 5

Design Freeze!

Build the frozen design in tiny modules with min coupling - modules do ONLY one thing. When it works, you will be much smarter about how to fix any shortcomings. Minimum coupling and max coherence makes it easy to cut out defective parts. Don't scrap anything that works.

Regards, Bill Drissel

Answer 6

I was wondering how you guys actually develop large applications when you are your own boss.

I started by writing pages and pages and pages of functional specification ... describing in various levels of detail (including the UI) what I wanted the software to do.

I also tried to minimize risk by writing some prototype, proof-of-concept software components of the riskest or least-well-understood bits.

Anyway, every time I reengineer a script or a table or something, I need to scrap all ...

I certainly do refactor what I've written (I've been working on this since 2005), but only to add new functionality. Note that "refactor" is a technical term (look it up: there are books about it): note that "refactor" isn't at all the same as "scrapping" what I've already written (instead, I'm modifying-it-and-adding-to-it).

I have been at it for two years now; no results yet. ... Anyways, I have been developing at full speed these last 2 months ...

I don't know how much experience you have as a programmer. Unless you're very experienced, some options are:

• Take on a smaller project (don't expect to finish too big a project by yourself) ... the effort and skill to complete a project isn't linear with the project size (e.g. a project that's twice as big might be five times more difficult to complete)

• Start, but don't expect to finish (it's a learning opportunity which you'll learn from)

• Work with someone else

• Work more slowly and methodically (if you're only going 100 yards you can just start running, but if you're going 100 miles then you need to prepare a map etc.).

The hardest part for me is scraping my documentation for I spend more time documenting than coding ...

Someone (not necessarily me) ought to reply this paragraph of yours, or perhaps you could say more about it.

I'm very interested in this topic (in fact the product which I've developing is for documenting software development), but I don't know what to reply to this particular paragraph of yours.

Answer 7

Just make it work. Over time make it better, bit by bit. Take it as an opportunity to learn refactoring (in the strict sense--behavior-preserving transformations) and unit testing.

Answer 8

Release early, release often.