views:

115

answers:

3

It could be as simple as one-liner scripts, or as complex as a complete programming language with parser and debugger.

But there's the well-known danger of creating a labor-saving tool, that takes more labor to create than the original task (unless you can it amortize it over several projects).

I'm concerned about getting carried away with this secondary project, trying to make it perfect, and extending it... while the primary project languishes (eg: Knuth taking years away from writing "The Art of Computer Programming" to create TeX to help him typeset it).

I'm not thinking of standard support/dev tools, like build tools, test systems, bug-trackers and source control, but ones that you create for a specific project, to support your own development, that only developers will ever use (i.e. it's not a deliverable of the project).

+3  A: 

It's so easy to get caught up in the fun of tool creation. We try to manage it by looking at the tools we need to write, spending some decent time looking for opensource that we can use or bend to our will, and only then resort to rolling are own. We also do this as an iteration zero, exercise, with sprints and scrum - if a tool takes more than one sprint (2 weeks) it's far too big.

MrTelly
+1  A: 

It's easy.

Start by writing use cases (or user stories). Be sure to include operations and support staff as well as end users.

Then build software to deliver on the use cases.

Any "support/dev tools" must be part of a use case to justify it.

Edit When it comes to coding support, it's simple cost/benefit. What's the cost of building a code-generation tool vs. the net savings the project? Developer tooling is rarely of much value. Software is operated and supported for a very long time; it's only in development for a very short time. You might spend 2 years developing it; a customer might spend 10 or more years operating it. Saving a few person months in development is often irrelevant.

That's all there is to it.

Remember that support and help desk are first-class users. The support use cases have to be put into the mix along with end-user user cases. The use cases are prioritized and built as needed. At some point operability considerations have to take precedence over user features.

For example, we just finished building a couple of pages that would only be used by operations (and possible help-desk) staff to help a customer through a particularly important transaction.

Writing new infrastructure (languages, compilers, debuggers, OS's, RDBMS's, ESB's, etc.) is rarely necessary.

Writing new protocols and the associated clients and servers, however, is often necessary. Not everything is trivially solved by desktop software using ODBC/JDBC database connections. Similarly, not everything is a web app over HTTP.

Inventing a new programming language is an example of letting your hobby take over your job. If you can't do it with on of the top 50 languages on the TIOBE Index, you're just playing.

S.Lott
Interesting, but my question is about tools that you, as a developer, would use to develop the project - that would help you to write code The examples you mention, of support pages for operations/help desk and new protocols, are part of the delivered code (and not used to make that code).
13ren
+1  A: 

Thanks for the question! Long reply and late, but hopefully worth the read.

Conventions

I call these temporary tools "scaffolding" by analogy to construction industry, in manufacturing they're commonly referred to as "jigs and fixtures" and if you prefer a more academically sounding name it's "tooling and production technology".

“Proper” Way

It might seem that the decision whether to invest into the scaffolding can be determined (as the project management science would have it) by a simple cost-benefit analysis: estimate the development cost, compare to predicted benefits and if the value of scaffolding is greater than the cost there is a good point in building it:

COST               BENEFIT
Spec         £500  Time saved over 2 years £3000
Development £2000  Savings on training      £600
Total:      £2500                          £3600

Well, but what if a similar tool is already available off-the-shelf, but doesn’t provide quite the required functionality and hence the same level of benefit? The analysis gets an extra dimension “buy vs. build” (note that for many community tools it's only licensing cost that is free, many other expenses still apply, hence these tools need to be evaluated under the "buy" category):

       COSTS              BENEFIT
BUILD  Spec         £500  Time saved over 2 years £3000
       Development £2000  Savings on training      £600
       Total:      £2500                          £3600

BUY    License      £500  Time saved over 2 years £1000
       Setting up   £500  Savings on training      £300
       Total:      £1000                          £1300

Obviously, an off-the-shelf tool is cheaper to buy; however, its return on investment (ROI) is just 30%, when for do-it-yourself solution it’s 44%.

Well, if only things were that simple, because in real life actual costs and benefits are hard to identify and quantify in advance. How do you quantify an opportunity cost of developers working on scaffolding instead of project deliverable? Or a benefit of having some production knowledge incorporated within scaffolding so new starters don’t have to spend time learning a manual build or release process? Or a benefit of anyone being able to build any version of software in one step in less than half an hour instead of a senior developer spending most of the day trying to complete the task?

Additionally there inevitably going to be some variance between predicted and actual figures and the bigger the expected variance (also referred to as “risk”) the less you can rely on the figures to materialise. The risk gives our analysis a third dimension and given that some of the costs and benefits are entirely contingent the whole cost-benefit thingy really start getting out of hand.

Practical Approach

I believe there is simpler approach. Instead of trying to treat the scaffolding as a capital investment and quantify costs and benefits for years to come, try to assess the immediate situation and take more agile approach, i.e.:

  • Would it take significantly longer to automate the task than to do it manually?

  • Can a task be completed reliably without the scaffolding and any requirement for highly specialised talent or superhuman abilities?

  • Does the cost exceed benefits that can be extracted in the immediate future?

If any of the answers in “no”, when you’re most likely be better off investing into the scaffolding.

Some Notes

There are a few more things to keep in mind:

  • Scaffolding is highly specific, temporary and sometimes even one-off. Avoid the trap of approaching its development in the same way as you would approach application or component, the scaffolding is much closer to a working prototype by nature. Remember that writing an application that covers the same functionality will take roughly ten times as much resource as the scaffolding script whereas writing a generic component will take 10 times as much effort as the application. Benji Smith has made a brilliant illustration of the point in his creative outburst "Why I Hate Frameworks". Tools are supposed to serve as means to achieve a goal, but not to be a goal in itself.

  • Scaffolding should be separated from the application; it’s not really a deliverable and in most cases should not replace parts of application aimed at maintenance and configuration.

  • Having custom scaffolding gives a real advantage over competition:

    • Speeds up development.

    • Avoids duplicate work.

    • Decreases the need for accuracy and magical superpowers that are as a rule in short supply and very expensive.

    • Reduces human error.

    • Provides consistency of the outcome.

    • Gives the ability to do things in the exact way required (as opposed to off-the-shelf tools).

    • Helps preserving knowledge within the team (it never quits and can always be reverse-engineered).

    • Boosts morale, since developers don’t have to spend their time on mundane tasks.

  • By being crude non-critical solution scaffolding can serve as a test ground for new tools, techniques, technologies, ideas that later can become a foundation for a killer app (FogBugz is a great example of an in-house tool that had been initially developed to test FogCreek founders’ assumptions about the correct way of handling software issues, because they strongly believed it has to be done in an exact manner that was not supported by any COTS tools. As we all know later the scaffolding was capitalised upon to become a standalone flagship product). By building scaffolding you create intellectual property.

  • Test Driven Development (TDD) is a notable and practical way of building scaffolding for testing. TDD provides some excellent guidelines on “when enough is enough” that can be extended on other bespoke tools.

Totophil
Thanks! I especially liked: scaffolding is much closer to a working prototype by nature [tho it can become a nightmare to maintain] and non-critical solution scaffolding can serve as a test ground for new tools/techniques/technologies/ideas.
13ren
13ren, my pleasure. The idea is that there is no point in designing and testing the scaffolding to work in different enviroments, or making it more mallable, easier to change up front. If the enviroment changes it's possible either to refactor the scaffolding and make it more flexible in regard to these concrete changes or dismiss it completely and build another set of scripts. Especially if the initial investment has already paid itself off.
Totophil