Developing using pre-release dev tools

Question

We're developing a web site. One of the development tools we're using has an alpha release available of its next version which includes a number of features which we really want to use (ie they'd save us from having to implement thousands of lines to do pretty much exactly the same thing anyway).

I've done some initial evaluations on it and I like what I see. The question is, should we start actually using it for real? ie beyond just evaluating it, actually using it for our development and relying on it?

As alpha software, it obviously isn't ready for release yet... but then nor is our own code. It is open source, and we have the skills needed to debug it, so we could in theory actually contribute bug fixes back.

But on the other hand, we don't know what the release schedule for it is (they haven't published one yet), and while I feel okay developing with it, I wouldn't be so sure about using it in production so if it isn't ready before we are then it may delay our own launch.

What do you think? Is it worth taking the risk? Do you have any experiences (good or bad) of similar situations?

[EDIT] I've deliberately not specified the language we're using or the dev-tool in question in order to keep the scope of the question broad, as I feel it's a question that can apply to pretty much any dev environment.

[EDIT2] Thank you to Marjan for the very helpful reply. I was hoping for more responses though, so I'm putting a bounty on this.

Answer 1

My personal take on this: don't. If they don't come through for you in your time scale, you're stuck and will still have to put in the thousands of lines yourself and probably under a heavy time restriction.

Having said that, there is one way I see you could try and have your cake and eat it too.

If you see a way to abstract it out, that is to insulate your own code from the library's, for example using adapter or facade patterns, then go ahead and use the alpha for development. But determine beforehand what the latest date is according to your release schedule that you should start developing your own thousands of lines version behind the adapter/facade. If the alpha hasn't turned into an RC by then: grin and bear it and develop your own.

Answer 2

It depends.

For opensource environments it depends more on the quality of the release than the label (alpha/beta/stable) it has. I've worked with alpha code that is rock solid compared to alleged production code from another producer.

If you've got the source then you can fix the any bugs, whereas with closed source (usually commercially supported) you could never release production code built with a beta product because it's unsupported by the vendor who has the code, and so you can't fix it.

So in your position I'd be assessing the quality of the alpha version and then deciding if that could go into production.

Of course all of the above doesn't apply to anything even remotely safety critical.

Answer 3

I've had experience contributing to an open source project once, like you said you hope to contribute. They ignored the patch for one year (they have customers to attend of course, although they don't sell the software but the support). After one year, they rejected the patch with no alternative solution to the problem, and without a sound foundation to do that. It was just out of their scope at that time, I guess.

In your situation I would try to solve one or two of their not-so-high priority, already reported bugs and see how responsive they are, and then decide. Because your success on deadlines will be compromised to theirs. If you have to maintain a copy of their artifacts, that's guaranteed pain.

In short: not only evaluate the product, evaluate the producers.

Regards.

Answer 4

It is just a question of managing risks. In open source, alpha release can mean a lot of different things. You need to be prepared to:

• handle API changes;
• provide bug fixes and workarounds;
• test stability, performance and scalability yourself;
• track changes much more closely, and decide whether to adopt then yet;
• track the progress they are making and their responsiveness to patches/issues.

You do use continuous integration, do you?