Can Hudson branch promotion get based on project stability?

Question

Hudson CI server displays stability "weather" which is cool. And it allows one project build to kick off based on the successful build of another. However, how can you make that secondary project dependent additionally on the stability of multiple builds of the first project?

Specifically, project "stable_deploy" needs to only kick off to promote a version to "stable" if project "integrate" with version 8.3.4.1233 has built and tested successfully at least 8 times--in a row. Until then, it's still in integration mode.

IMPORTANT: A significant caveat to this is that a single set of Hudson projects gets used as a "pipeline" to process each new version through to release. So a project may have built successfully 8 times in a rolw but the latest version 8.3.4.1233 may be only the 2 most recent builds. The builds prior to that may be an earlier version.

We're open to completely reorganizing this but the pipeline idea seemed to greatly reduce the amount of manually project creation and deletion. Is there a better way to track version release "pipeline"? In particular, we will have multiple versions in this pipeline simultaneously in the future due to fixes or patches to older versions. We don't see how to do that yet, except to create new pipeline projects for each version which is a real hassle.

Here's some background details:

The TickZoom application has some very complete unit tests some of which simulates real time trading environments. Add to that TickZoom makes elaborate use of parallelization for leveraging multi-core computers. Needless to say, during development of a new version, there can be stability issues during integration testing which get uncovered by running the build and auto tests repeatedly. A version which builds and tests cleanly 8 times in a row without change plus has undergone some real world testing by users can be deemed "stable" and promoted to the stable branch.

Our Hudson projects look like this:

test - Only for testing a build, zero user visibility. integrate_deploy - Promotes a test project build to integrate branch and makes it available to public for UA testing. integrate - Repeatedly builds the integrate branch to determine if it's stable enough to promote to stable branch. This runs the builds and test hourly throughout every night. stable_deploy - Promotes an integrate project build to the stable branch and makes it public for users who want the latest and greatest. stable - Builds the stable branch once every night. After 2 weeks of successful builds (14 builds) it can go to "release candidate".

And so on... it continues with "release candidate" and then "release".

Answer 1

I can see the point of demonstrating stability by having multiple successive builds succeed without error, but I'd suggest a slightly different approach to make things more simple. Rather than trying to aggregate the results of multiple builds to determine whether you promote the latest build to the stable branch, run your tests 8 times against the same build; you can either do this by adding a repeat count parameter to the tests, or just repeat the test steps multiple times in the Hudson job setup.

If the build passes cleanly every time, you could use that as a gateway to send the build to your users for "real world" testing before you promote it to the stable branch.

This has a couple of advantages; it makes the Hudson setup more simple as per your request, and it gives you added confidence in the stability of the build because you're running the tests multiple times against the same code base, rather than against a different code base each time.

Answer 2

I guess you have to either implement some solution outside of hudson, that produces trigger files to be used in Hudson or you extend the promotion plugin with your company specific rules.

Answer 3

The answer is to create a separate pipeline of jobs for each new minor version of the software.

So they'll be like this.

integrate_0.8.3 stable_0.8.3 candidate_0.8.3 release_0.8.3

We will use the Hudson API to generate the jobs for each new version with the script.

The promotion can't be totally automated because other factors than stable builds like user reported errors can delay a version from moving through the pipeline.

sincerely, Wayne