How to scale buildbot in a company

Question

I've been looking into buildbot lately, and the lack of good documentation and sample configurations makes it hard to understand how buildbot is commonly used.

According to the buildbot manual, each buildmaster is responsible for 1 code base. That means that a company that wants to use buildbot on, say, 10 projects, needs to maintain 10 different sets of buildbot installations (master-slaves configurations, open ports, websites with output etc.). Is this really the way things are done? Am I missing an option that creates a mash-up that is easy to maintain and monitor?

Thanks!

Answer 1

At my place of work we use Buildbot to test a single program over several architectures and versions of Python. I use one build master to oversee about 16 slaves. Each set of slaves pulls from a different repo and tests it against Python 2.X.

From my experience, it would be easy to configure a single build master to run a mash-up of projects. This might not be a good idea because the waterfall page (where the build slaves report results) can get very congested with more than a few slaves. If you are comfortable scrolling through a long waterfall page, then this will not be an issue.

EDIT:

The update command in master.cfg:

test_python26_linux.addStep(ShellCommand, name = "update pygr",
    command = ["/u/opierce/PygrBuildBot/update.sh","000-buildbot","ctb"], workdir=".")

000-buildbot and ctb are additional parameters to specify which branch and repo to pull from to get the information. The script update.sh is something I wrote to avoid an unrelated git problem. If you wanted to run different projects, you could write something like:

builder1.addStep(ShellCommand, name = "update project 1",
    command = ["git","pull","git://github.com/your_id/project1.git"], workdir=".")

 (the rest of builder1 steps)

builder2.addStep(ShellCommand, name = "update project 2",
    command = ["git","pull","git://github.com/your_id/project2.git"], workdir=".")

(the rest of builder2 steps)

The two projects don't have to be related. Buildbot creates a directory for each builder and runs all the steps in that directory.

Answer 2

FYI, BuildBot 0.8.x supports several repositories on one master, simplifying things a bit.