"Serialize" Bamboo builds?

Question

We are using Bamboo v3.1.1 as our continuous integration build server, and it works quite well - most of the time.

One issue we're having is that we're doing a fair amount of database-oriented testing, e.g. the builds do some of their unit and integration tests on a shared database instance.

This causes issues when we happen to have multiple Bamboo builds for the same build plan running at the same time - they're stumbling over each other's feet and cause deadlocks and usually, all builds involved will fail due to this.

So while parallel builds are great - in theory - we'd really like to be able to define a build plan to "serialize" the builds, e.g. never execute more than one build in parallel.

Does anyone know how can we do this?? Is there a setting to tell Bamboo "don't parallelize this build plan - just do one build at a time, in a serial fashion"

Update:

My build process currently has two stages:

• core build (building VS solution, updating test database to latest scripts)
• testing (NUnit 2.4)

The "core build" can easily be run multiple times in parallel - no problems there. However, the "Testing" stage cannot be run more than once since some of those tests access the one and only shared "unit test" database; if more than 1 "testing" stage process are running, they'll end up deadlocking each other.

So how do I tell Bamboo it's OK to parallelize the "core build" stage, but for the "testing", always only run one instance at a time, no matter how many builds are running??

Quotidian

Answer 1

My approach would be to put the core build in one plan, and the testing in another plan. The core build would trigger the testing plan as a child plan.

Then as soon as the core build finishes, a testing plan would spawn.

The core build plan presumably could be set to run multiple instances in parallel on many machines. The testing plan would be limited to a single instance of the plan running at once.

My only confusion is that you said:

• core build

  (building VS solution, updating test database to latest scripts)

Doesn't updating the test database cause a problem for a running testing plan?

Mic

Thanks for your suggestion - interesting. But the main question remains: HOW would I tell that testing plan that it can only run "one at a time" ?? Also: good point about the DB update - I'll have to investigate a bit more ... – Quotidian

Plans only run one at a time by default, unless you enable and increase the default number of concurrent builds server wide. Even in that case though, you can configure a plan to only run 1 instance of itself concurrently. It's under the miscellaneous tab of a plan. – Mic

interesting - thanks! Wonder why our Atlassian gurus never told me about this :-) – Quotidian

Geez - this is just way too simple.... but it works flawlessly !! Thanks so much - you've made my day (and my team's, too!) – Quotidian

Answer 2

Aye. There are two ways to do this: tasks and stages. My guess is that you want stages.

To have your builds running in parallel, you must have several jobs running inside one stage. If you take the jobs that can't be run in parallel and put them in separate stages, they will run serially. For example, if you have:

Test Foo Stage:
    Init Foo Database Job
    Hammer Foo Database Job
    Smash Foo Database Job
Test Baz Stage:
    Init Baz Database Job
    Bamboozle Baz Database Job
    Befuddle Baz Database Job

Then the Init/Hammer/Smash of the foo stage will run problematically in parallel. However, you can put each in its own stage:

Test Foo Init Stage:
    Init Foo Database Job
Test Foo Hammer Stage:
    Hammer Foo Database Job
Test Foo Smash Stage:
    Smash Foo Database Job
Test Baz Init Stage:
    Init Baz Database Job
Test Baz Bamboozle Stage:
    Bamboozle Baz Database Job
Test Baz Befuddle Stage:
    Befuddle Baz Database Job

Then, each task will run in serial, not parallel. Of course, this effectively limits you to a single useful agent.

If you really want to only use a single agent, you can always disable all but one agent, but that will affect all the builds, so that wouldn't be a good idea if you want anything to run in parallel.

And as a last comment, it is also possible to get where you want with tasks instead of stages. Concatenate the tasks from each Job and they will be run in serial by a single agent. Of course, each task will see the changed files and state from the previous task, so you'll want to make sure they won't interfere.

Cetane

As it suddenly occurs to me that my experience is with newer versions of Bamboo than you listed. You might have to preface my advice with "Upgrade to a newer version of Bamboo" to use some of those features. – Cetane

OK, so I would need to separate the "Build" from the "Test" stage - but how could I make sure when two processes start building at the same time, the two don't both go on into the "Test" stage?? We have 3-5 build agents available most of the time. Can I limit the "Test" stage to just one single build agent?? – Quotidian

No worries - Bamboo 3.1.1 already has stages, I know that term :-) – Quotidian

If the Test stage uses a different executable, you can remove that executable from list of Shared Capabilities for the agents and add it as an Agent Specific capability to a single agent. – Cetane