Starting new Docker container with every new Bamboo build run and using the container to run the build in

Question

I am new to Bamboo and are trying to get the following process flow using Bamboo and Docker:

1. Developer commits code to a Bitbucket branch
2. Build plan detects the change
3. Build plan then starts a Docker container on a dedicated AWS instance where Docker is installed. In the Docker container a remote agent is started as well. I use the atlassian/bamboo-java-agent:latest docker container.
4. Remote agent registers with Bamboo
5. The rest of the build plan runs in the container
6. Container and agent gets removed when plan completes

I setup a test build plan and in the plan My first task is to start a Docker instance like follows:

sudo docker run -d --name "${bamboo.buildKey}_${bamboo.buildNumber}" \
   -e HOME=/root/ -e BAMBOO_SERVER=http://x.x.x.x:8085/ \
   -i -t atlassian/bamboo-java-agent:latest

The second task is to get the source code and deploy. 3rd task is test and 4th task is shutting down the container.

There are other agents online on Bamboo as well and my build plan sometimes uses those and not the Docker container that I started as part of the build plan.

Is there a way for me to do the above?

I hope it all makes sense. I am truly new to this and any help will be appreciated.

Manicdepressive

Is it necessary to start and stop the bamboo agent for every build? Normally the extern agent is running all the time (in this example in docker) and running the build jobs as usually. This doesn't need any further docker handling. Or do you have any advantages with this procedure? – Fellner

@Fellner the problem is when the docker image is changed so the fingerprint is not valid anymore and you must inform your bamboo admin to accept the modified docker remote agent, really annoying Jacobus van heerden do you found a solution? – Tailband

Answer 1

We (Atlassian Build Engineering) have created a set of plugins to run Docker based agents in a cluster (ECS) that comes online, builds a single job and then exits. We've recently open sourced the solution.

See https://bitbucket.org/atlassian/per-build-container for more details.

Lebna

we implemented the same but we are facing issues as the agents take around 5-7min to come online and then start the build, I would like to know how can i reduce the time to register an agent in PBC and can it utilise the same agent if the same build has ran. – Lloyd

@AbhijeetKamble hit me at [email protected] with more details about your setup. We track our startup times with datadog and the agent typically starts up around a minute (unless the cluster is scaling up). there's defo a penalty of about 20+ seconds for bamboo agent java agent startup. – Lebna

Thanks for your reply, will send you the complete details on email. – Lloyd

Answer 2

first you need to make sure the "main" docker container is not exiting when you run it.

check with

docker ps -a

you should see it is running

now assuming it is running you can execute commands inside the container

to get into the container

docker exec -it containerName bash

to execute commands inside the container from outside the container

docker exec -it containerName commandToExecuteInsideTheContainer

you could as part of the containers dockerfile COPY a script in it that does something.

Then you can execute that script from outside the container using the above approach.

Hope this gives some insight.

Hubie