Test execution in parallel with Docker

Jenkins Docker delivery pipeline tests

The problem

Jenkins is good old CI where you can set up everything from simple cron job to complicated delivery pipeline. At the company where I’m working at the moment we ourselfes built an pretty-damn good Continuous Delivery pipeline which includes:

As I ordered above, tests takes from 30s (unit tests) to around 30 minutes (UAT). When taking in consideration all delivery pipeline action (code deployment, various procedures to be executed and so on) it becomes 45 minutes for a single release. So the math is 8hrs / 45min = up to 10 releases /day. That’s the physical limit. Well we are A-class team (A-for-Agile) so we need more release power.

Solution for Speeding up Continuous delivery

Parallel execution FTW!

As we have two types of unit tests (well, you know, legacy), we can securely execute them in parallel with no fear for clashes. The more difficult part is UAT. The best way to achieve things in our believe is incremental and not to invest too much time and get value faster, decided to bring Docker to out Cucumber testing suite.

Preparing Docker on Jenkins

The first thing – let’s make Jenkins server avare of Docker engine:

Afterwards, let’s verify docker is successfully installed and ready to do the heavy lifting:

sudo docker run hello-world

As you noticed, docker command has to be executed with sudo in front.

Docker container for testing (execution)

The first time we are going to use docker container, we need to build the “Tester” image, so I placed all the required files including Dockerfile itself inside our project’s repository infrastructure/tester:

build_tester.sh:

As simple as it can get! :)

Dockerfile:

As our UAT testing requires Cucumber and all the gems it requires (via Gemfile.lock), I did a small hack/improvement here. To make everything as fast as possible, when building the Docker image, I include bundle install alongside rake and bundler installation to achieve a Docker filesystem “layer”. Now when the time comes to run the container, it will still run bundle install but won’t install anything. It would install only if your actual Gemfile.lock differs from the one placed in tester’s directory (next to Dockerfile) Now current Gemfile.lock and all dependencies are included in Docker image. And if it DOES change, it would still work, but installs only new dependencies.

Inside our UAT directory, I placed script to execute tests in Docker container:

Most important file now – testing-ci.sh:

And to execute tests locally – testing-local.sh:

As we don’t use any other tools here (for now), we need a bit of Docker argument magic to do what’s needed:

  • Add host of our testing environment (can be omitted if hardcoding some environment like testing)
  • Mount volume, so that features and testing results would be accessible from outside the container ( -v pwd:/usr/src/app)

The docker run -it simply launches interactive terminal for you. You can do as minimal as docker run -d to execute in detached mode. Both bash files accept argument as a tests suite name (configured in Rakefile as a task). So simply:

Jenkins to Execute Jobs in parallel

As we are using Delivery pipeline Jenkins plugin, we can easily set up new job group (stage) and place all our jobs inside it.

Sample CD pipeline (incomplete):

Jenkins_ci_continuous_delivery

As in example, you can have as many groups as you wish (in another words, how much money/infrastructure you have).

Jenkins_trigger_builds

Here single Jenkins job triggers each group of User Acceptance tests and it can run in parallel, though you have to do one thing – aggregate results manually. Therefor I created Job called “Sync results”. How you do that, it’s up to your imagination (Jenkins plugin, not even using Jenkins or collecting output results from each Job execution).

In each job I added “Execute shell” step, where I added:

It’s important to add --login -i which makes shell interactive ant tty, otherwise you will end up banging your head into a wall and it wouldn’t work. In each UAT job only group of tests differs, ./tests-ci.sh api where api defines a group of tests to exeucte.

Conclusion

I managed to really fast set up ant run 2 groups of Acceptance (Cucumber) tests (4-5h). The only limitations I foresee are resources. Speed, dependency management – all great and easy. Dockerized approach adds much agility and power to how you want to shape your Continuous Integration. I think what I will do next is try to come up with a easy way to set up and run cheap and powerful Jenkins grid. Interested? Subscribe!

Parallel Testing Results

Coming Up!

C U,
ME

Leave a Reply

Your email address will not be published. Required fields are marked *