How to test and auto deploy a web (or any) project with Docker and docker-compose and Heroku

If you use Docker and docker-compose in your project, it’s really simple to run it on bitrise.io and have fully automated tests and deploy for the project.

A base template bitrise.yml / bitrise build config, for automatic tests and deploys, with tagged releases:

format_version: "2"
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
trigger_map:
- push_branch: master
  workflow: deploy-to-heroku
- pull_request_source_branch: '*'
  workflow: ci-with-docker-compose
workflows:
  _ci_prepare:
    steps:
    - activate-ssh-key@3.1.1: {}
    - git-clone@3.4.2: {}
  _tag-with-build-number:
    steps:
    - script@1.1.3:
        title: git tag with BITRISE_BUILD_NUMBER
        inputs:
        - content: |-
            #!/bin/bash
            set -ex

            if [ -z "${BITRISE_BUILD_NUMBER}" ] ; then
              echo " [!] No BITRISE_BUILD_NUMBER defined!"
              exit 1
            fi

            tag_name="b${BITRISE_BUILD_NUMBER}"
            git tag "${tag_name}"
            git push origin : "${tag_name}"
  _tag-with-deployed:
    envs:
    - TAG_TO_APPLY: deployed
    steps:
    - script@1.1.3:
        title: git tag with "deployed" - move the tag
        inputs:
        - content: |-
            #!/bin/bash
            set -ex

            tag_name="${TAG_TO_APPLY}"
            git tag --force "${tag_name}"
            git push --force origin : "${tag_name}"
  ci-with-docker-compose:
    before_run:
    - _ci_prepare
    after_run:
    - test
  deploy-to-heroku:
    before_run:
    - _ci_prepare
    - _tag-with-build-number
    after_run:
    - _tag-with-deployed
    steps:
    - heroku-deploy@0.9.3: {}
  test:
    steps:
    - script:
        title: run tests with docker-compose
        inputs:
        - content: |
            #!/bin/bash
            set -ex
            # preboot containers, so that the DB is ready when the tests start (docker-compose bug)
            docker-compose run --rm app sleep 1
            # run tests / full ci test suit (including integration tests)
            docker-compose run --rm app RUN TEST COMMAND
    - script:
        title: cleanup
        is_always_run: true
        inputs:
        - content: |
            #!/bin/bash
            set -ex
            docker-compose stop

What does this config do?

Workflows

There are three “runnable” workflows and three “utility” workflows in this config. Utility workflows are the ones which start with an underscore, and those workflows are not meant to be executed directly, only as part of another workflow (through before_run and after_run - you can find more info about workflow chaining on our DevCenter).

The utility workflows

  • _ci_prepare is simply the common part of the builds, it includes the steps to retrieve the code.
  • _tag-with-build-number tags the current commit with the BITRISE_BUILD_NUMBER
  • _tag-with-deployed tags the current commit with a deployed tag - moves the tag if that already exists

Workflow: test

This workflow can be executed locally too, with the open source Bitrise CLI! To run it locally just save the bitrise.yml into the repository, and run bitrise run test (after installing the Bitrise CLI of course ;)).

In short, this workflow is responsible for executing a docker-compose run, where docker-compose will create all the service docker containers (database, redis, etc.) automatically, and then it runs RUN TEST COMMAND (where RUN TEST COMMAND can be any command you run your tests with, e.g. in case of Go" go test ./..., or in case of Rails/Ruby: bundle exec rspec spec, …) in the “main” docker container (called app in the example).

An example docker-compose.yml we use for our API project:

version: '2'
services:
  db:
    image: postgres:9.4.4
    ports:
      - "5432:5432"
  app:
    build: .
    volumes:
      - .:/src
    ports:
      - "3001:3001"
    links:
      - db:postgres
    environment:
      PORT: 3001
      DB_HOST: postgres
      DB_USER: postgres
      DB_PSW: postgres
      DB_NAME: bitriseapitest
      DB_SSL_MODE: disable

Workflow: ci-with-docker-compose

This workflow doesn’t have any steps, it basically just runs _ci_prepare and then the test workflow. This workflow is the one you want to run in a CI environment like bitrise.io, as it makes sure that the code is available before it’d continue with the test workflow. You could of course run this locally too, but in local you most likely don’t want to clone the code every time :wink:

Note: running ci-with-docker-compose locally on your Mac in this form - as it is in the example - is “safe”, it won’t clone the repository, as both the Activate SSH Key and the Git Clone steps have a default flag to only run in “CI mode”. But our experience is that it’s usually better on the long run to have a “test” workflow, which just runs the tests, and a “wrapper” workflow for CI, which prepares the code in an empty (CI) environment and then runs the “test” workflow.

Workflow: deploy-to-heroku

This is a really simple workflow which performs, well, a deploy to Heroku :wink:

OK, OK, it does a little bit more; it tags the deployed commit. How that works: it tags the commit before the deploy with a “build number” tag, e.g. b11, then performs the deploy to Heroku and if that’s successful it moves the deployed tag to be applied on the exact commit which was just deployed to Heroku.

This way you can check and follow the deployment of the server in your git history directly. Every previous deploy is marked with a bX tag, where X is the related build’s Bitrise.io build number (so you can quickly check the related build if you want to), and the current live state/commit is marked with a deployed tag, which is moved after every successful deploy.

Note: don’t forget to set HEROKU_API_TOKEN and HEROKU_APP_ID in your Secret Env Vars!

The trigger_map - continuous everything

Now that we have our CI/test (ci-with-docker-compose) and deploy (deploy-to-heroku) workflows, as well as our “local test” workflow which you can run on your own Mac/Linux (test), it’s time to specify the Triggers to do continuous testing as continuous deployment.

This is the trigger_map we use for our API project (copied from the example above, just so you don’t have to scroll up ;)):

trigger_map:
- push_branch: master
  workflow: deploy-to-heroku
- pull_request_source_branch: '*'
  workflow: ci-with-docker-compose

What this does is:

  1. For every time a PR is opened or updated, it runs a build with the ci-with-docker-compose workflow.
  2. And every time the master branch is updated it runs a build with the deploy-to-heroku workflow.

In addition to this we enabled GitHub’s protected branch feature for the master branch, as well as the required CI pass check before a PR could be merged, and we also enforce that a PR can only be merged if it’s “up to date” before the merge. Basically enabled everything what you can on GitHub for the protected master branch :wink:

With this setup our dev workflow works like:

  1. We work on feature/ branches
  2. Once the feature is ready for review/merge we start a PR, which start a CI build
  3. The PR can’t be merged unless the CI build passes
  4. A PR review is also required
  5. Once the PR / CI build passes and someone adds an approval review on GitHub, we merge the PR (if in the meantime another PR was merged and this PR is no longer “up to date” we re-run the CI build to ensure that it still passes - this is also enforced by GitHub before the PR could be merged)
  6. The merge into master automatically triggers a deployment and tags the deployed commit

In short, the only manual things we have to do:

  1. Write the code
  2. Start a PR when it’s ready for deploy
  3. Do a manual code review and if approved merge the PR

That’s all, everything else is handled automatically. Now you know how the Bitrise API project is configured and how we work on it.

If you have any questions feel free to leave a comment below!
Happy Building!