In this article we’ll look at how to automate the deploy of services to your docker 1.12 swarm. If you’re interested in automating the deploy of a dockerized application for Continuous Deployment or highly automatic workflows then this is a must-read.

Stacks are coming soon to docker but until they get into the stable version of docker the process of deploying services is a little more complicated, but it’s still not hard to do. While we wait on stacks we’re going to write a simple script to help us deploy. When stacks come natively to docker we will be able to deploy multiple services with one docker command and so we won’t need our script anymore, but for now a script will solve the small nuisances and make it much easier to deploy repeatably and automatically.

Prerequisites

First you’ll need to have your image in a docker registry, so you’ll already have set up your docker build and push. If you haven’t automated the build and push yet, have a look at NimbleCI which does this easily and automatically.

Next you’ll need a docker swarm running at least docker 1.12, we’ll deploy our services on this swarm.

Updating A Service Using Its Image Hash

In the simplest sense our objective when deploying is to update a service definition to tell the docker engine to use a different (newer) image, instead of the old image. If you’re unfamiliar with services and how they work you can read more about them here and here. There are a few complications however. The first complication is that there is a bug in 1.12 that doesn’t pull newest version of tag when applying an update. This is better illustrated with an example:

# Start the service, this will ensure 1 pinger container is
# up at all times. The latest version of the alpine image is used.
$ docker service create --name pinger --replicas 1 alpine:latest ping google.com

# Now let's imaging that a newer version of the alpine image is
# released. We'd like to update our image, so we do this (doesn't work):
$ docker service update --image alpine:latest pinger

As you can see the image is not pulled to see if the tag has a new version when doing an update of the image. This is being worked on and will probably be fixed soon.

The workaround until this is fixed is to specify the SHA digest of the exact image that you want to deploy. Think of it as specifying the exact GIT commit that you want to deploy. By specifying the digest the docker engine will always pull the exact image when we deploy.

Here’s an example of what we’re going to do:

# Here we specify the sha digest instead of 'latest'.
$ docker service create --name pinger --replicas 1 alpine@sha256:7cceaf9b93a7ac5fc627739ed5d059d53672d294eb2c6394d009a0704fdc4734 ping google.com

# When a new image is available, we can specify the new digest like so:
$ docker service update --image alpine@sha256:ca7b185775966003d38ccbd9bba822fb570766e4bb69292ac23490f36f8a742e pinger

The sha digest can be found in the output of a pull:

$ docker pull alpine
Using default tag: latest
latest: Pulling from library/alpine
c0cb142e4345: Already exists 
Digest: sha256:ca7b185775966003d38ccbd9bba822fb570766e4bb69292ac23490f36f8a742e
Status: Downloaded newer image for alpine:latest

You can also find it by inspecting the image:

$ docker inspect -f '{{ .RepoDigests }}' alpine:latest
[alpine@sha256:ca7b185775966003d38ccbd9bba822fb570766e4bb69292ac23490f36f8a742e]

A Script To Update The Entire Stack

automate-all-the-things

We know how to update a service on a swarm, we just use the docker service update command. However we don’t want to run this manually on the server each time we deploy, our aim is to automate it. Since a stack is usually made up of more than one service, we’ll write a simple script which will contain the update commands and the definition of the services.

To address the problem with the digests and the tag not being pulled (explained above) the script will accept the SHA digest as an argument.

#!/bin/bash
#
# deploy-page-hit-counter.sh
#
# Usage:
#   ./deploy-page-hit-counter.sh <digest>
#
# Where:
#   <digest> is the SHA digest of the image, eg
#       "5e497f9794d9e1a64ade0ed97bd2b673bb76b5756ba188fc797b517738941210"

DIGEST=$1

# Update the redis service
docker service update \
        --image redis@sha256:38e873a0db859d0aa8ab6bae7bcb03c1bb65d2ad120346a09613084b49185912 \
        redis

# Update the page-hit-counter service
docker service update \
        --image emmetog/page-hit-counter@sha256:$DIGEST \
        page-hit-counter
        
# Our stack is simple (just a webapp and a redis), if
# your stack has more services, add them here.

The script is pretty simple, it just updates the services and makes sure that the images are up to date.

There is one last complication though: this script will fail if we run it on a swarm where the services are not initially created. To avoid these errors, let’s check if the service exists and create it if not:

#!/bin/bash
#
# deploy-page-hit-counter.sh
#
# Usage:
#   ./deploy-page-hit-counter.sh <digest>
#
# Where:
#   <digest> is the SHA digest of the image, eg
#       "5e497f9794d9e1a64ade0ed97bd2b673bb76b5756ba188fc797b517738941210"

DIGEST=$1

# Ensure redis service is running
SERVICES=$(docker service ls --filter name=redis --quiet | wc -l)
if [[ "$SERVICES" -eq 0 ]]; then
    docker service create \
            --name redis \
            --network my-net \
            --restart-condition any \
            --restart-delay 5s \
            --update-delay 10s \
            --update-parallelism 1 \
            redis@sha256:38e873a0db859d0aa8ab6bae7bcb03c1bb65d2ad120346a09613084b49185912
fi

# Update the redis service
docker service update \
        --image redis@sha256:38e873a0db859d0aa8ab6bae7bcb03c1bb65d2ad120346a09613084b49185912 \
        redis

# Ensure page-hit-counter service is running
SERVICES=$(docker service ls --filter name=page-hit-counter --quiet | wc -l)
if [[ "$SERVICES" -eq 0 ]]; then
    docker service create \
            --name page-hit-counter \
            --network my-net \
            --restart-condition any \
            --restart-delay 5s \
            --update-delay 10s \
            --update-parallelism 1 \
            emmetog/page-hit-counter@sha256:$DIGEST
fi

# Update the page-hit-counter service
docker service update \
        --image emmetog/page-hit-counter@sha256:$DIGEST \
        page-hit-counter
        
# Our stack is simple (just a webapp and a redis), if
# your stack has more services, add them here.

Now we have a script that we can run to deploy our stack of services. Run this on any swarm node using ansible/puppet/chef or your favourite provisioning tool to make your life easier.

If your stack has more services you can of course add them in.

Wrapping Up

It’s easy to see how this script will be replaced by docker stacks when they come, our script is just a definition of all of the services in our stack. However our script solves some of the problems that version 1.12 of docker has, for example our script is idempotent (we can run it again and again with the same digest and nothing will change, even if the service doesn’t exist on the swarm). The docker service create and docker service update commands are not idempotent in this way, since an update will fail if the service does not exist. Also, by explicitly specifying the digests we’ve got around the bug where the latest image is not pulled even if a newer image exists for the latest tag.

With this script you can automate the deploy of a stack to a swarm, you don’t have to wait for native docker stacks. When docker stacks do become native you will be able to replace this script with a command like docker deploy -f my-stack.json my-stack.

I hope you found this useful! Leave your comments below!