Docker Stacks and Distributed Application Bundles (we’ll call them “bundles” in this post) are a way to describe multiple services in one single file.

Why We Need Bundles

So why do we need them? Imagine you had a simple application that required two services: a redis database and your application container. When it comes to deploying this stack to a non-dev environment you’re probably doing something like this right now:

#!/bin/bash

# Ensure redis service is running
SERVICES=$(docker service ls --filter name=redis --quiet | wc -l)
if [[ "$SERVICES" -gt 0 ]]; then
    # Update the service
    docker service update \
            --image redis:3.0.5 \
            redis
else
    docker service create \
            --name redis \
            --network my-net \
            --restart-condition any \
            --restart-delay 5s \
            redis:3.0.5
fi

# Ensure page-hit-counter service is running
SERVICES=$(docker service ls --filter name=page-hit-counter --quiet | wc -l)
if [[ "$SERVICES" -gt 0 ]]; then
    # Update the service
    docker service update \
            --image emmetog/page-hit-counter:latest \
            page-hit-counter
else
    docker service create \
            --name page-hit-counter \
            --network my-net \
            --restart-condition any \
            --restart-delay 5s \
            emmetog/page-hit-counter:latest
fi

Two things here: first off you’ll notice that we have to run a separate service command for each service in our stack. If we have lots of services in the stack then maintaining this file can become a bit unwieldy and error prone.

Second, the ifs. These are needed because there is currently no idempotent way to ensure the state of a service is up to date if you don’t know if it already exists or not, or put another way, there is no way to say “I want the state of the service like this, if the service doesn’t exist then create it and if it does exist make sure the state matches what I want”. This is a bit of a pain but thankfully bundles will make this easier since the command to deploy/update a stack will be idempotent.

The idea behind bundles is that we will be able to specify all of the services of our stack in one single file, replacing our hacked together bash script above and making it easier to deploy all of the services in one command. Put simply:

Stacks will make it easier to deploy your application services and remove the need to maintain a bash script to define services.

Let’s take these bundles for a spin and generate a bundle file and deploy it to a Swarm.

What about Docker Compose?

A lot of people wonder why we need stacks when we already have Docker Compose. The most important difference is that docker-compose defines containers while stacks define services.

Services

Compose has no notion of the new “services” concept in Docker 1.12. In other words when you run a docker-compose up command, docker-compose will ensure that the state of the containers is correct in the moment that the command is run. If a container needs to be brought up because it’s not already up right now, then it will be done. However compose does not monitor the state continuously, so if a container dies unexpectedly in 5 minutes time then composer won’t restart the container (unless you run docker-compose up again).

The concept of services introduced in Docker 1.12 allows the docker engine to know about the state of the services that we want up at all times. Now that the engine knows about the desired state, it can take immediate measures to correct things when the actual state differs. If a container dies unexpectedly at any time, the engine can bring another one up automatically. If an entire node goes down taking a few containers with it, the engine detects it and can bring these containers back up on a healthy node.

Stacks are very similar to docker-compose except they define services while docker-compose defines containers. Stacks allow us to tell the docker engine the definition of the services that should be running, so the engine can monitor and orchestrate the services.

For more info on services and why they’re great, read this.

A Note on ‘Experimental’

experimental

At the moment these features are experimental so you’ll need to install the experimental version of docker-engine if you want to use them. To install the experimental version of Docker run this:

$ curl -fsSL https://experimental.docker.com/ | sh

You’ll also need docker-compose 1.8 in order to generate the bundles from your existing docker-compose.yml files.

Since the feature is experimental you will have to distribute the bundle files yourself for now, there is no support in the docker registry to distribute the bundles yet, although it is coming. If you have the experimental build of docker-engine installed you will be able to do everything with bundles, but for now you’ll have to copy the bundle file to your servers manually.

Making a bundle

Since a bundle file is just a definition of a set of services you can write one by hand if you really want to (you’ll see what the bundle file look like later). However it’s far easier to generate a bundle file from an existing docker-compose.yml file.

Let’s take an example of this docker-compose.yml file:

version: "2"
services:
  redis:
    image: redis:3.0.5
    networks:
      - my-net

  app:
    image: emmetog/page-hit-counter
    ports:
      - "127.0.0.1:80:5000"
    networks:
      - my-net
      
networks:
  my-net:

Now we’ll generate a bundle file from this:

$ docker-compose --file docker-compose.yml bundle

In my case this creates a file called page-hit-counter.dab which is the bundle file.

A bundle file contains almost the same information that a docker-compose.yml file contains. Let’s have a look at the generated bundle file.

{
  "Services": {
    "app": {
      "Image": "emmetog/page-hit-counter@sha256:d1264d35545b05fa704902be42ca7ff0ff7b731889a834fbfafc34dde9b8d7cb", 
      "Networks": [
        "my-net"
      ], 
      "Ports": [
        {
          "Port": 5000, 
          "Protocol": "tcp"
        }
      ]
    }, 
    "redis": {
      "Image": "redis@sha256:f8829e00d95672c48c60f468329d6693c4bdd28d1f057e755f8ba8b40008682e", 
      "Networks": [
        "my-net"
      ]
    }
  }, 
  "Version": "0.1"
}

Running The Stack

In the experimental version of docker-engine you’ll find a new docker deploy command. This command is used to create new service stacks and update existing stacks.

You’ll also need to have docker-engine in swarm mode before you can deploy stacks.

Let’s deploy the stack from the bundle file that we just generated:

$ docker deploy --file page-hit-counter.dab page-hit-counter

Let’s have a look to see which services were created:

$ docker service ls
ID            NAME                    REPLICAS  IMAGE                                                                                             COMMAND
8paljh76w7cv  page-hit-counter_redis  1/1       redis@sha256:f8829e00d95672c48c60f468329d6693c4bdd28d1f057e755f8ba8b40008682e                     
dp3xbcqw63y8  page-hit-counter_app    1/1       emmetog/page-hit-counter@sha256:d1264d35545b05fa704902be42ca7ff0ff7b731889a834fbfafc34dde9b8d7cb

Awesome!

Managing stacks

Docker has a few new commands to manage stacks, although these could change while the feature is in experimental:

$ docker stack --help

Usage:  docker stack COMMAND
  
Manage Docker stacks
  
Options:
      --help   Print usage
  
Commands:
  config      Print the stack configuration
  deploy      Create and update a stack
  rm          Remove the stack
  services    List the services in the stack
  tasks       List the tasks in the stack

Run 'docker stack COMMAND --help' for more information on a command.

Right now stacks are managed from the stack files themselves in the sense that all of the commands to manage stacks in docker-compose take the definition of the stack from the file before performing the action. In other words, if you deploy a stack from a bundle file and then you delete the bundle file, you won’t be able to use the docker stack rm command, you’ll have to remove each service individually instead. For now, keep those bundle files around somewhere handy.

Wrapping Up

When it comes to deploying multi-service application stacks, Docker Stacks will make our lives even easier.

It seems like the people behind Docker also intend on making it easier to distribute these bundle files in the future, so we might see it become very easy to spin up a LAMP stack or a MEAN stack for example, we would just download the bundle file from some central “bundle repository” and spin it up.

I hope this was useful! Feel free to post your thoughts in the comments.