In a previous article we deployed a Jenkins instance in Docker using Ansible. However after the Jenkins is deployed we had to configure it manually (Jenkins was a pet).

You’ll love this post if you want all your Jenkins configuration in version control so you can deploy Jenkins repeatably and reliably and you can treat your Jenkins as “disposable”.

Since we’re big Docker fans, we’ll deploy our “disposable jenkins” in a docker container, although the concept is the same no matter how you deploy your Jenkins.

Pets vs Cattle

You’ve probably heard about the pets vs cattle analogy in system administration. It says that you should treat your servers as cattle instead of like pets. You should not have servers that have cute names and need to be cared for and nurtured. Instead, if a server misbehaves just replace it with a new one. This makes scaling and managing servers much easier.

Up to now we’re used to treating our Jenkins instances as “pets”. We deploy a Jenkins instance somewhere and then carefully configure it until it’s in the right shape. We constantly need to be monitoring it and taking backups because if one day Jenkins decides to fall over and die then we’re in big trouble, we don’t want to have to redeploy a new Jenkins and spend all that time reconfiguring everything.

Luckily, there is an alternative: we can move all of the configuration of Jenkins into source control. This means that when we deploy Jenkins it is deployed fully configured, so we can treat the instances as cattle instead of pets.

This approach has advantages and disadvantages so it might not fit all situations. Let’s look at them now.

Advantages

Some of the advantages of moving the Jenkins configuration to source control are:

  • You can redeploy a Jenkins instance fully configured at the touch of a button (Jenkins is a cow instead of a pet :P)
  • Changes to Jenkins configuration will be in a VCS repository, so every change is recorded and can be traced in the same way as changes to any other VCS project.
  • You don’t need any backups, since you can redeploy a new identical Jenkins instance at any time.
  • You can now deploy multiple Jenkins instances in different environments. Devs could deploy the fully configured jenkins to their local machines for example, allowing them to run any job at any time on their own machines.

Disadvantages

As with all things, this approach has it’s disadvantages too:

  • Changes to the configuration are not as easy as just changing the setting in the Jenkins UI. Depending on the complexity of the change you might have to make the change in the Jenkins UI, see what changed in the underlying configs, copy this change to your provisioning code and then redeploy all Jenkins instances to put the changes into effect.
  • We might lose the job build histories when we redeploy (only if we wipe the old Jenkins and redeploy a fresh, but treating Jenkins as cattle implies that this can happen at any time). If keeping the build histories intact is very important to you then you’ll have to solve this yourself or keep treating Jenkins as a pet.

Overview of the Pieces

The concept is simple: we’re going to provision the Jenkins configuration files onto the Jenkins server using our favourite provisioning tool. In this post we’ll use Ansible but you can just as easily use your own favourite provisioning tool.

Ansible will install a jenkins server. We love Docker so our Jenkins will be in a docker container. You could easily install a jenkins server without docker too, just tweak in the right places.

Once the jenkins server is running, we’ll install any plugins that we need using the Jenkins remote API. Finally we’ll template accross our pre-configured configuration files and we’ll reload Jenkins so it loads the latest configs.

I’ve put all these steps into an ansible galaxy role so if you are using Ansible you have a lot less work to do :) If you’re not using Ansible then you can replicate the role using your favourite tool.

That’s the big picture, let’s jump in to the details!

Requirements

For this tutorial you’ll need:

  • Ansible installed locally
  • Docker installed on the server you want to deploy Jenkins to

Install The Ansible Role

First, we’ll install the ansible role that will do most of the work for us. Let’s imagine that we already use Ansible and we have a GIT repo called ansible-provisioner where we store all of our playbooks and ansible configuration that we have created up to now. To give you an idea of the directory structure as we go along the tutorial, at the beginning our directory structure might look like something like this (this is the basic structure of an Ansible project):

$ tree ansible-provisioner
provisioner/
├── inventory/
│   └── ...
├── roles/
│   └── ...
├── some-playbook.yml
└── another-playbook.yml

Let’s install the galaxy role:

$ ansible-galaxy install emmetog.jenkins

This role will deploy a Jenkins server and it will allow us to specify the exact XML configuration files that we want for the global Jenkins configuration as well as for each job.

The Playbook

First up we’ll create the playbook. Create a new playbook called deploy-jenkins.yml with this inside:

- hosts: 127.0.0.1

  vars:
    jenkins_version: "1.642.4"
    jenkins_url: http://127.0.0.1
    jenkins_port: 8080
    jenkins_install_via: "docker"
    jenkins_jobs: [
        "my-first-job",
      ]
      
  roles:
    - emmetog.jenkins

To get started we’ll deploy to localhost on port 8080, you can change this as needed. For a full list of the available configuration options take a look at the roles documentation.

Jenkins Configuration Files

Next up let’s create a simple XML file containing our first shot at the global configuration of Jenkins.

<?xml version='1.0' encoding='UTF-8'?>
<hudson>
    <disabledAdministrativeMonitors/>
    <version></version>
    <numExecutors>2</numExecutors>
    <mode>NORMAL</mode>
    <useSecurity>true</useSecurity>
    <authorizationStrategy class="hudson.security.AuthorizationStrategy$Unsecured"/>
    <securityRealm class="hudson.security.SecurityRealm$None"/>
    <disableRememberMe>false</disableRememberMe>
    <projectNamingStrategy class="jenkins.model.ProjectNamingStrategy$DefaultProjectNamingStrategy"/>
    <workspaceDir>${ITEM_ROOTDIR}/workspace</workspaceDir>
    <buildsDir>${ITEM_ROOTDIR}/builds</buildsDir>
    <jdks/>
    <viewsTabBar class="hudson.views.DefaultViewsTabBar"/>
    <myViewsTabBar class="hudson.views.DefaultMyViewsTabBar"/>
    <quietPeriod>5</quietPeriod>
    <scmCheckoutRetryCount>0</scmCheckoutRetryCount>
    <views>
        <hudson.model.AllView>
            <owner class="hudson" reference="../../.."/>
            <name>All</name>
            <filterExecutors>false</filterExecutors>
            <filterQueue>false</filterQueue>
            <properties class="hudson.model.View$PropertyList"/>
        </hudson.model.AllView>
    </views>
    <primaryView>All</primaryView>
    <slaveAgentPort>50000</slaveAgentPort>
    <label></label>
    <nodeProperties/>
    <globalNodeProperties/>
</hudson>

This is just the first iteration so it doesn’t have to be completely correct, we’ll update it again later after tweaking Jenkins.

Next up we’ll create the initial credentials.xml file, let’s put this in for starters:

<?xml version='1.0' encoding='UTF-8'?>
<com.cloudbees.plugins.credentials.SystemCredentialsProvider plugin="credentials@1.24">
    <domainCredentialsMap class="hudson.util.CopyOnWriteMap$Hash">
        <entry>
            <com.cloudbees.plugins.credentials.domains.Domain>
                <specifications/>
            </com.cloudbees.plugins.credentials.domains.Domain>
            <java.util.concurrent.CopyOnWriteArrayList>

                <com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey plugin="ssh-credentials@1.12">
                    <scope>GLOBAL</scope>
                    <id>github-deploy-key-jenkins</id>
                    <description>github-deploy-key-jenkins</description>
                    <username>git</username>
                    <passphrase></passphrase>
                    <privateKeySource class="com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey$DirectEntryPrivateKeySource">
                        <privateKey></privateKey>
                    </privateKeySource>
                </com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey>

            </java.util.concurrent.CopyOnWriteArrayList>
        </entry>
    </domainCredentialsMap>
</com.cloudbees.plugins.credentials.SystemCredentialsProvider>

The emmetog.jenkins role expects these files to be in ./jenkins-configs/ by default. After you create these files your directory structure should look like this:

$ tree ansible-provisioner
provisioner/
├── jenkins-configs/
│   ├── config.xml
│   └── credentials.xml
├── inventory/
│   └── ...
├── roles/
│   ├── emmetog.jenkins/
│   │   └── ...
│   └── ...
├── some-playbook.yml
└── another-playbook.yml

In our deploy-jenkins.yml playbook we also specified one job to be created, let’s add the first configuration for this now. Create a new file in ./jenkins-configs/jobs/my-first-job.xml with this inside:

<?xml version='1.0' encoding='UTF-8'?>
<project>
  <actions/>
  <description>My first job, it says "hello world"</description>
  <keepDependencies>false</keepDependencies>
  <properties/>
  <scm class="hudson.scm.NullSCM"/>
  <canRoam>true</canRoam>
  <disabled>false</disabled>
  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
  <triggers/>
  <concurrentBuild>false</concurrentBuild>
  <builders>
    <hudson.tasks.Shell>
      <command>echo &quot;Hello World!&quot;</command>
    </hudson.tasks.Shell>
  </builders>
  <publishers/>
  <buildWrappers/>
</project>

Now our directory structure should look like this:

$ tree ansible-provisioner
provisioner/
├── jenkins-configs/
│   ├── jobs/
│       └── my-first-job.xml
│   ├── config.xml
│   └── credentials.xml
├── inventory/
│   └── ...
├── roles/
│   ├── emmetog.jenkins/
│   │   └── ...
│   └── ...
├── some-playbook.yml
└── another-playbook.yml

Now we can run the playbook to deploy Jenkins using our configuration files:

$ ansible-playbook playbooks/jenkins.yml

That’s it! If you have a look at http://127.0.0.1:8080 you should see the new Jenkins which has been fully configured, including the jobs.

Making Changes

Now we have a working Jenkins but it’s not exactly configured the way we want it, we just put placeholder XML in to begin with. Now, what happens if we want to make a change in the configuration? To do this just make the change in the Jenkins UI first and then SSH into the server and copy the resulting XML back and paste it into your VCS. The next time you deploy jenkins the new configs will overwrite the old ones.

Adding a new job follows a similar process, just add the new job manually in Jenkins first, then copy the config.xml file that Jenkins created and put this in the ./jenkins-configs/jobs/ so that this job will be deployed and kept up to date every time you deploy.

Wrapping Up

This concept of keeing the configuration in version control and not treating Jenkins as a “pet” has big advantages, especially as your team and company grow larger, but it should only be done when the extra work of bringing any changes back into the configs in the code is worth it.

I hope you found this post useful, if you have any questions, comments or suggestions please feel free to leave them below.