How to Run Puppet on Container Infrastructure using Docker

Docker is an Opensource Container based technology. It is giving us a workflow around containers which is much easy to use. Docker separates application from underlying operating system using container technology, similar to how Virtual Machines separate the operating system from underlying hardware.

Docker Container Vs Virtual Machines

The Virtual Machines includes applications, necessary binaries and libraries along with an entire guest operating systems which may weigh around 10s of GBs

While, the Docker Engine container comprises just the application and its dependencies. It runs as an isolated process in the user space on the host operating system, sharing the kernel with other containers. Thus, it enjoys the resource isolation and allocation benefits of VMs, but is much more fast, portable, scalable and efficient.

Docker Benefits

Scalability : These containers are extremely lightweight which makes scaling up and scaling down very fast and very easy to launch more containers as we need them or shut them down as we no longer need them.

Portablility : We can move them very easily. We're going to get into images and registries. But essentially, we can take  snapshots of  our environment and upload it to the public/private registry and then download that images for making containers of it anywhere.

Deployments : We can run  these containers almost anywhere  to deploy it namely Desktops, laptops, Virtual machines, Public/Private clouds etc.

In this article, I'm explaining how to install Docker on Ubuntu 1604 server and run Puppet inside a Docker container.

Installing Docker

It is supported in almost all operating Systems. To install Docker in a Ubuntu server, it requires a 64 bit architecture and a kernal version  of atleast or above 3.10. Let's start with the installation prerequisites.

Pre-requisites

Check the Kernel version and Architecture

We can use this commands to confirm the architecture and kernel version of our OS.

root@ubuntu:~# arch
x86_64
root@ubuntu:~# uname -r
4.4.0-21-generic

Now, next step is to update the APT repository packages. In addition, we need to ensure that it runs with https and install the required CA certificates. Run the following command to achieve this.

root@ubuntu:~# apt-get update

root@ubuntu:~# apt-get install apt-transport-https ca-certificates
Reading package lists... Done
Building dependency tree
Reading state information... Done
ca-certificates is already the newest version (20160104ubuntu1).
The following packages will be upgraded:
apt-transport-https
1 upgraded, 0 newly installed, 0 to remove and 54 not upgraded.
Need to get 25.7 kB of archives.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://mirrors.linode.com/ubuntu xenial-updates/main amd64 apt-transport-https amd64 1.2.12~ubuntu16.04.1 [25.7 kB]
Fetched 25.7 kB in 0s (2,540 kB/s)
(Reading database ... 25186 files and directories currently installed.)
Preparing to unpack .../apt-transport-https_1.2.12~ubuntu16.04.1_amd64.deb ...
Unpacking apt-transport-https (1.2.12~ubuntu16.04.1) over (1.2.10ubuntu1) ...
Setting up apt-transport-https (1.2.12~ubuntu16.04.1) ...

Creating Repository file for Docker

Make sure your repository configuration file is properly set to download the packages for Docker.

root@ubuntu:/etc/apt/sources.list.d# cat /etc/apt/sources.list.d/docker.list
deb https://apt.dockerproject.org/repo ubuntu-xenial main

Once it's added, you can update the packages once more by running "apt-get update". Make sure it takes the updates from the right repos. Remove any old docker package if it exists.

root@ubuntu:/etc/apt/sources.list.d# apt-get purge lxc-docker
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package 'lxc-docker' is not installed, so not removed
0 upgraded, 0 newly installed, 0 to remove and 54 not upgrad

root@ubuntu:~# apt-cache policy docker-engine
docker-engine:
Installed: (none)
Candidate: 1.11.2-0~xenial
Version table:
1.11.2-0~xenial 500
500 https://apt.dockerproject.org/repo ubuntu-xenial/main amd64 Packages
1.11.1-0~xenial 500
500 https://apt.dockerproject.org/repo ubuntu-xenial/main amd64 Packages
1.11.0-0~xenial 500
500 https://apt.dockerproject.org/repo ubuntu-xenial/main amd64 Packages

Install Kernel packages

For Ubuntu Xenial 16.04 version, it's recommended to install the linux_extra_image package which is compatible with the Kernel package. This package enables the Aufs storage driver.  AUFS storage driver takes multiple directories on a single host, stacks them on top of each other, providing a single unified view.

root@ubuntu:~# apt-get install linux-image-extra-$(uname -r)
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
crda iw libnl-3-200 libnl-genl-3-200 wireless-regdb
The following NEW packages will be installed:
crda iw libnl-3-200 libnl-genl-3-200 linux-image-extra-4.4.0-21-generic wireless-regdb
0 upgraded, 6 newly installed, 0 to remove and 54 not upgraded.
Need to get 39.0 MB of archives.

Installation

Now we can go ahead with the installation of the Docker.

root@ubuntu:~# apt-get install docker-engine
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
aufs-tools cgroupfs-mount git git-man liberror-perl libltdl7 libperl5.22 patch perl perl-modules-5.22 rename xz-utils
Suggested packages:
mountall git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk gitweb git-arch git-cvs git-mediawiki git-svn
diffutils-doc perl-doc libterm-readline-gnu-perl | libterm-readline-perl-perl make
The following NEW packages will be installed:
aufs-tools cgroupfs-mount docker-engine git git-man liberror-perl libltdl7 libperl5.22 patch perl perl-modules-5.22 rename xz-utils
0 upgraded, 13 newly installed, 0 to remove and 54 not upgraded.
Need to get 24.8 MB of archives.
After this operation, 139 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
WARNING: The following packages cannot be authenticated!

Start and confirm the Docker status

root@ubuntu:~# service docker start

root@ubuntu:~# docker version
Client:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 22:00:43 2016
OS/Arch: linux/amd64

Server:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 22:00:43 2016
OS/Arch: linux/amd64
root@ubuntu:~#

This below command downloads a test image namely hello-world from the Docker registry and runs it in a container. When the container runs, it prints an informational message. Then, it exits. Thus we can confirm the Docker working.

root@ubuntu:~# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
4276590986f6: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:a7d7a8c072a36adb60f5dc932dd5caba8831ab53cbf016bcdd6772b3fbe8c362
Status: Downloaded newer image for hello-world:latest

Hello from Docker.
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker Hub account:
https://hub.docker.com

For more examples and ideas, visit:
https://docs.docker.com/engine/userguide/

Now we're ready to start with Docker. We can download all required images from the Docker Hub using the command

docker pull image_name. For instance, let see how I'm downloading the some of the useful images.

root@ubuntu:~# docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
5ba4f30e5bea: Pull complete
9d7d19c9dc56: Pull complete
ac6ad7efd0f9: Pull complete
e7491a747824: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:46fb5d001b88ad904c5c732b086b596b92cfb4a4840a3abd0e35dbb6870585e4
Status: Downloaded newer image for ubuntu:latest

This has downloaded the Ubuntu image from the Docker Hub and we can use this for creating a Ubuntu container with this image.

root@ubuntu:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu latest 2fa927b5cdd3 11 days ago 122 MB
hello-world latest 94df4f0ce8a4 6 weeks ago 967 B

Creating Puppet inside a Docker container

For creating puppet containers, first we need to download the Puppet packages from the docker hub.

• puppet/puppet-agent-ubuntu
• puppet/puppetserver
• puppet/puppetdb
• puppet/puppetdb-postgres

Let's see how I downloaded these images from the Docker hub. You can use the command docker pull Image_name for that.

root@ubuntu:~# docker pull puppet/puppetserver
Using default tag: latest
latest: Pulling from puppet/puppetserver
5ba4f30e5bea: Already exists
9d7d19c9dc56: Already exists
ac6ad7efd0f9: Already exists
e7491a747824: Already exists
a3ed95caeb02: Already exists
158cd0fe54d8: Pull complete
7a15dfe1145a: Pull complete
0bb8d51ae57c: Pull complete
7b09944cb025: Pull complete
6bf96d82eed5: Pull complete
58fa7008c2bc: Pull complete
659b4b2b3359: Pull complete
0e205bb6d03b: Pull complete
915e3853b669: Pull complete
750b3208f97d: Pull complete
8fec247907de: Pull complete
Digest: sha256:c43290ca040a7693d9f41448eab4ff2444c61757aa303bd7979f7f1ef3e4ae95
Status: Downloaded newer image for puppet/puppetserver:latest

root@ubuntu:~# docker pull puppet/puppetdb
Using default tag: latest
latest: Pulling from puppet/puppetdb
0be59000882d: Pull complete
f20b6f990572: Pull complete
53662c966c9f: Pull complete
a3ed95caeb02: Pull complete
5eae59cbe62c: Pull complete
2b8ff6279504: Pull complete
612d7a4576b7: Pull complete
60577ed4c036: Pull complete
f99ad2d50f6f: Pull complete
9da7f43c61dc: Pull complete
e4c4271df64b: Pull complete
Digest: sha256:6532e4e3750183cd6951df6deb7bb1adb1e0e0ed37aa9e1e0294e257d73d9b1f
Status: Downloaded newer image for puppet/puppetdb:latest

root@ubuntu:~# docker pull puppet/puppetdb-postgres
Using default tag: latest
latest: Pulling from puppet/puppetdb-postgres
8b87079b7a06: Pull complete
a3ed95caeb02: Pull complete
ff6abb23e531: Pull complete
8364ca902ad3: Pull complete
84179c1b7ff6: Pull complete
be951654637c: Pull complete
4841dfc8333f: Pull complete
8e92fd62d485: Pull complete
13e5de4be2f2: Pull complete
d6aaf4d83b1c: Pull complete
3113f93aec6d: Pull complete
055e85b433f4: Pull complete
a97f9981bfe1: Pull complete
6c162fdd1104: Pull complete
Digest: sha256:d42428f0ecf75f7a0dbebee79cb45afaebfd193051fa1002e64fa026b2060f13
Status: Downloaded newer image for puppet/puppetdb-postgres:latest

root@ubuntu:~# docker pull puppet/puppet-agent-ubuntu
Using default tag: latest
latest: Pulling from puppet/puppet-agent-ubuntu
0be59000882d: Already exists
f20b6f990572: Already exists
53662c966c9f: Already exists
a3ed95caeb02: Already exists
576aca0f90fb: Pull complete
b1842b47756f: Pull complete
Digest: sha256:1867bcbe733adcbdfa004ec76ce8940a0927eef8877ee4f07b1ace4e68e7c5fa
Status: Downloaded newer image for puppet/puppet-agent-ubuntu:latest

Now we've downloaded all required images. You can view it by running docker images command.

root@ubuntu:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
puppet/puppetserver latest 0ac3058fad18 4 days ago 379.9 MB
puppet/puppetdb latest f3f9d8b3e54f 6 days ago 368.4 MB
puppet/puppet-agent-ubuntu latest 57fe50639909 6 days ago 202.9 MB
puppet/puppetdb-postgres latest 4f4ed55af431 10 days ago 265.8 MB
ubuntu latest 2fa927b5cdd3 11 days ago 122 MB
hello-world latest 94df4f0ce8a4 6 weeks ago 967 B

Before creating our Puppet container, we need to create a Docker network to add these Puppet containers as below.

root@ubuntu:~# docker network create puppet
e1ebd861dbb39be31da81a88e411e7f4762814ee203b371fca7643a7bb6840eb

Creating Puppet Master server

We can create puppet server with the image "puppet/puppetserver" with name puppet in the puppet network with hostname "puppet-linoxide".

root@ubuntu:~# docker run --net puppet --name puppet --hostname puppet.linoxide puppet/puppetserver
Warning: The following options to parse-opts are unrecognized: :flag
2016-06-08 09:36:24,348 INFO [o.e.j.u.log] Logging initialized @27125ms
2016-06-08 09:36:36,393 INFO [p.s.v.versioned-code-service] No code-id-command set for versioned-code-service. Code-id will be nil.
2016-06-08 09:36:36,394 INFO [p.s.v.versioned-code-service] No code-content-command set for versioned-code-service. Attempting to fetch code content will fail.
2016-06-08 09:36:36,396 INFO [p.t.s.w.jetty9-service] Initializing web server(s).
2016-06-08 09:36:36,450 INFO [p.s.j.jruby-puppet-service] Initializing the JRuby service
2016-06-08 09:36:36,455 WARN [p.s.j.jruby-puppet-service] The 'jruby-puppet.use-legacy-auth-conf' setting is set to 'true'. Support for the legacy Puppet auth.conf file is deprecated and will be removed in a future release. Change this setting to 'false' and migrate your authorization rule definitions in the /etc/puppetlabs/puppet/auth.conf file to the /etc/puppetlabs/puppetserver/conf.d/auth.conf file.
2016-06-08 09:36:36,535 INFO [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 1.
2016-06-08 09:36:53,825 WARN [puppetserver] Puppet Comparing Symbols to non-Symbol values is deprecated
(file & line not available)
2016-06-08 09:36:54,019 INFO [puppetserver] Puppet Puppet settings initialized; run mode: master
2016-06-08 09:36:56,811 INFO [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 1 of 1
2016-06-08 09:36:56,849 INFO [p.s.c.puppet-server-config-core] Initializing webserver settings from core Puppet
2016-06-08 09:36:59,780 INFO [p.s.c.certificate-authority-service] CA Service adding a ring handler
2016-06-08 09:36:59,827 INFO [p.s.p.puppet-admin-service] Starting Puppet Admin web app
2016-06-08 09:37:06,473 INFO [p.s.m.master-service] Master Service adding ring handlers
2016-06-08 09:37:06,558 WARN [o.e.j.s.h.ContextHandler] Empty contextPath
2016-06-08 09:37:06,572 INFO [p.t.s.w.jetty9-service] Starting web server(s).
2016-06-08 09:37:06,606 INFO [p.t.s.w.jetty9-core] webserver config overridden for key 'ssl-cert'
2016-06-08 09:37:06,607 INFO [p.t.s.w.jetty9-core] webserver config overridden for key 'ssl-key'
2016-06-08 09:37:06,608 INFO [p.t.s.w.jetty9-core] webserver config overridden for key 'ssl-ca-cert'
2016-06-08 09:37:06,608 INFO [p.t.s.w.jetty9-core] webserver config overridden for key 'ssl-crl-path'
2016-06-08 09:37:07,037 INFO [p.t.s.w.jetty9-core] Starting web server.
2016-06-08 09:37:07,050 INFO [o.e.j.s.Server] jetty-9.2.z-SNAPSHOT
2016-06-08 09:37:07,174 INFO [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@18ee4ac3{/puppet-ca,null,AVAILABLE}
2016-06-08 09:37:07,175 INFO [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@4c1434a7{/puppet-admin-api,null,AVAILABLE}
2016-06-08 09:37:07,176 INFO [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@7eef9da2{/puppet,null,AVAILABLE}
2016-06-08 09:37:07,177 INFO [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@26ad2d06{/,null,AVAILABLE}
2016-06-08 09:37:07,364 INFO [o.e.j.s.ServerConnector] Started ServerConnector@66b8635c{SSL-HTTP/1.1}{0.0.0.0:8140}
2016-06-08 09:37:07,365 INFO [o.e.j.s.Server] Started @70146ms
2016-06-08 09:37:07,381 INFO [p.s.m.master-service] Puppet Server has successfully started and is now ready to handle requests
2016-06-08 09:37:07,393 INFO [p.s.l.legacy-routes-service] The legacy routing service has successfully started and is now ready to handle requests

Now we've our Puppet Server created and running.

root@ubuntu:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f4b9f456a4c2 puppet/puppetserver "dumb-init /docker-en" 3 minutes ago Up 3 minutes 8140/tcp puppet

Creating Puppet Client

By running this command, you're creating another container as Puppet client with hostname Puppeet-client-linoxide with the docker image puppet/puppet-agent-ubuntu agent. You can either use this command to create the Puppet client or you can just use docker run --net puppet puppet/puppet-agent-ubuntu  to built one. If you're running this command,  with a onetime flag which means, Puppet exits after the first run.

root@ubuntu:~# docker run --net puppet --name puppet-client --hostname puppet-client-linoxide puppet/puppet-agent-ubuntu agent --verbose --no-daemonize --summarize
Info: Creating a new SSL key for puppet-client-linoxide.members.linode.com
Info: Caching certificate for ca
Info: csr_attributes file loading from /etc/puppetlabs/puppet/csr_attributes.yaml
Info: Creating a new SSL certificate request for puppet-client-linoxide.members.linode.com
Info: Certificate Request fingerprint (SHA256): 62:E2:37:8A:6E:0D:18:AC:81:0F:F1:3E:D6:08:10:29:D4:D6:21:16:59:B7:6D:3F:AA:5C:7A:08:38:B6:6B:07
Info: Caching certificate for puppet-client-linoxide.members.linode.com
Info: Caching certificate_revocation_list for ca
Info: Caching certificate for ca
Notice: Starting Puppet client version 4.5.1
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for puppet-client-linoxide.members.linode.com
Info: Applying configuration version '1465378896'
Info: Creating state file /opt/puppetlabs/puppet/cache/state/state.yaml
Notice: Applied catalog in 0.01 seconds
Changes:
Events:
Resources:
Total: 7
Time:
Schedule: 0.00
Config retrieval: 1.55
Total: 1.56
Last run: 1465378896
Filebucket: 0.00
Version:
Config: 1465378896
Puppet: 4.5.1

But if you're using this above command, then the container won't exit, It stays online and updates Puppet every 30 minutes based on the latest content from the Puppet Server. Now we've our Puppet server/Client running on our Docker.

root@ubuntu:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5f29866a103b puppet/puppet-agent-ubuntu "/opt/puppetlabs/bin/" 8 minutes ago Up 8 minutes puppet-client
f4b9f456a4c2 puppet/puppetserver "dumb-init /docker-en" 13 minutes ago Up 13 minutes 8140/tcp puppet

 

Creating PuppetDB

We can run a PuppetDB server in a docker container. Inorder to run a PuppetDB, we need a Postgres Server running. Docker supports only PostgreSQL server. This too can be another container instance, or RDS (Relational Database Service) end point or a Physical DB somewhere.  In addition, It requires a Puppet Master running. In order to use SSL certs during the initialization, you will need at least a token puppet master running that the container can connect to initialize the certs.

root@ubuntu:~# git clone https://github.com/tizzo/docker-puppetdb.git
Cloning into 'docker-puppetdb'...
remote: Counting objects: 12, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 12 (delta 3), reused 12 (delta 3), pack-reused 0
Unpacking objects: 100% (12/12), done.
Checking connectivity... done.

root@ubuntu:~# cd docker-puppetdb/

Create a Docker file compatible with Ubuntu 16.04. I got my Dockerfile and run the docker build.

root@ubuntu:~/docker-puppetdb# docker build .
Sending build context to Docker daemon 68.1 kB
Step 1 : FROM ubuntu:16.04
16.04: Pulling from library/ubuntu
5ba4f30e5bea: Already exists
9d7d19c9dc56: Already exists
ac6ad7efd0f9: Already exists
e7491a747824: Already exists
a3ed95caeb02: Already exists
Digest: sha256:f5edf3b741a08b573eca6bf25257847613540538a17b86e2b76e14724a0be68a
Status: Downloaded newer image for ubuntu:16.04
---> 2fa927b5cdd3
Step 2 : MAINTAINER Gareth Rushgrove "gareth@puppet.com"
---> Running in 555edbbd1017
---> a3d4cea623ac
Removing intermediate container 555edbbd1017
Step 3 : ENV PUPPETDB_VERSION "4.1.0" PUPPET_AGENT_VERSION "1.5.1" DUMB_INIT_VERSION "1.0.2" UBUNTU_CODENAME "xenial" PUPPETDB_USER puppetdb PUPPETDB_PASSWORD puppetdb PUPPETDB_JAVA_ARGS "-Djava.net.preferIPv4Stack=true -Xms256m -Xmx256m" PATH /opt/puppetlabs/server/bin:/opt/puppetlabs/puppet/bin:/opt/puppetlabs/bin:$PATH
---> Running in 4cb8a8220b1c

Once this is completed we can create our PuppetDB container.

root@ubuntu:~# docker run --net puppet --name puppetdb-postgres -e POSTGRES_PASSWORD=puppetdb -e POSTGRES_USER=puppetdb -d postgres
855a6b13fefa4123d5e16cdde84ebc7174ba149e66699e4c94c14e8fbfcac22f

root@ubuntu:~# docker run --net puppet -d -P --name puppetdb --link puppetdb-postgres:postgres puppet/puppetdb
bfe56b64bd980d20570374ed8204136303d82de8cbf1a4279c2f2fd25a798f59

All our containers are  running, we can even confirm its status by with this command, docker ps as shown below:

We can access our PuppetDB Dashboard at the URL >>http://Docker-Server-IP:32771

Hurray! This is how we can make Puppet to run on a container infrastructure inside Docker. I hope you enjoyed reading this article. I would recommend your valuable comments and suggestions on this.

Thank you! Have a Nice Day :)

The post How to Run Puppet on Container Infrastructure using Docker appeared first on LinOxide.