This tutorial is adapted from Web Age course Microservices Development Bootcamp with Immersive Project.
1.1 What is Docker
Docker is an open-source (and 100% free) project for IT automation. You can view Docker as a system or a platform for creating virtual environments which are extremely lightweight virtual machines. Docker allows developers and system administrators to quickly assemble, test, and deploy applications and their dependencies inside Linux containers supporting the multi-tenancy deployment model on a single host. Docker’s lightweight containers lend themselves to rapid scaling up and down. A container is a group of controlled processes associated with a separate tenant executed in isolation from other tenants. It is written in the Go programming language.
1.2 Where Can I Run Docker?
Docker runs on any modern-kernel 64-bit Linux distributions. The minimum supported kernel version is 3.10. Kernels older than 3.10 lack some of the features required by Docker containers. You can install Docker on VirtualBox and run it on OS X or Windows. Docker can be installed natively on Windows using Docker Machine, but requires Hyper-V. Docker can be booted from the small footprint Linux distribution boot2docker.
1.3 Installing Docker Container Engine
Installing on Linux:
Docker is usually available via the package manager of the distributions. For example, on Ubuntu and derivatives:
sudo apt-get update && sudo apt install docker.io
Installing on Mac
Download and install the official Docker.dmg from docker.com
Installing on Windows
Hyper-V must be enabled on Windows. Download the latest installer from docker.com
1.4 Docker Machine
Though Docker runs natively on Linux, it may be desirable to have two different host environment, such as Ubuntu and CentOS. To achieve this, VMs running Docker may be used. To simplify management of different Docker host, it is possible to use Docker Machine. Docker Machine is a tool that lets you install Docker Engine on virtual hosts, and manage the hosts with docker-machine commands. Docker Machine enables you to provision multiple remote Docker hosts on various flavors of Linux. Additionally, Machine allows you to run Docker on older Mac or Windows systems as well as cloud providers such as AWS, Azure and GCP. Using the docker-machine command, you can start, inspect, stop, and restart a managed host, upgrade the Docker client and daemon, and configure a Docker client to talk to your host.
1.5 Docker and Containerization on Linux
Docker leverages resource isolation features of the modern Linux kernel offered by cgroups and kernel namespaces. The cgroups and kernel namespaces features allow creation of strongly isolated containers acting as very lightweight virtual machines running on a single Linux host. Docker helps abstract operating-system-level virtualization on Linux using abstracted virtualization interfaces based on libvirt, LXC (LinuX Containers) and systemd-nspawn. As of version 0.9, Docker has the capability to directly use virtualization facilities provided by the Linux kernel via its own libcontainer library.
1.6 Linux Kernel Features: cgroups and namespaces
The control group kernel feature (cgroup) is used by the Linux kernel to allocate system resources such as CPU, I/O, memory, and network subject to limits, quotas, prioritization, and other control arrangements. The kernel provides access to multiple subsystems through the cgroup interface.
Examples of subsystems (controllers) are:
The memory controller for limiting memory use
The cpuacct controller for keeping track of CPU usage
The cgroups facility was merged into the Linux kernel version 2.6.24. Systems that use cgroups: Docker, Linux Containers (LXC), Hadoop, etc. The namespaces feature is a related to cgroups facility that enables different applications to act as separate tenants with completely isolated views of the operating environment, including users, process trees, network, and mounted file systems.
1.7 The Docker-Linux Kernel Interfaces
Source: Adapted from http://en.wikipedia.org/wiki/Docker_(software)
1.8 Docker Containers vs Traditional Virtualization
System virtualization tools or emulators such as VirtualBox, Hyper-V or VMware, boot virtual machines from a complete guest OS image (of your choice) and emulate a complete machine, which results in a high operational overhead. Virtual environments created by Docker run on the existing operating system kernel of the host’s OS without a need for a hypervisor. This leads to very low overhead and significantly faster container startup time. Docker-provisioned containers do not include or require a separate operating system (it runs in the host’s OS). This circumstance puts a significant limitation on your OS choices.
1.9 Docker Containers vs Traditional Virtualization
Overall, traditional virtualization has advantages over Docker in that you have a choice of guest OSes (as long as the machine architecture is supported). You can get only some (limited) choice of Linux distros. You still have some choice: e.g. you can deploy a Fedora container on a Debian host. You can, however, run a Windows VM inside a Linux machine using virtual machine emulators like VirtualBox (with less engineering efficiency). With Linux containers, you can achieve a higher level of deployed application density compared with traditional VMs (10x more units!). Docker runs everything through a central daemon which is not a particularly reliable and secure processing model.
1.10 Docker Integration
Docker can be integrated with a number of IT automation tools that extend its capabilities, including Ansible, Chef, Jenkins, Puppet, Salt. Docker is also deployed on a number of Cloud platforms like Amazon Web Services, Google Cloud Platform, Microsoft Azure, OpenStack and Rackspace.
1.11 Docker Services
Docker deployment model is application-centric and in this context provides the following services and tools:
◊ A uniform format for bundling an application along with its dependencies which is portable across different machines.
◊ Tools for automatic assembling a container from source code: make, maven, Debian packages, RPMs, etc.
◊ Container versioning with deltas between versions.
1.12 Docker Application Container Public Repository
Docker community maintains the repository for official and public domain.
Docker application images: https://hub.docker.com
1.13 Competing Systems
- Rocket container runtime from CoreOS (an open source lightweight Linux kernel-based operating system).
- LXD for Ubuntu from Canonical (the company behind Ubuntu)
- The LXC (Linux Containers), used by Docker internally
1.14 Docker Command Line
The following commands are shown as executed by the root (privileged) user:
docker run ubuntu echo ‘Yo Docker!’
This command will create a docker container based on the ubuntu image, execute the echo command on it, and then shuts down.
docker ps -a
This command will list all the containers created by Docker along with their IDs
1.15 Starting, Inspecting, and Stopping Docker Containers
docker start -i <container_id>
This command will start an existing stopped container in interactive (-i) mode (you will get container’s STDIN channel)
docker inspect <container_id>
This command will provide JSON-encoded information about the running container identified by container_id
docker stop <container_id>
This command will stop the running container identified by container_id
For the Docker command-line reference, visit-https://docs.docker.com/engine/reference/commandline/cli/
1.16 Docker Volume
If you destroy a container and recreate it, you will lose data. Ideally, data should not be stored in containers. Volumes are mounted file systems available to containers. Docker volumes are a good way of safely storing data outside a container. Docker volumes can be shared across multiple containers.
Creating a Docker volume
docker volume create my-volume
Mounting a volume
docker run -v my-volume:/my-mount-path -it ubuntu:12.04
/bin/bash
Viewing all volumes
docker volume ls
Deleting a volume
docker volume rm my-volume
1.17 Dockerfile
Rather than manually creating containers and saving them as custom images, it’s better to use Dockerfile to build images
Sample script
# let’s use ubuntu docker image
FROM openjdk
RUN apt-get update -y
RUN apt-get install sqlite -y
# deploy the jar file to the container
COPY SimpleGreeting-1.0-SNAPSHOT.jar
/root/SimpleGreeting-1.0-SNAPSHOT.jar
The Dockerfile filename is case sensitive. The ‘D’ in Dockerfile has to be uppercase. Building an image using docker build. (Mind the space and period at the end of the docker build command)
docker build -t my-image:v1.0 .
Or, if you want to use a different file name:
docker build -t my-image:v1.0 -f mydockerfile.txt
1.18 Docker Compose
A container runs a single application. However, most modern application rely on multiple service, such as database, monitoring, logging, messages queues, etc. Managing a forest of containers individually is difficult especially when it comes to moving the environment from development to test to production, etc. Compose is a tool for defining and running multi-container Docker applications on the same host. A single configuration file, docker-compose.yml, is used to define a group of container that must be managed as a single entity.
1.19 Using Docker Compose
- Define as many Dockerfile as necessary
- Create a docker-compose.yml file that refers to the individual Dockerfile
Sample Dockerfile
version: ‘3’
services:
greeting:
build: .
ports:
– “8080:8080”
links:
– mongodb
mongodb:
image: mongodb
environment:
MONGO_INITDB_ROOT_USERNAME: wasadmin
MONGO_INITDB_ROOT_PASSWORD: secret
volumes:
– my-volume:/data/db
volumes:
my-volume: {}
1.20 Dissecting docker-compose.yml
The Docker Compose file should be named either docker-compose.yml or docker-compose.yaml. Using any other names will require to use the -f argument to specify the filename. The docker-compose.yml file is writing in YAML
https://yaml.org/
The first line, version, indicates the version of Docker Compose being used. As of this writing, version 3 is the latest.
1.21 Specifying services
A ‘service’ in docker-compose parlance is a container. Services are specified under the service: node of the configuration file.
You choose the name of a service. The name of the service is meaningful within the configuration. A service (container) can be specified in one of two ways: Dockerfile or image name.
Use build: to specify the path to a Dockerfile
Use image: to specify the name of an image that is accessible to the host
1.22 Dependencies between containers
Some services may need to be brought up before other services. In docker-compose.yml, it is possible to specify which service relies on which using the links: node. If service C requires that service A and B be brought up first, add a link as
follows:
A:
build: ./servicea
B:
build: ./serviceb
C:
build: ./servicec
link:
– A
– B
It is possible to specify as many links as necessary. Circular links are not permitted (A links to B and B links to A).
1.23 Injecting Environment Variables
In a microservice, containerized application, environment variables are often used to pass configuration to an application.
It is possible to pass environment variable to a service via the dockercompose.
yml file
myservice:
environment:
MONGO_INITDB_ROOT_USERNAME: wasadmin
MONGO_INITDB_ROOT_PASSWORD: secret
1.24 runC Overview
Over the last few years, Linux has gradually gained a collection of features. Windows 10 and Windows Server 2016+, also added similar features. Those individual features have esoteric names like “control groups”, “namespaces”, “seccomp”, “capabilities”, “apparmor” and so on. Collectively, they are known as “OS containers” or sometimes lightweight virtualization”. Docker makes heavy use of these features and has become famous for it. Because “containers” are actually an array of complicated, sometimes arcane system features, they are integrated into a unified low-level component called runC. runC now available as a standalone tool which is a lightweight, portable container runtime. It includes all of the plumbing code used by Docker to interact with system features related to containers. It has no dependency on the rest of the Docker platform.
1.25 runC Features
- Full support for Linux namespaces, including user namespaces
- Native support for all security features available in Linux: Selinux, Apparmor, seccomp, control groups, capability drop, pivot_root, uid/gid dropping, etc. If Linux can do it, runC can do it.
- Native support for live migration, with the help of the CRIU team at Parallels
- Native support of Windows 10 containers is being contributed directly by Microsoft engineers
- Planned native support for Arm, Power, Sparc with direct participation and support from Arm, Intel, Qualcomm, IBM, and the entire hardware manufacturers ecosystem.
- Planned native support for bleeding edge hardware features – DPDK, sriov, tpm, secure enclave, etc.
1.26 Using runC
In order to use runc you must have your container in the format of an Open Container Initiative (OCI) bundle. If you have Docker installed you can use its export method to acquire a root filesystem from an existing Docker container.
# create the topmost bundle directory
mkdir /mycontainer
cd /mycontainer
# create the rootfs directory
mkdir rootfs
# export busybox via Docker into the rootfs directory
docker export $(docker create busybox) | tar -C rootfs -xvf –
After a root filesystem is populated you just generate a spec in the format of a config.json file inside your bundle.
runc spec
1.27 Running a Container using runC
The first way is to use the convenience command run that will handle creating, starting, and deleting the container after it exits.
# run as root
cd /mycontainer
runc run mycontainerid
The second way is to implement the entire lifecycle (create, start, connect, and delete the container), manually
# run as root
cd /mycontainer
runc create mycontainerid
# view the container is created and in the “created” state
runc list
# start the process inside the container
runc start mycontainerid
# after 5 seconds view that the container has exited and is now in the stopped
state
runc list
# now delete the container
runc delete mycontainerid
1.28 Summary
- Docker is a system for creating virtual environments which are, for all intents and purposes, lightweight virtual machines.
- Docker containers can only run the type of OS that matches the host’s OS.
- Docker containers are extremely lightweight (although not so robust and secure), allowing you to achieve a higher level of deployed application density compared with traditional VMs (10x more units!).
- On-demand provisioning of applications by Docket supports the Platform as- a-Service (PaaS)–style deployment and scaling.
- runC is a container runtime which has support for various containerization solutions.
