giovanni.ciatto@unibo.it
Compiled on: 2024-11-21 — printable version
[Software] Deployment
all of the activities that make a software system available for use
To govern the deployment of software systems onto some infrastructure
software system: multiple interacting software components, each one having its computational requirements
infrastructure: the available hardware and software substratum supporting the execution of software components
deployment: orderly instantiating software components onto the infrastructure, while
Short-lived tasks (a.k.a. jobs): tasks which eventually terminate
Long-lived tasks (a.k.a. services) tasks which run indefinitely
Short-lived tasks are bare computational activities aimed at processing/generating some data
Pure algorithms accepting data as input and/or producing data as output.
They will eventually reach their natural termination
Common workflow:
Parallelism may be exploited to speed up the computations
Long-lived are computational activities which are not meant to terminate automatically (unless explicitly stopped)
Infinite loops, just waiting for requests, and serving them as soon as possible
Interactivity is a key aspect:
Common workflow:
Parallelism may be exploited to support replication
Regardless of their life-span, computational tasks may require computational resources of various sorts:
specific hardware capabilities
specific operative systems (Linux, MacOS, Windows, etc.) or specific architectures (x86, ARM, etc.)
specific runtime platforms (e.g. JVM, Python, .NET etc.)
specific infrastructural components (e.g. database, reverse proxy, load-balancer, broker, etc.)
specific libraries (e.g. CUDA, numpy
, etc.)
Why can’t we simply configure bare-metal machines to host our tasks?
Need for multi-tenancy (i.e. sharing) on computational resources
Need for isolation
Need to formalize / control / reproduce the computational environment of applications
Need to automate deployment of applications into production / testing environments
Need for flexibility (in re-deployment), and scalability
Some abstraction is needed to encapsulate applications, and their computational environment, into a single unit
Encapsulation is the hiding of the implementation details of a component from its clients. Encapsulated components have a clear interface which is the only way to interact with them.
Virtual machines (VM) on top of VM hypervisors (VMH)
Containers on top of container engines (CE)
VMH run on/as the OS of a bare-metal machine, abstracting HW and SW peculiarities away
VMH may instantiate multiple VM on the same physical machine
each VM runs its own OS, and may host multiple applications
VM may be paused, snapshot (into file), resumed, and possibly migrated
VM are coarse-grained encapsulation units
Many industry-ready technologies:
CE run on the OS of a bare-metal/virtual machine, abstracting the runtime, storage, and network environment away
one may instantiate multiple containers on the same machine
each container shares the OS kernel of the host, yet having its own runtime(s), storage, and network facilities
each container is instance of an image, i.e. a read-only template containing deployment instructions
containers are fine-grained encapsulation units
Many industry-ready technologies:
Containers provide runtime isolation without operating system replication
Cluster
a set of similarly configured machines (a.k.a. nodes) interconnected via a network in order to let users exploit their joint computational power
Users may want to deploy their applications on the cluster
Deployment shall allocate tasks on the cluster efficiently
Infrastructure-as-a-service (IaaS) cloud technologies support deploying tasks on clusters, as VM
Container orchestrators support deploying tasks on clusters, as containers
Containerisation
Orchestration
Containerisation is a pre-requisite for orchestration
Two syntaxes involved:
Docker: the most famous container technology, actually consisting of several components
docker
group
sudo usermod -aG docker $USER
sudo systemctl enable docker; sudo systemctl start docker
docker run hello-world
docker --help
docker <resource> <command> <options> <args>
<resource>
can be omittedSubsequent examples work on Linux, but they should work on any platform
- provided that a
bash
orzsh
shell is available
docker pull adoptopenjdk
docker run adoptopenjdk
Every image provides a default command, running without options runs such default in a non-interactive terminal.
Running in interactive mode can be achieved with the -i
option
Running a custom command inside the container can be achieved with writing the command after the image name
docker run -i adoptopenjdk bash
t
option to run in a pseudo-tty (always use it whenever you use -i
)--rm
to remove the container after useA docker container runs in isolation, w.r.t. the host and other containers.
Environment variables, network ports, and file system folders are not shared.
Sharing must be explicit and requires options to be specified after docker run
:
-e <name>=<value>
-v <host>:<guest>:<options>
<host>
is the path (or volume name) on the host system<guest>
is the location where it will be mounted on the container<options>
can be optionally specified as mount options (e.g., rw
, ro
)-p <host>:<guest>
<host>
is the port on the host system<guest>
is the corresponding port on the containerEvery image has a unique ID, and may have an associated tag
The subcommand images
lists the pulled images and their associated information
The subcommand image
allows for running maintenance tasks, e.g.
docker image ls
– same as docker images
docker image prune
– removes unused imagesdocker image rm
– removes images by namedocker image tag
– associates a tag to an imageDocker images are written in a Dockerfile
Every command inside a Dockerfile generates a new layer
The final stack of layers creates the final image
The docker build
command interprets the Dockerfile
commands to produce a sequence of layers
Changes to a layer do not invalidate previous layers
# Pulls an image from docker hub with this name. Alternatively, "scratch" can be used for an empty container
FROM alpine:latest
# Runs a command
RUN apk update; apk add nodejs npm
# Copies a file/directory from the host into the image
COPY path/to/my/nodejs/project /my-project
# Sets the working directory
WORKDIR /my-project
# Runs a command
RUN npm install
# Adds a new environment variable
ENV SERVICE_PORT=8080
# Exposes a port
EXPOSE 8080
# Configures the default command to execute
CMD npm run service
to be built by means of the command: docker build -t PATH
PATH
is the host path containing the Dockerfile
path/to/my/nodejs/project
DIRECTORYDockerfile
as for the previous slide
File .dockerignore
node_modules/
package.json
{
"name": "example",
"version": "1.0.0",
"description": "Example Express WS for exemplifying Docker usage",
"main": "index.mjs",
"scripts": {
"service": "node index.mjs"
},
"dependencies": {
"express": "^4.18.2"
},
"author": "gciatto",
"license": "ISC"
}
index.mjs
import { env } from 'process'
import express from 'express'
import { randomBytes } from 'crypto'
const port = 'SERVICE_PORT' in env ? env.SERVICE_PORT : 8080
const hostname = 'HOSTNAME' in env ? env.HOSTNAME : 'localhost'
const serverID = randomBytes(8).toString('hex') // 8-char random string
const server = express()
let counter = 0
server.get('/', function (req, res) {
res.send(`[${serverID}@${hostname}:${port}] Hit ${++counter} times`)
})
console.log(`Service ${serverID} listening on ${hostname}:${port}`)
server.listen(port)
i.e. a Web service listening on the port indicated by the SERIVICE_PORT
env var, showing Web pages of the form
[$SERVER_ID@$HOST_NAME:PORT] Hit $VIEW_COUNT times
Every line in a Dockerfile
generates a new layer:
A layer is a diff w.r.t. the previous one
In other words, Docker keeps track of what information is added to the image at each step
When a Dockerfile
is built, Docker checks whether the layer has already been built in the recent past
If so, it reuses the cached layer, and skips the execution of the corresponding command
When the container is run, the images layers are read-only, and the container has a read-write layer on top of them
So, when a container is stopped, the read-write layer is discarded, and the image layers are kept
In this way, the space occupied by the container is minimal
Image naming is done via tags
The easiest way to do so is assigning tags at build time with the -t
options of docker build
The option can be repeated multiple times to make multiple tags
docker build -t "myImage:latest" -t "myImage:0.1.0" /path/to/Dockerfile/container
latest
is usually used to identify the most recent version of some image
For instance, you may build the image from a few slides ago with:
docker build -t my-service path/to/my/nodejs/project
my-service:latest
is automatically createdImages get published in registries
The most famous, free for publicly available images, is Docker Hub
By default, Docker uses Docker Hub as registry (both for pull
and push
operations)
Docker Hub requires registration and CLI login:
docker login docker.io
Once done, publication is performed via push
:
docker push <image name>
Of course, as any other software, custom docker images should get built in CI
Several integrators use containers as build environments:
More in general: there is no inherent limit to nesting containers
For instance:
docker run --privileged --rm -it docker:dind
docker run -it --rm docker:cli
Volumes are bridges between the host’s file system and the container’s one
Their support several use cases:
<host path>:<guest path>
(both must be absolute paths)
docker run -v /home/user/my-project:/my-project ...
<host path>
can be read within the container at <guest path>
<guest path>
can be accessed on <host path>
docker volume create <name>
<name>:<guest path>
docker run -v my-volume:/my-project ...
/var/lib/docker/volumes/<name>/_data
tmpfs:<guest path>
--tmpfs
option to docker run
docker run --tmpfs <guest path> ...
Let’s open a Linux shell in a container: docker run -it --rm alpine:latest sh
Let’s create folder in there: mkdir -p /data
Let’s create a file in there: echo "Hello world" > /data/hello.txt
Does the file exist? ls -la /data
/ # ls -la /data
total 12
drwxr-xr-x 2 root root 4096 Nov 2 09:28 .
drwxr-xr-x 1 root root 4096 Nov 2 09:28 ..
-rw-r--r-- 1 root root 5 Nov 2 09:28 hello.txt
Let’s exit the container: exit
Let’s open a new shell in a new container: docker run -it --rm alpine:latest sh
Does the file exist? ls -la /data
ls: /data: No such file or directory
Why?
docker run ...
command created a new containerLet’s create a folder on the host: mkdir -p $(pwd)/shared
Let’s create 10 containers
./shared/
container-<i>.txt
/dev/random
, encoded in base64for i in $(seq 1 10); do
docker run --rm -d \ # detached mode
-v $(pwd)/shared:/data \ # bind mount
--hostname container-$i \ # parametric hostname inside the container
alpine sh -c \
'cat /dev/random | head -n 1 | base64 > /data/$(hostname).txt'
done
Let’s have at the shared directory: ls -la ./shared
ls -la ./shared
totale 56
drwxr-xr-x 2 user user 4096 2 nov 10.51 .
drwxr-xr-x 79 user user 12288 2 nov 10.56 ..
-rw-r--r-- 1 root root 329 2 nov 10.51 container-10.txt
-rw-r--r-- 1 root root 333 2 nov 10.51 container-1.txt
-rw-r--r-- 1 root root 381 2 nov 10.51 container-2.txt
-rw-r--r-- 1 root root 167 2 nov 10.51 container-3.txt
-rw-r--r-- 1 root root 110 2 nov 10.51 container-4.txt
-rw-r--r-- 1 root root 102 2 nov 10.51 container-5.txt
-rw-r--r-- 1 root root 9 2 nov 10.51 container-6.txt
-rw-r--r-- 1 root root 106 2 nov 10.51 container-7.txt
-rw-r--r-- 1 root root 766 2 nov 10.51 container-8.txt
-rw-r--r-- 1 root root 203 2 nov 10.51 container-9.txt
root
(inside the container)root
on the host (outside the container)
user
user
has no superuser privileges, they cannot delete the files$(pwd)
to get the current working directoryLet’s create a named volume: docker volume create my-volume
Same 10 containers as before, but using the named volume instead of the bind mount
for i in $(seq 1 10); do
docker run --rm -d \ # detached mode
-v my-volume:/data \ # reference to volume my-volume
--hostname container-$i \ # parametric hostname inside the container
alpine sh -c \
'cat /dev/random | head -n 1 | base64 > /data/$(hostname).txt'
done
How to access the data now?
docker run --rm -it -v my-volume:/data alpine sh
ls -la /data
total 48
drwxr-xr-x 2 root root 4096 Nov 2 10:15 .
drwxr-xr-x 1 root root 4096 Nov 2 10:15 ..
-rw-r--r-- 1 root root 608 Nov 2 10:14 container-1.txt
-rw-r--r-- 1 root root 349 Nov 2 10:15 container-10.txt
-rw-r--r-- 1 root root 369 Nov 2 10:14 container-2.txt
-rw-r--r-- 1 root root 304 Nov 2 10:14 container-3.txt
-rw-r--r-- 1 root root 240 Nov 2 10:14 container-4.txt
-rw-r--r-- 1 root root 434 Nov 2 10:14 container-5.txt
-rw-r--r-- 1 root root 150 Nov 2 10:14 container-6.txt
-rw-r--r-- 1 root root 41 Nov 2 10:14 container-7.txt
-rw-r--r-- 1 root root 33 Nov 2 10:15 container-8.txt
-rw-r--r-- 1 root root 1212 Nov 2 10:15 container-9.txt
Alternatively, let’s find where the volume is stored on the host
docker volume inspect my-volume
Let’s analyse the JSON description of the volume:
[
{
"CreatedAt": "2023-11-02T11:13:56+01:00",
"Driver": "local",
"Labels": null,
"Mountpoint": "/var/lib/docker/volumes/my-volume/_data",
"Name": "my-volume",
"Options": null,
"Scope": "local"
}
]
The Mountpoint
entry reveals the location of the volume on the host’s file system
Let’s look into that position (may require super-user access rights):
sudo ls -la /var/lib/docker/volumes/my-volume/_data
totale 48
drwxr-xr-x 2 root root 4096 2 nov 11.15 .
drwx-----x 3 root root 4096 2 nov 11.13 ..
-rw-r--r-- 1 root root 349 2 nov 11.15 container-10.txt
-rw-r--r-- 1 root root 608 2 nov 11.14 container-1.txt
-rw-r--r-- 1 root root 369 2 nov 11.14 container-2.txt
-rw-r--r-- 1 root root 304 2 nov 11.14 container-3.txt
-rw-r--r-- 1 root root 240 2 nov 11.14 container-4.txt
-rw-r--r-- 1 root root 434 2 nov 11.14 container-5.txt
-rw-r--r-- 1 root root 150 2 nov 11.14 container-6.txt
-rw-r--r-- 1 root root 41 2 nov 11.14 container-7.txt
-rw-r--r-- 1 root root 33 2 nov 11.15 container-8.txt
-rw-r--r-- 1 root root 1212 2 nov 11.15 container-9.txt
Volumes are NOT automatically cleaned up when containers are deleted
Volumes must be explicitly deleted by the user
docker volume rm <volume name>
In our example, remember to delete my-volume
:
docker volume rm my-volume
Sometimes one may be willing to let a container access the Docker daemon of its host
To achieve that one may exploit bind mounts
Explanation:
/var/run/docker.sock
docker
command is just a CLI program writing/reading to/from the socket to govern the daemonOne may simply create a Docker container with the Docker CLI tool installed on it
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock --name sudo docker:cli sh
Inside that container one may run the docker
command as usual, to govern the host’s Docker daemon
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3d9016cd067e docker:cli "docker-entrypoint.s…" 5 seconds ago Up 3 seconds sudo
Docker may virtualise container’s networking facilities
This is useful to let containers communicate with any external entity
most commonly, other containers or the host
Networks are virtual entities managed by Docker
- they mimic the notion of layer-3 network
- in the eyes of the single container, they are virtual network interfaces
- overall, their purpose is to delimit the communication scope for containers
Different sorts of networks are available in Docker, depending on the driver:
Available drivers:
none: no networking facilities are provided to the container, which is then isolated
host: remove network isolation between the container and the Docker host, and use the host’s networking directly
bridge (default): a virtual network, internal to the host, letting multiple containers communicate with each other
overlay: a virtual network, spanning multiple hosts, letting multiple containers communicate with each other among different nodes of the same cluster
Other drivers are available as well, cf. https://docs.docker.com/network/drivers/
No networking facilities are provided to the container at all, which is then isolated w.r.t the external world
This is useful for containers that do not need to communicate via the network
Example: docker run --network none -it --rm alpine sh
/ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: sendto: Network unreachable
No isolation between the container and the host
The main consequence is that the container will be assigned with the same IP address as the host
Example: docker run --network host my-service
Service 540c3b7bb7860c6a listening on <your pc's hostname>:8080
Virtual network spanning through one or more containers on the same host
Each container gets assigned with a private IP address on the network
172.x.y.z
To exemplify the usage of IP:
let’s create a container using the my-service
image created a few slides ago
docker run --network bridge -d --rm --name my-container my-service
8080
my-container
--network bridge
is useless, because that’s the defaultlet’s inspect the container’s IP address
docker inspect my-container | grep IPAddress
172.17.0.2
open your browser and browse to http://172.17.0.2:8080
[$SERVER_ID@$HOST_NAME:PORT] Hit $VIEW_COUNT times
bridge
-like networks can be contacted by IP, from the hosttry to contact the container:
Let’s create a non-trivial scenario with 2 container attached to the same network:
dind
) will run the Docker daemon (Docker-in-Docker)cli
) will run the Docker CLImy-network
let’s create the network: docker network create --driver bridge my-network
let’s create the daemon:
docker run --privileged -d --rm --network my-network --hostname dind docker:dind
let’s create the client: docker run -it --rm --network my-network --hostname cli docker:cli
inside cli
, let’s check if dind
is contactable by hostname: ping dind
inside cli
, let’s check if the Docker client works: docker ps
error during connect: Get "http://docker:2375/v1.24/containers/json": dial tcp: lookup docker on 127.0.0.11:53: no such host
apparently the docker:cli
image is preconfigured to contact a daemon on the docker
hostname
let’s stop dind
:
docker ps
on the host to get the name of the dind
containerdocker stop <container name>
docker stop $(docker ps -q --filter ancestor=docker:dind)
-q
(quiet), only display container IDs--filter ancestor=X
selects containers whose image is X
--name
let’s restart a Docker-in-Docker daemon, using the docker
hostname:
docker run --privileged -d --rm --network my-network --name dind --hostname docker docker:dind dockerd --host=tcp://0.0.0.0:2375
--name X
and --hostname X
dockerd --host=tcp://0.0.0.0:2375
is necessary to force the port the daemon will listen on
let’s restart the client: docker run -it --rm --network my-network --hostname cli docker:cli
let’s run a command on the CLI, say docker run hello-world
docker
hostnameWhen a container wraps a service listening on layer-4 port…
When creating a Docker image, the EXPOSE
command in the Dockerfile
is used to declare ports exposed by default
EXPOSE 8080
The port exposed by a container can be inspected via docker ps
docker run -d --rm my-service; docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7a14f3bf95cc my-service "/bin/sh -c 'npm run…" 1 second ago Up Less than a second 8080/tcp wonderful_goldwasser
(look at the PORTS
column)
When running a Docker container, the exposed port can be mapped to host’s ports via -P
option
EXPOSE
d port in the image mapped to some random port of the hostdocker run -d --rm -P my-service; docker ps
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b1470831b62b my-service "/bin/sh -c 'npm run…" 6 seconds ago Up 5 seconds 0.0.0.0:32773->8080/tcp, :::32773->8080/tcp sad_benz
One may control the mapping of ports via the -p <host port>:<guest port>
option
docker run -d --rm -p 8888:8080 my-service; docker ps
8080
to the host’s port 8080
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f034da93c476 my-service "/bin/sh -c 'npm run…" 4 seconds ago Up 3 seconds 0.0.0.0:8888->8080/tcp, :::8888->8080/tcp bold_booth
Operating containers manually is error prone and low-level
Docker Compose lets you increase the abstraction level on the single machine
Starting from a user-provided specification of a stack…
… Docker Compose can:
The only thing the user must do is: creating a docker-compose.yml
file
The example application is composed of the following parts:
webapp
and database
version: 3.9 # version of the specification
services: # section defining services composing the stack
frontend: # name of the first service (frontend)
image: example/webapp # image to use for the first service
depends_on: # section for def
- backend # this should be started AFTER the backend service is healthy
environment: # section for environment variables
SERVICE_PORT: 8043 # env var dictating the port the service will listen on
DB_HOST: backend # env variable dictating the hostname of the backend service
# (equal to the service name)
DB_PORT: 3306 # env variable dictating the port the backend service will listen on
ports: # section for ports to be exposed / mapped on the host
- "443:8043" # exposing port 8043 of the container as port 443 of the host
networks: # section for networks to be attached to
- front-tier # attaching the service to the front-tier network (referenced by name)
- back-tier # attaching the service to the back-tier network (referenced by name)
# configs are files to be copied in the service' container, without needing to create a new image
configs: # section for configuration files to be injected in the container
- source: httpd-config # reference to the configuration file (by name)
target: /etc/httpd.conf # path on the container where the config file will be mounted as read-only
# secrets are reified as read-only files in /run/secrets/<secret_name>
secrets: # section for secrets to be injected in the service' container
- server-certificate # reference to the secret (by name)
- db-credentials # reference to the secret (by name)
# assumption: image example/webapp is programmed to look in /run/secrets/<secret_name>
# and load the secrets therein contained
backend: # name of the second service (backend)
image: example/database # image to use for the second service
# this is a command to be run inside the service to check whether it is healthy
healthcheck:
test: ["CMD", "command", "which", "should", "fail", "if", "db not healthy"]
interval: 1m30s
timeout: 10s
retries: 3
start_period: 40s
volumes: # section for volumes to be attached to the container
- db-data:/etc/data # the volume db-data (reference by name) will be mounted
# as read-write in /etc/data
networks:
- back-tier # attaching the service to the back-tier network (referenced by name)
secrets:
- db-credentials # reference to the secret (by name)
volumes: # section for volumes to be created in the stack
db-data: # name of the volume, to be referenced by services
driver_opts:
type: "none"
o: "bind" # this volume is a bind mount
device: "/path/to/db/data/on/the/host"
configs: # section for configuration files to be created in the stack
httpd-config: # name of the configuration file, to be referenced by services
file: path/to/http/configuration/file/on/the/host
secrets: # section for secrets to be created in the stack
server-certificate: # name of the secret, to be referenced by services
file: path/to/certificate/file/on/the/host
db-credentials: # name of the secret, to be referenced by services
external: true # this secret is not created by the stack, but it is expected to be created by the user
name: my-db-creds # name of the secret, as created by the user
networks:
# The presence of these objects is sufficient to define them
front-tier: {}
back-tier: {}
A stack is a set of inter-related services, networks, volumes, secrets, configs
docker-compose.yml
fileFrom Docker Compose documentation:
services
section of the docker-compose.yml
file
docker-compose.yml
fileFrom Docker Compose documentation:
configs
section of the docker-compose.yml
fileone config may either be created stack-wise out of a local file from the host…
configs:
config_name:
file: /path/to/config/on/the/host
… or it can be manually created by means of the docker config create [OPTIONS] NAME PATH
command
docker-compose.yml
files:
configs:
config_name:
external: true
name: "the name used when creating the config"
# this may be different than config_name
in any case, the config’s mount point should be specified at the service level:
services:
my-service:
...
configs:
- source: config_name # reference to the configs section (by YAML name)
target: /path/to/config/on/the/container
From Docker Compose documentation:
secrets are configs containing sensitive data that should be kept secret
put it simply, Docker makes it harder to inspect the content of secrets both on the host
one secret may be either be created stack-wise out of a local file or an environment variable from the host…
secrets: secrets:
secret_name: secret_name:
file: "/path/to/secret/on/the/host" environment: "NAME_OF_THE_VARIABLE_ON_THE_HOST"
… or it can be manually created by means of the docker secret create [OPTIONS] NAME PATH|-
command
docker-compose.yml
files:
secrets:
secret_name:
external: true
name: "the name used when creating the secret"
# this may be different than secret_name
in any case, the secret’s mount point should be specified at the service level:
services:
my-service:
...
secrets:
# short syntax, mounted on /run/secrets/<secret_name>
- secret_name # reference to the secrets section (by YAML name)
# long syntax
- source: secret_name # reference to the secrets section (by YAML name)
target: /path/to/secret/on/the/container
Important remark:
PATH
PATH
From Docker Compose documentation:
volumes
section of the docker-compose.yml
filevolumes:
volume_name:
driver: local
driver_opts:
type: "none"
o: "bind"
device: "/path/to/folder/on/the/host"
volumes:
volume_name:
driver: local
driver_opts:
type: "nfs"
# meaning of options: https://wiki.archlinux.org/title/NFS#Mount_using_/etc/fstab
o: "addr=NFS_SERVER_ADDRESS,nolock,soft,rw" # cf. nfs
device: ":/path/on/the/nfs/server/"
… or it can be manually created by means of the docker volume create [OPTIONS] NAME
command
docker-compose.yml
files:
volumes:
volume_name:
external: true
name: "the name used when creating the volume"
# this may be different than volume_name
in any case, the volume’s mount point should be specified at the service level:
services:
my-service:
...
volumes:
- type: volume
source: volume_name # reference to the volumes section (by YAML name)
target: /path/to/folder/on/the/container
# alternatively, one may define a service-specific bind mount on the fly
- type: bind
source: /path/to/folder/on/the/host
target: /path/to/folder/on/the/container
From Docker Compose documentation:
networks in compose are no different than the ones we have already discussed
they are declared in the networks
section of the docker-compose.yml
file
one network may be created stack-wise out of a local folder from the host…
networks:
network_name:
driver: overlay # default choice in swarm
driver_opts:
# many options, covering many advanced use cases for network engineering
… or it can be manually created by means of the docker network create [OPTIONS] NAME
command
docker-compose.yml
files:
networks:
network_name:
external: true
name: "the name used when creating the network"
# this may be different than network_name
in any case, the network’s mount point should be specified at the service level:
services:
my-service:
...
networks:
- network_name # reference to the networks section (by YAML name)
See all possibilities with docker compose --help
docker compose up
starts the stack
--help
)
-d
(detached mode): run containers in the background--wait
wait for services to be running or healthy (implies detached mode)docker compose down
stops the stack
--help
)
--remove-orphans
remove containers for services not defined in the Compose file (e.g. residuals from previous versions)-v
remove named volumes declared in the “volumes” section of the Compose file and anonymous volumes attached to containersAll commands assume a
docker-compose.yml
file is present in the current working directory
version: "3.9"
services:
db:
image: mariadb:latest
command: '--default-authentication-plugin=mysql_native_password'
volumes:
- db_data:/var/lib/mysql
restart: always
networks:
- back-tier
environment:
MYSQL_ROOT_PASSWORD: "..Password1.."
MYSQL_DATABASE: wordpress_db
MYSQL_USER: wordpress_user
MYSQL_PASSWORD: ",,Password2,,"
healthcheck:
test: [ "CMD", "healthcheck.sh", "--su-mysql", "--connect", "--innodb_initialized" ]
start_period: 1m
start_interval: 10s
interval: 1m
timeout: 5s
retries: 3
wordpress:
depends_on:
- db
image: wordpress:latest
volumes:
- wordpress_data:/var/www/html
ports:
- "8000:80"
restart: always
networks:
- back-tier
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpress_user
WORDPRESS_DB_PASSWORD: ",,Password2,,"
WORDPRESS_DB_NAME: wordpress_db
volumes:
db_data: {}
wordpress_data: {}
networks:
back-tier: {}
Beware: this is unsafe as passwords are in clear text!
Copy-paste this code in a file named your-dir/docker-compose.yml
cd your-dir
docker compose up
and then inspect the logs
open your browser and browse to http://localhost:8000
docker ps
and have a look to the running containers
docker network ls
docker volume ls
Press Ctrl+C
to stop the stack
docker compose down --volumes
to delete the stack
Example of docker compose up
logs:
docker compose up
[+] Building 0.0s (0/0) docker:default
[+] Running 5/5
✔ Network your-dir_back-tier Created 0.1s
✔ Volume "your-dir_wordpress_data" Created 0.1s
✔ Volume "your-dir_db_data" Created 0.0s
✔ Container your-dir-db-1 Created 0.8s
✔ Container your-dir-wordpress-1 Created 0.9s
Attaching to db-1, wordpress-1
db-1 | 2023-11-23 14:24:57+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.2.2+maria~ubu2204 started.
db-1 | 2023-11-23 14:24:57+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
db-1 | 2023-11-23 14:24:57+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.2.2+maria~ubu2204 started.
db-1 | 2023-11-23 14:24:57+00:00 [Note] [Entrypoint]: Initializing database files
wordpress-1 | WordPress not found in /var/www/html - copying now...
wordpress-1 | Complete! WordPress has been successfully copied to /var/www/html
wordpress-1 | No 'wp-config.php' found in /var/www/html, but 'WORDPRESS_...' variables supplied; copying 'wp-config-docker.php' (WORDPRESS_DB_HOST WORDPRESS_DB_NAME WORDPRESS_DB_PASSWORD WORDPRESS_DB_USER)
wordpress-1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.22.0.3. Set the 'ServerName' directive globally to suppress this message
wordpress-1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.22.0.3. Set the 'ServerName' directive globally to suppress this message
wordpress-1 | [Thu Nov 23 14:24:58.289485 2023] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.56 (Debian) PHP/8.0.30 configured -- resuming normal operations
wordpress-1 | [Thu Nov 23 14:24:58.289516 2023] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
db-1 | 2023-11-23 14:24:58 0 [Warning] 'default-authentication-plugin' is MySQL 5.6 / 5.7 compatible option. To be implemented in later versions.
...
Example of docker ps
logs:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4691a9641135 wordpress:latest "docker-entrypoint.s…" 9 minutes ago Up 9 minutes 0.0.0.0:8000->80/tcp, :::8000->80/tcp your-dir-wordpress-1
314feb9d42ab mariadb:latest "docker-entrypoint.s…" 9 minutes ago Up 9 minutes (healthy) 3306/tcp your-dir-db-1
Example of docker network ls
logs:
NETWORK ID NAME DRIVER SCOPE
b304d1ae5404 bridge bridge local
1bb39315ff98 host host local
03c8700dee6a none null local
470d81d26296 your-dir_back-tier bridge local
Let’s create a file containing the DB root password, say db-root-password.txt
:
echo -n "..Password1.." > your-dir/db-root-password.txt
Let’s create a file containing the DB user password, say db-password.txt
:
echo -n ",,Password2,," > your-dir/db-password.txt
Let’s edit the stack as described on the right
version: "3.9"
services:
db:
...
environment:
...
MYSQL_ROOT_PASSWORD_FILE: /run/secrets/db_root_password
MYSQL_PASSWORD_FILE: /run/secrets/db_password
...
secrets:
- db_root_password
- db_password
wordpress:
...
environment:
...
WORDPRESS_DB_PASSWORD_FILE: /run/secrets/db_password
...
secrets:
- db_password
volumes:
db_data: {}
wordpress_data: {}
networks:
back-tier: {}
secrets:
db_password:
file: db_password.txt
db_root_password:
file: db_root_password.txt
Docker Compose is also useful for software testing
E.g. whenever the system under test relies on some infrastructural component
Usual workflow for testing:
Docker Compose may be coupled with some test framework of choice to automate the process
class TestMariaDBCustomerRepository {
companion object {
@BeforeClass @JvmStatic fun setUpMariaDb() { TODO("start MariaDB via Docker Compose") }
@AfterClass @JvmStatic fun tearDownMariaDb() { TODO("stop MariaDB via Docker Compose") }
}
private val connectionFactory = MariaDBConnectionFactory(DATABASE, USER, PASSWORD, HOST, PORT)
private lateinit var repository: SqlCustomerRepository
@Before
fun createFreshTable() {
repository = SqlCustomerRepository(connectionFactory, TABLE)
repository.createTable(replaceIfPresent = true)
}
private val taxCode = TaxCode("CTTGNN92D07D468M")
private val person = Customer.person(taxCode, "Giovanni", "Ciatto", LocalDate.of(1992, 4, 7))
private val person2 = person.clone(birthDate = LocalDate.of(1992, 4, 8), id = TaxCode("CTTGNN92D08D468M"))
private val vatNumber = VatNumber(12345678987L)
private val company = Customer.company(vatNumber, "ACME", "Inc.", LocalDate.of(1920, 1, 1))
@Test
fun complexTestWhichShouldActuallyBeDecomposedInSmallerTests() {
assertEquals(
expected = emptyList(),
actual = repository.findById(taxCode),
)
assertEquals(
expected = emptyList(),
actual = repository.findById(vatNumber),
)
repository.add(person)
repository.add(company)
assertEquals(
expected = listOf(person),
actual = repository.findById(taxCode),
)
assertEquals(
expected = listOf(company),
actual = repository.findById(vatNumber),
)
repository.remove(vatNumber)
repository.update(taxCode, person2)
}
}
Unit thest for the SqlCustomerRepository
class
Requires an instance of MariaDB to be tested
Let’s use Docker Compose to start / stop MariaDB
Try to complete the implementation of the setUpMariaDb
and tearDownMariaDb
methods
ProcessBuilder
s to call docker compose
commandsTemplate for Docker Compose files available among test resources
docker-compose.yml.template
Code for the exercise:
On one node running the Docker deamon (say, the teacher’s machine), initialise Swarm mode as follows:
docker swarm init
Swarm initialized: current node (<NODE_ID>) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join --token <SECRET_TOKEN> <NODE_ADDRESS>:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
On other nodes running the Docker daemon (say students’ machines), initialise Swarm mode as follows:
docker swarm join --token <SECRET_TOKEN> <NODE_ADDRESS>:2377
SECRET_TOKEN
and NODE_ADDRESS
are the ones provided by the docker swarm init
command aboveOn the master node, one may inspect the node composition of the cluster via docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
4q3q3q3q3q3q3q3q3q3q3q3q3q * node-1 Ready Active Leader 24.0.7
5q4q4q4q4q4q4q4q4q4q4q4q4q node-2 Ready Active 24.0.7
6q5q5q5q5q5q5q5q5q5q5q5q5q node-3 Ready Active 24.0.7
The master node may promote other nodes to manager nodes via docker node promote <NODE_ID>
…
docker node demote <NODE_ID>
docker node rm <NODE_ID>
Whenever done with this lecture, one may leave the Swarm via docker swarm leave --force
There may be a firewall blocking the communication among nodes
docker swarm join
hangs or fails with a timeout
2377/tcp
, 7946/tcp
, 7946/udp
, 4789/udp
You use Docker on Windows or Mac, hence the Docker daemon runs on a virtual machine
NODE_ADDRESS
in docker swarm join --token <SECRET_TOKEN> <NODE_ADDRESS>:2377
returned by docker swarm init
is the IP of the virtual machineipconfig
on Windows, ifconfig
on Mac), let’s call it ACTUAL_IP
ACTUAL_IP
towards NODE_ADDRESS
2377
, 7946
, 4789
)ACTUAL_IP
insteaf of NODE_ADDRESS
in docker swarm join
when joining the clusterOn an administrator Powershell, run the following commands:
netsh advfirewall firewall add rule name="Docker Swarm Intra-manager" dir=in action=allow protocol=TCP localport=2377
netsh advfirewall firewall add rule name="Docker Swarm Overlay Network Discovery TCP" dir=in action=allow protocol=TCP localport=7946
netsh advfirewall firewall add rule name="Docker Swarm Overlay Network Discovery UDP" dir=in action=allow protocol=UDP localport=7946
netsh advfirewall firewall add rule name="Docker Swarm Overlay Network Traffic" dir=in action=allow protocol=UDP localport=4789
On an administrator Powershell, run the following commands:
netsh interface portproxy add v4tov4 listenport=2377 listenaddress=$ACTUAL_IP connectport=2377 connectaddress=$NODE_ADDRESS
netsh interface portproxy add v4tov4 listenport=7946 listenaddress=$ACTUAL_IP connectport=7946 connectaddress=$NODE_ADDRESS
netsh interface portproxy add v4tov4 listenport=4789 listenaddress=$ACTUAL_IP connectport=4789 connectaddress=$NODE_ADDRESS
One may deploy a stack on a Swarm cluster via docker stack deploy -c <COMPOSE_FILE_PATH> <STACK_NAME>
COMPOSE_FILE_PATH
is the path to a .yml
file following the Docker Compose syntaxSTACK_NAME
is the name of the stack to be deployedThe command must be run on a client connected to some manager node
Other operations one may do on stacks (via a master node):
docker stack ls
to list stacksdocker stack ps <STACK_NAME>
to list containers in a stackdocker stack services <STACK_NAME>
to list services in a stackdocker stack rm <STACK_NAME>
to remove a stackSemantics of stack deployment on Swarms is different than Docker Compose:
overlay
driver, to support inter-node communication among containersversion: '3.2'
services:
agent:
image: portainer/agent:latest
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker/volumes:/var/lib/docker/volumes
networks:
- agents_network
deploy:
mode: global # i.e., one (and only one) replica per node
webservice:
image: portainer/portainer-ce:latest
command: -H tcp://tasks.agent:9001 --tlsskipverify
ports:
- "443:9443"
- "9000:9000"
- "8000:8000"
volumes:
- portainer_data:/data
networks:
- agents_network
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.role == manager
networks:
agents_network:
driver: overlay
attachable: true
volumes:
portainer_data:
Portainer is a Web-based dashboard for cluster administration
It is composed of two services:
webservice
, the actual dashboardagent
, a service running on each node, which exposes the Docker API to the webservice
serviceLet’s save the code on the left in a file named portainer.yml
Let’s deploy the stack via docker stack deploy -c portainer.yml portainer
Let’s then browse to https://localhost on some master node
let’s use software-process-engineering
as password for the admin
user
Upon login, one is presented with the home page, asking you to select one enviroment
Upon environment selection, one is presented with the dashboard, showing the status of the cluster
After selecting the only environment available, one may observe:
which and how many nodes compose the cluster (Dashboard section)
which stacks are running on the cluster (Services section)
portainer
stack is runningwhich services are running on the cluster (Services section)
portainer
stack is composed of two services: agent
and webservice
which containers are running on the cluster (Containers section)
portainer_agent
service is composed by one container per nodeportainer_webservice
service is composed by one container which is running on some master nodewhich volumes are present on the cluster (Volumes section)
portainer_portainer_data
volume is present on the same master node of portainer_webservice
which networks are present on the cluster (Networks section)
portainer_agents_network
network is present, using the overlay
driverNodes may be labelled with
key=value
pairs
Labels may be used to mark nodes with respect to their properties
capabilities.cpu=yes
capabilities.web=yes
capabilities.storage=yes
This can be done programmatically….
docker node update --label-add KEY=VALUE NODE_ID
… or via Portainer:
labels can be used to set placement constraints or preferences for services:
version: "3.9"
services:
db:
image: mariadb:latest
...
deploy:
placement:
constraints:
- node.labels.capabilities.storage==yes
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- "8889:80"
...
deploy:
placement:
constraints:
- node.labels.capabilities.web==yes
volumes:
db_data: {}
wordpress_data: {}
networks:
back-tier: {}
Let’s define a stack for Wordpress, with placement constraints
db
service on a node with storage capabilitieswordpress
service on a node with web capabilitiesOn the starwai
cluster:
storage1.stairwai.ce.almaai.unibo.it
has storage capabilitiesinference2.stairwai.ce.almaai.unibo.it
has web capabilitiesThe deployment is as follows:
When a service exposes a port, and that’s deployed on a Swarm:
the service is exposed on all master nodes in the Swarm
so, the wordpress
service above could be visited from several URL:
… as well as from the node which is actually hosting the service:
Services may be replicated for the sake scalability / load-balancing / fault-tolerance, etc.
Two types of services:
global
: replicated exactly once per nodereplicated
: replicated N
times (with N=1
by default)General case:
services:
service-name:
...
deploy:
mode: replicated
replicas: N # this should be a number!
placement:
constraints:
- node.role == manager
- node.labels.mylabel=myvalue
preferences:
- spread: node.labels.mylabel
max_replicas_per_node: M # this should be a number!
How to read it:
N
replicas of the service will be deployedmylabel
label set to myvalue
M
replicas will be put on the same nodeversion: "3.9"
services:
ws:
image: pikalab/hitcounter:latest
ports:
- "8890:8080"
deploy:
replicas: 10
placement:
constraints:
- node.labels.capabilities.cpu==yes
preferences:
- spread: node.labels.capabilities.cpu
Try to query it multiple times:
while true; do INT ✘
curl http://clusters.almaai.unibo.it:8890;
echo
sleep 1
done
expected outcome:
[9a6f89a39f8fd379@a3838ab6a606:8080] Hit 1 times
[26c39b9f3d7f0cab@4ad25c3e148c:8080] Hit 1 times
[97945332d20e71ed@969586109dee:8080] Hit 1 times
[3b3256852103312a@5d3479044fac:8080] Hit 2 times
[49e75aaae823c1e4@36e31b946486:8080] Hit 2 times
[497583c21d17ad58@7c10a84858f3:8080] Hit 2 times
[cf82689c910dfdf3@8aaa34535824:8080] Hit 2 times
[9a456666c413f4a4@5ba876916f68:8080] Hit 2 times
[41cadc9e0d7b0225@6cf54f65cda6:8080] Hit 2 times
[84dbbff08ac91e74@4006352af1e0:8080] Hit 2 times
[9a6f89a39f8fd379@a3838ab6a606:8080] Hit 2 times
[26c39b9f3d7f0cab@4ad25c3e148c:8080] Hit 2 times
[97945332d20e71ed@969586109dee:8080] Hit 2 times
[3b3256852103312a@5d3479044fac:8080] Hit 3 times
[49e75aaae823c1e4@36e31b946486:8080] Hit 3 times
...
Serval replicas of the same service are created:
Volumes are always local w.r.t. the node the current container is deployed onto
version: "3.9"
services:
simulation:
image: pikalab/fakesim:latest
restart: on-failure
deploy:
replicas: 3
placement:
constraints:
- node.labels.capabilities.cpu==yes
max_replicas_per_node: 1
volumes:
- type: volume
source: shared_volume
target: /data
volumes:
shared_volume: {}
Several replicas of the simulation are created…
… as well as several volumes!
How to share data?