Getting-Started | Haobin Tan

Getting Started

Thu, 06 Feb 2020 00:00:00 +0000

What is Docker?

Sat, 12 Dec 2020 00:00:00 +0000

Overview

Docker is an open platform for developing, shipping, and running applications.
Docker enables you to separate your applications from your infrastructure so you can deliver software quickly.
With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can significantly reduce the delay between writing code and running it in production. 👏

Docker architecture

Docker engine

Docker Engine is a client-server application with these major components:

A server which is a type of long-running program called a daemon process (the dockerd command).
A REST API which specifies interfaces that programs can use to talk to the daemon and instruct it what to do.
A command line interface (CLI) client (the docker command).

Docker images

An image is a read-only template with instructions for creating a Docker container.
Often, an image is based on another image, with some additional customization.
- For example, you may build an image which is based on the ubuntu image, but installs the Apache web server and your application, as well as the configuration details needed to make your application run.
You might create your own images or you might only use those created by others and published in a registry.
- To build your own image, you create a Dockerfile with a simple syntax for defining the steps needed to create the image and run it.
  - Each instruction in a Dockerfile creates a layer in the image.
  - When you change the Dockerfile and rebuild the image, only those layers which have changed are rebuilt.

Docker containers

A container is a runnable instance of an image.
You can create, start, stop, move, or delete a container using the Docker API or CLI. You can connect a container to one or more networks, attach storage to it, or even create a new image based on its current state.
Containerization is increasingly popular because containers are:
- Flexible: Even the most complex applications can be containerized.
- Lightweight: Containers leverage and share the host kernel, making them much more efficient in terms of system resources than virtual machines.
- Portable: You can build locally, deploy to the cloud, and run anywhere.
- Loosely coupled: Containers are highly self sufficient and encapsulated, allowing you to replace or upgrade one without disrupting others.
- Scalable: You can increase and automatically distribute container replicas across a datacenter.
- Secure: Containers apply aggressive constraints and isolations to processes without any configuration required on the part of the user.

Registries

A Docker registry stores Docker images. Docker Hub is a public registry that anyone can use, and Docker is configured to look for images on Docker Hub by default. You can even run your own private registry.
When you use the docker pull or docker run commands, the required images are pulled from your configured registry. When you use the docker push command, your image is pushed to your configured registry.

Docker Vs. Virtual Machines (VMs)

A container runs natively on Linux and shares the kernel of the host machine with other containers. It runs a discrete process, taking no more memory than any other executable, making it lightweight.

By contrast, a virtual machine (VM) runs a full-blown “guest” operating system with virtual access to host resources through a hypervisor. In general, VMs incur a lot of overhead beyond what is being consumed by your application logic.

Get Docker

See: Get Docker

Workflow overview

Tutorials

Reference

Docker overview

Container

Sat, 12 Dec 2020 00:00:00 +0000

TL;DR

Lifecycle

docker create creates a container but does not start it.
docker rename allows the container to be renamed.
docker run creates and starts a container in one operation.
docker rm deletes a container.
docker update updates a container’s resource limits.

Starting and stpping

docker start starts a container so it is running.
docker stop stops a running container.
docker restart stops and starts a container.
docker pause pauses a running container, “freezing” it in place.
docker unpause will unpause a running container.
docker wait blocks until running container stops.
docker kill sends a SIGKILL to a running container.
docker attach will connect to a running container.

Info

docker ps shows running containers.
- docker ps -a shows running and stopped containers.
docker logs gets logs from container. (You can use a custom log driver, but logs is only available for json-file and journald in 1.10).
docker inspect looks at all the info on a container (including IP address).
docker events gets events from container.
docker port shows public facing port of container.
docker top shows running processes in container.
docker stats shows containers’ resource usage statistics.
- docker stats --all shows a list of all containers, default shows just running.
docker diff shows changed files in the container’s FS.

Import/Export

docker cp copies files or folders between a container and the local filesystem.
docker export turns container filesystem into tarball archive stream to STDOUT.

Executing commands

docker exec to execute a command in container.

What is Docker container?

Containers are an abstraction at the app layer that packages code and dependencies together.
Multiple containers can run on the same machine and share the OS kernel with other containers, each running as isolated processes in user space.
Containers take up less space than VMs (container images are typically tens of MBs in size), can handle more applications and require fewer VMs and Operating systems.

Basic Usages

Pulling image

We can use docker pull to pull images from registry

E.g.: Pull ubuntu image

docker pull ubuntu

Running a container

Running of containers is managed with the Docker run command.

For example, we creat a container using an ubuntu:15.10 image and start it:

docker run -it ubuntu:15.10 /bin/bash

-i : interactive
-t : terminal
ubuntu:15.10: ubuntu image (version tag 15.10)
/bin/bash : command behind image

If we want our docker service to run in background, we can use argument -d. For example:

docker run -itd --name ubuntu-test-1 ubuntu /bin/bash

--name: assign a name ubuntu-test-1 to this container
-d: If we add this argument, it won’t enter the container by default

Listing containers

Listing running containers:

docker ps

Listing all containers

docker ps -a

Starting a container

docker start <stopped-container-ID>=

Ruuning a command in a running container

docker exec <container-name> <command>

For example, we run a container firstly in background

docker run -itd --name ubuntu-test-1 ubuntu /bin/bash

Check the running container:

docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
67cbb5de95c2 ubuntu "/bin/bash" 10 minutes ago Up 10 minutes ubuntu-test-1

Now we want to print Hello world in the terminal of container ubuntu-test-1 :

docker exec -it ubuntu-test-1 echo "Hello World"

Hello World

Stopping a container

docker stop <container-ID>

Removing a container

docker rm -f <container-ID>

Reference

Image

Sun, 13 Dec 2020 00:00:00 +0000

TL;DR

Lifecycle

docker images shows all images.
docker import creates an image from a tarball.
docker build creates image from Dockerfile.
docker commit creates image from a container, pausing it temporarily if it is running.
docker rmi removes an image.
docker load loads an image from a tar archive as STDIN, including images and tags (as of 0.7).
docker save saves an image to a tar archive stream to STDOUT with all parent layers, tags & versions (as of 0.7).

Info

docker history shows history of image.
docker tag tags an image to a name (local or registry).

Cleaning up

docker image prune is also available for removing unused images. (See Prune).

Basic usages

In Docker, everything is based on Images. Images are templates for creating docker containers.

Listing images

docker images

REPOSITORY TAG IMAGE ID CREATED SIZE
docker-hello-world_web latest 85f9e3024e99 14 hours ago 196MB
my_image 0.0.1 9f3378bbf7e4 36 hours ago 133MB
nginx v3 6ed4b5e97df7 37 hours ago 133MB
python 3.7-alpine 0ce5215b0b31 43 hours ago 41.1MB
nginx latest 7baf28ea91eb 2 days ago 133MB
ubuntu latest f643c72bc252 2 weeks ago 72.9MB

REPOSITORY : repository source of image
TAG : tag of image
- There could be a number of tags in the same repository source, representing different version of this repository source. For example, in ubuntu repository source, there are many different versions such as 15.10, 14.04, etc. We use REPOSITORY:TAG to specify different images
- For example, if we want to run container with ubuntu version 15.10:
```
docker run -t -i ubuntu:15.10 /bin/bash
```
IMAGE ID : ID of image
CREATED: creation time of image
SIZE : size of image

Downloading images

Docker automatically downloads a non-existent image when we use it on the local host. If we want to pre-download the image, we can use the docker pull command to download it.

For example, we download ubuntu:13.10, which doesn’t exist in the local machine:

docker pull ubuntu:13.10

Searching images

We can search images in Docker Hub.

Removing images

Removing image using image ID:

docker rmi <image-ID>

Removing image using image name

docker rmi <image-name>

Building image

We use the command docker build to create a new image from scratch. To do this, we need to create a Dockerfile file that contains a set of instructions to tell Docker how to build our image. (More see Dockerfile)

Tagging image

docker tag <image-ID> <image-tag>

Reference

Dockerfile

Sun, 13 Dec 2020 00:00:00 +0000

TL;DR

What is Dockerfile?

A Dockerfile is a text file that defines a Docker image. You’ll use a Dockerfile to create your own custom Docker image, in other words to define your custom environment to be used in a Docker container.

A Dockerfile is a step by step definition of building up a Docker image. The Dockerfile contains a list of instructions that Docker will execute when you issue the docker build command. Your workflow is like this:

Create the Dockerfile and define the steps that build up your images
Issue the docker build command which will build a Docker image from your Dockerfile
Use this image to start containers with the docker run command

Dockerfile commands

FROM

The FROM instruction initializes a new build stage and sets the Base Image for subsequent instructions.
As such, a valid Dockerfile must start with a FROM instruction.
- Note: ARG is the only instruction that may precede FROM in the Dockerfile.
The image can be any valid image – it is especially easy to start by pulling an image from the Public Repositories.

Syntax:

FROM [--platform=<platform>] <image>[:<tag>] [AS <name>]

Example:
```
FROM ubuntu:18.04
```

RUN

Execute a command in a shell or exec form.
The RUN instruction adds a new layer on top of the newly created image.
The committed results are then used for the next instruction in the DockerFile.
Syntax
- Shell form (the command is run in a shell, which by default is /bin/sh -c on Linux)
```
RUN <command>
```
  In the shell form you can use a \ (backslash) to continue a single RUN instruction onto the next line.
  
  E.g.
```
RUN /bin/bash -c 'source $HOME/.bashrc; \
echo $HOME'
```
  Together they are equivalent to this single line:
```
RUN /bin/bash -c 'source $HOME/.bashrc; echo $HOME'
```
- Exec form
```
RUN ["executable", "param1", "param2"]
```
  E.g.
```
RUN ["/bin/bash", "-c", "echo hello"]
```

CMD

Give the default commands when the image is instantiated, it doesn’t execute while build stage.
The CMD instruction has three forms:
- CMD ["executable","param1","param2"] (exec form, this is the preferred form)
  - It will NOT involke a command shell. This means that normal shell processing does NOT happen.
  E.g
```
CMD ["echo","hello", "world"]
```
- CMD ["param1","param2"] (as default parameters to ENTRYPOINT)
  - If you would like your container to run the same executable every time, then you should consider using ENTRYPOINT in combination with CMD. See ENTRYPOINT.
- CMD command param1 param2 (shell form)
  - The <command> will execute in /bin/sh -c.
  - E.g.
```
CMD echo $HW
```
    It will execute the command /bin/sh -c echo $HW
There can only be ONE CMD instruction in a Dockerfile. If you list more than one CMD then only the last CMD will take effect.
The main purpose of a CMD is to provide defaults for an executing container. These defaults can include
- an executable, or
- they can omit the executable, in which case you must specify an ENTRYPOINT instruction as well.

If the user specifies arguments to docker run then they will override the default specified in CMD.

ENTRYPOINT

Configures a container that will run as an executable. I.e., a specific application can be set as default and run every time a container is created using the image.

ENTRYPOINT has two forms:

exec form (the preferred form)

ENTRYPOINT ["executable", "param1", "param2"]

shell form:
```
ENTRYPOINT command param1 param2
```

Unlike CMD, command line arguments to docker run <image> will NOT override the default specified in CMD. Instead, they will be appended after all elements in an exec form ENTRYPOINT, and will override all elements specified using CMD.
Only the last ENTRYPOINT instruction in the Dockerfile will have an effect.
You can override the ENTRYPOINT instruction using the docker run --entrypoint flag.

Understand how CMD and ENTRYPOINT interact

Both CMD and ENTRYPOINT instructions define what command gets executed when running a container. There are few rules that describe their co-operation.

Dockerfile should specify at least one of CMD or ENTRYPOINT commands.
ENTRYPOINT should be defined when using the container as an executable.
CMD should be used as a way of defining default arguments for an ENTRYPOINT command or for executing an ad-hoc command in a container.
CMD will be overridden when running the container with alternative arguments.

The table below shows what command is executed for different ENTRYPOINT / CMD combinations:

	No ENTRYPOINT	ENTRYPOINT exec_entry p1_entry	ENTRYPOINT [“exec_entry”, “p1_entry”]
No CMD	error, not allowed	`/bin/sh -c exec_entry p1_entry`	`exec_entry p1_entry`
CMD [“exec_cmd”, “p1_cmd”]	`exec_cmd p1_cmd`	`/bin/sh -c exec_entry p1_entry`	`exec_entry p1_entry exec_cmd p1_cmd`
CMD [“p1_cmd”, “p2_cmd”]	`p1_cmd p2_cmd`	`/bin/sh -c exec_entry p1_entry`	`exec_entry p1_entry p1_cmd p2_cmd`
CMD exec_cmd p1_cmd	`/bin/sh -c exec_cmd p1_cmd`	`/bin/sh -c exec_entry p1_entry`	`exec_entry p1_entry /bin/sh -c exec_cmd p1_cmd`

A common use case is: the executable is defined with ENTRYPOINT, while CMD specifies the default parameter.

Example:

Dockerfile

FROM node:8.11-slim

CMD ["world"]

ENTRYPOINT ["echo", "Hello"]

Build image:

docker built -t my-docker-image .

Run container:

without argument
```
docker run -it my-docker-image
```
output:
```
Hello world
```
If we run docker run without argument, the default argument world defined in CMD will be used. I.e, the command echo Hello World will be executed.
with argument
```
docker run -it my-docker-image James Bond
```
output:
```
Hello James Bond
```
If we specify arguments when running docker run, CMD will be overriden by our given arguments. In our case, CMD ["World"] is overriden to CMD ["James", "Bond"] and then applied to echo. Thus, the command echo Hello James Bond will be executed.

Summary of CMD and ENTRYPOINT see: Docker CMD Vs Entrypoint Commands: What’s The Difference?

EXPOSE

Specify the port on which the container will be listening at runtime by running the EXPOSE instruction.
Syntax
```
EXPOSE <port>
```

ENV

Set environment variables using the ENV instruction.
This value will be in the environment for all subsequent instructions in the build stage and can be replaced inline in many as well.
They come as key value pairs and increases the flexibility of running programs.

Syntax

ENV <key>=<value> ...

e.g.

ENV PATH=usr/node

ENV <key> <value>

e.g.

ENV PATH usr/node

ADD

ADD has two forms:

ADD [--chown=<user>:<group>] <src>... <dest>

ADD [--chown=<user>:<group>] ["<src>",... "<dest>"]

Copies new files, directories or remote file URLs from <src> and adds them to the filesystem of the image at the path <dest>.
Invalidates caches. Avoid ADD and use COPY instead.

COPY

Syntax

COPY [--chown=<user>:<group>] <src>... <dest>

COPY [--chown=<user>:<group>] ["<src>",... "<dest>"]

Copy new files or directories from <src> and adds them to the filesystem of the container at the path <dest>.
- Each <src> may contain wildcards and matching will be done using Go’s filepath.Match rules.COPY obeys the following rules:
Example: copy test.txt to <WORKDIR>/relativeDir/:
```
COPY test.txt relativeDir/
```
COPY obeys the following rules:
- The <src> path must be inside the context of the build
- If <src> is a directory, the entire contents of the directory are copied, including filesystem metadata.
  
  (The directory itself is not copied, just its contents.)
- If <src> is any other kind of file, it is copied individually along with its metadata. In this case, if <dest> ends with a trailing slash /, it will be considered a directory and the contents of <src> will be written at <dest>/base(<src>).
- If multiple <src> resources are specified, either directly or due to the use of a wildcard, then <dest> must be a directory, and it must end with a slash /.
- If <dest> does not end with a trailing slash, it will be considered a regular file and the contents of <src> will be written at <dest>.
- If <dest> doesn’t exist, it is created along with all missing directories in its path.

VOLUMN

Creates a mount point for externally mounted volumes or other containers

WORKDIR

Sets the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile.
The WORKDIR instruction can be used multiple times in a Dockerfile. If a relative path is provided, it will be relative to the path of the previous WORKDIR instruction. E.g.
```
WORKDIR /usr/node
WORKDIR app
```
The resulted work directory will be /usr/mode/app
The WORKDIR instruction can resolve environment variables previously set using ENV. E.g.
```
ENV WORK_DIR=/usr/node/app

WORKDIR ${WORK_DIR}
```

ARG

Defines a build-time variable
Syntax
```
ARG <name>[=<default value>]
```
ARG is the only instruction that can occur before the FROM.

E.g. we can use ARG to specify the version of base image
```
ARG NODE_VERSION=8.11-slim

FROM node:${NODE_VERSION}
```

ONBUILD

Adds a trigger instruction when the image is used as the base for another build

STOPSIGNAL

Sets the system call signal that will be sent to the container to exit.

LABEL

Add key/value metadata to your images, containers, or daemons.

Syntax

LABEL <key>=<value> <key>=<value> <key>=<value> ...

Example

LABEL version="1.0"
LABEL description="Just a demo"

If a label already exists but with a different value, the most-recently-applied value overrides any previously-set value.
To view an image’s labels, use the docker image inspect command. You can use the --format option to show just the labels
```
docker image inspect --format='' <image>
```

SHELL

Override default shell is used by docker to run commands.

HEALTHCHECK

Tells docker how to test a container to check that it is still working.

Example with common commands

# ARG is used to pass some arguments to consecutive instructions
# this is only command other than a comment can be used before FROM.
ARG NODE_VERSION=8.11-slim

# from base image node
FROM node:${NODE_VERSION}

# LABEL add metadata to your images, containers, or daemons
LABEL "about"="This file is just an example to demostrate the basic usage of dockerfile commands"

# ENV sets the environment variables for the subsequent instructions in the build stage
ENV WORK_DIR /usr/node

# WORKDIR sets the working directory for all the consecutive commands.
# We can have multiple WORKDIR commands and will be appended with a relative path.
# E.g. we have two WORKDIR commands leads to /usr/node/app
WORKDIR ${WORK_DIR}
WORKDIR app

# VOLUME is used to create a mount point with the specified name
RUN mkdir /dockerexample
VOLUME /dockerexample

# COPY is used to copy files or directories 
# from source host filesystem to a destination in the container file system.
# Here we're gonna copy package.json from our system to container file system
COPY package.json .

# RUN executes the instructions in a new layer on top of the existing image and commit those layers
# The resulted layer will be used for the next instructions in the Dockerfile
RUN ls -ll && \
 npm install

# USER instruction sets the user name and optionally the user group to use
# when running the image and for any instructions that follow it in the Dockerfile
RUN useradd ecko
USER ecko

# ADD is used to add files or directories and remote files 
# from URL from source host filesystem to a destination in the container file system.
# Avoid ADD and use COPY instead!
ADD index.js .

# CMD command is used to give the default commands when the image is instantiated, 
# it doesn’t execute while build stage. 
# There should be only ONE CMD per Dockerfile, 
# you can list multiple but the last one will be executed.
CMD ["echo", "Hello World"]

# EXPOSE informs Docker that 
# the container listens on the specified network ports at runtime. 
EXPOSE 3070

# ENTRYPOINT is used as an executable for the container.
# We can use ENTRYPOINT for executable command 
# and use CMD command to pass some default commands to the executable.
ENTRYPOINT ["echo", "Hello"]

Reference

Tutorials
- 🔥 Step by step tutorial with example: Docker — A Beginner’s guide to Dockerfile with a sample project
- Dockerfile tutorial by example - basics and best practices [2018]
- Docker Dockerfile
- Docker Tutorial Series, Part 3: Automation is the Word Using DockerFile
Reference: Dockerfile reference
Cheatsheet

Dockerfile Best Practice

Sun, 13 Dec 2020 00:00:00 +0000

General Guidelines

Write `.dockerignore`

When building an image, Docker has to prepare context first - gather all files that may be used in a process.
Default context contains all files in a Dockerfile directory. Usually we don’t want to include there .git directory, downloaded libraries, and compiled files.
Similar to .gitignore, .dockerignore can look like followings:

.dockerignore
```
.git/
node_modules/
dist/
```

Container should do one thing (Decouple applications)

Technically, you can start multiple processes (such as database, frontend, backend applications) inside Docker container. However, such a big container will bite you

long build times (change in e.g. frontend will force the whole backend to rebuild)
very large images
hard logging from many applications (no more simple stdout)
wasteful horizontal scaling
problems with zombie processes - you have to remember about proper init process

The proper way is

prepare separate Docker image for each component
use Docker Compose to easily start multiple containers at the same time.

Minimize the number of layers

Docker is all about layers.

Each command in Dockerfile creates so-called layer
Layers are cached and reused
Invalidating cache of a single layer invalidates all subsequent layers
Invalidation occurs after command change, if copied files are different, or build variable is other than previously
Layers are immutable, so if we add a file in one layer, and remove it in the next one, image STILL contains that file (it’s just not available in the container)!

Minimizing the number of steps in your image may improve build and pull performance. Therefore it’s a cool best practice to combine several steps into one line, so that they’ll create only one intermediary image.

Only RUN, COPY and ADD instructions create layers to improve build performance. Other instructions create temporary intermediate images, and do not increase the size of the build.

Example

❌

FROM alpine:3.4

RUN apk update
RUN apk add curl
RUN apk add vim
RUN apk add git

✅

FROM alpine:3.4

RUN apk update && \
 apk add curl && \
 apk add vim && \
 apk add git

After building this Dockerfile the usual way you’ll find that this time it has only taken 2 steps instead of 5.

Sort multi-line arguments

Whenever possible, ease later changes by sorting multi-line arguments alphanumerically.
This helps to avoid duplication of packages and make the list much easier to update. This also makes PRs a lot easier to read and review. Adding a space before a backslash (\) helps as well.

Example

RUN apt-get update && apt-get install -y \
 bzr \
 cvs \
 git \
 mercurial \
 subversion \
 && rm -rf /var/lib/apt/lists/*

Do not use ’latest’ base image tag

latest tag is a default one, used when no other tag is specified. E.g. our instruction FROM ubuntu in reality does exactly the same as FROM ubuntu:latest
But ’latest’ tag will point to a different image when a new version will be released, and your build may break. 😢
So, unless you are creating a generic Dockerfile that must stay up-to-date with the base image, provide specific tag!!!

Remove unneeded files after each RUN step

Let’s assume we updated apt-get sources, installed few packages required for compiling others, downloaded and extracted archives. We obviously don’t need them in our final images, so better let’s make a cleanup.

E.g. we can remove apt-get lists (created by apt-get update):

FROM ubuntu:16.04

RUN apt-get update \
 && apt-get install -y nodejs \
 # added lines
 && rm -rf /var/lib/apt/lists/*

ADD . /app
RUN cd /app && npm install

CMD npm start

Use proper base image

Use specilaized image instead of general-purpose base image. For example, if we just want to run node application, instead of using ubuntu as our base image, we should use node (or even alpine version).

Set `WORKDIR` and `CMD`

WORKDIR command changes default directory, where we run our RUN / CMD / ENTRYPOINT commands.
CMD is a default command run after creating container without other command specified. It’s usually the most frequently performed action.

Example

FROM node:7-alpine

WORKDIR /app

ADD . /app

RUN npm install

CMD ["npm", "start"]

Use `ENTRYPOINT` (optional)

Use “exec” inside entrypoint script

Prefer `COPY` over `ADD`

COPY is simpler.
ADD has some logic for downloading remote files and extracting archives (more see official documentation)
Just stick with COPY !💪

Use multi-stage builds

Multi-stage builds allow you to drastically reduce the size of your final image, without struggling to reduce the number of intermediate layers and files.
Because an image is built during the final stage of the build process, you can minimize image layers by leveraging build cache.
If your build contains several layers, you can order them from the less frequently changed (to ensure the build cache is reusable) to the more frequently changed:
- Install tools you need to build your application
- Install or update library dependencies
- Generate your application

Leverage build cache

When building an image, Docker steps through the instructions in your Dockerfile, executing each in the order specified. As each instruction is examined, Docker looks for an existing image in its cache that it can reuse, rather than creating a new (duplicate) image.
The basic rules that Docker follows are outlined below:
- Starting with a parent image that is already in the cache, the next instruction is compared against all child images derived from that base image to see if one of them was built using the exact same instruction. If not, the cache is invalidated.
- In most cases, simply comparing the instruction in the Dockerfile with one of the child images is sufficient.
- For the ADD and COPY instructions, the contents of the file(s) in the image are examined and a checksum is calculated for each file. The last-modified and last-accessed times of the file(s) are not considered in these checksums. During the cache lookup, the checksum is compared against the checksum in the existing images. If anything has changed in the file(s), such as the contents and metadata, then the cache is invalidated.
- Aside from the ADD and COPY commands, cache checking does not look at the files in the container to determine a cache match.
- Once the cache is invalidated, all subsequent Dockerfile commands generate new images and the cache is not used.

Specify default environment variables, ports and volumes

It’s a good practice to set default values in Dockerfile, which can make our dockerfile more consistent and flexible.

Example

FROM node:7-alpine

# ENV variables required during build
ENV PROJECT_DIR=/app

WORKDIR $PROJECT_DIR

COPY package.json $PROJECT_DIR
RUN npm install
COPY . $PROJECT_DIR

ENV MEDIA_DIR=/media \
 NODE_ENV=production \
 APP_PORT=3000
 
VOLUME $MEDIA_DIR
EXPOSE $APP_PORT

Add metadata to image using LABEL

Create ephemeral containers

The image defined by your Dockerfile should generate containers that are as ephemeral as possible. By “ephemeral”, we mean that the container can be stopped and destroyed, then rebuilt and replaced with an absolute minimum set up and configuration.

Don’t install unnecessary packages

To reduce complexity, dependencies, file sizes, and build times, avoid installing extra or unnecessary packages just because they might be “nice to have.”

Dockerfile instructions

FROM

Whenever possible, use current official images as the basis for your images.
We recommend the Alpine image as it is tightly controlled and small in size (currently under 5 MB), while still being a full Linux distribution.

LABEL

You can add labels to your image to help organize images by project, record licensing information, to aid in automation, or for other reasons. (More see: Understanding object labels)

Acceptable formats

One label per line

LABEL com.example.version="0.0.1-beta"
LABEL com.example.release-date="2015-02-12"

Multiple labels on one line

LABEL com.example.version="0.0.1-beta" com.example.release-date="2015-02-12"

Set multiple labels at once, using line-continuation characters to break long lines

LABEL com.example.version="0.0.1-beta" \
 com.example.release-date="2015-02-12"

RUN

Split long or complex RUN statements on multiple lines separated with backslashes to make your Dockerfile more readable, understandable, and maintainable.

`apt-get`

Avoid RUN apt-get upgrade and dist-upgrade
Always combine RUN apt-get update with apt-get install in the same RUN statement. This ensures your Dockerfile installs the latest package versions with no further coding or manual intervention. (“cache busting”)

E.g.
```
RUN apt-get update && apt-get install -y \
 package-bar \
 package-baz \
 package-foo \
 && rm -rf /var/lib/apt/lists/*
```

You can also achieve cache-busting by specifying a package version. (“cache busting”)

E.g.

RUN apt-get update && apt-get install -y \
 package-bar \
 package-baz \
 package-foo=1.3.*

Below is a well-formed RUN instruction that demonstrates all the apt-get recommendations.

RUN apt-get update && apt-get install -y \
 aufs-tools \
 automake \
 build-essential \
 curl \
 dpkg-sig \
 libcap-dev \
 libsqlite3-dev \
 mercurial \
 reprepro \
 ruby1.9.1 \
 ruby1.9.1-dev \
 s3cmd=1.1.* \
 && rm -rf /var/lib/apt/lists/*

CMD

The CMD instruction should be used to run the software contained in your image, along with any arguments.
CMD should almost always be used in the form of CMD ["executable", "param1", "param2"…].
CMD should rarely be used in the manner of CMD ["param", "param"] in conjunction with ENTRYPOINT, unless you and your expected users are already quite familiar with how ENTRYPOINT works.

EXPOSE

The EXPOSE instruction indicates the ports on which a container listens for connections.
You should use the common, traditional port for your application. E.g.
- an image containing the Apache web server would use EXPOSE 80
- an image containing MongoDB would use EXPOSE 27017

ENV

To make new software easier to run, you can use ENV to update the PATH environment variable for the software your container installs.
ENV instruction is also useful for providing required environment variables specific to services you wish to containerize

ENV can also be used to set commonly used version numbers so that version bumps are easier to maintain

E.g.

ENV PG_MAJOR=9.3
ENV PG_VERSION=9.3.4
RUN curl -SL https://example.com/postgres-$PG_VERSION.tar.xz | tar -xJC /usr/src/postgress && …
ENV PATH=/usr/local/postgres-$PG_MAJOR/bin:$PATH

Each ENV line creates a new intermediate layer, just like RUN commands
- This means that even if you unset the environment variable in a future layer, it still persists in this layer and its value can’t be dumped.
- You can separate your commands with ; or &&. Using \ as a line continuation character for Linux Dockerfiles improves readability.
- Or you could also put all of the commands into a shell script and have the RUN command just run that shell script.

ADD or COPY

COPY is preferred.
If you have multiple Dockerfile steps that use different files from your context, COPY them individually, rather than all at once. This ensures that each step’s build cache is only invalidated (forcing the step to be re-run) if the specifically required files change.

E.g.
```
COPY requirements.txt /tmp/
RUN pip install --requirement /tmp/requirements.txt
COPY . /tmp/
```
This results in fewer cache invalidations for the RUN step, than if you put the COPY . /tmp/ before it.

Using ADD to fetch packages from remote URLs is strongly discouraged 🙅‍♂️; you should use curl or wget instead.

That way you can delete the files you no longer need after they’ve been extracted and you don’t have to add another layer in your image.

E.g.

❌

ADD https://example.com/big.tar.xz /usr/src/things/
RUN tar -xJf /usr/src/things/big.tar.xz -C /usr/src/things
RUN make -C /usr/src/things all

✅

RUN mkdir -p /usr/src/things \
 && curl -SL https://example.com/big.tar.xz \
 | tar -xJC /usr/src/things \
 && make -C /usr/src/things all

For other items (files, directories) that do not require ADD’s tar auto-extraction capability, you should always use COPY.

ENTRYPOINT

The best use for ENTRYPOINT is to set the image’s main command, allowing that image to be run as though it was that command (and then use CMD as the default flags).
- Example: image for the command line tool s3cmd
```
ENTRYPOINT ["s3cmd"]
CMD ["--help"]
```
  Now the image can be run like this to show the command’s help:
```
docker run s3cmd
```
  or use the right parameters to execute a command:
```
docker run s3cmd ls s3://mybucket
```
The ENTRYPOINT instruction can also be used in combination with a helper script, allowing it to function in a similar way to the command above, even when starting the tool may require more than one step.

WORKDIR

For clarity and reliability, you should always use absolute paths for your WORKDIR.
You should use WORKDIR instead of proliferating instructions like RUN cd … && do-something, which are hard to read, troubleshoot, and maintain.

Reference

Official guide: Best practices for writing Dockerfiles
How to write excellent Dockerfiles

Docker Volume

Thu, 06 Feb 2020 00:00:00 +0000

In general, Docker containers are ephemeral, running just as long as it takes for the command issued in the container to complete. By default, any data created inside the container is ONLY available from within the container and only while the container is running.

Docker volumes can be used to share files between a host system and the Docker container.

Bindmounting a Volume

We can use -v flag in docker run to bind mount a volume. Let’s take a look at an example, which will create a directory called nginxlogs in your current user’s home directory and bindmount it to /var/log/nginx in the container:

docker run --name=nginx -d -v ~/nginxlogs:/var/log/nginx -p 5000:80 nginx

-v ~/nginxlogs:/var/log/nginx sets up a bindmount volume that links the /var/log/nginx directory from inside the Nginx container to the ~/nginxlogs directory on the host machine. Docker uses a : to split the host’s path from the container path, and the host path always comes first.

If the first argument of -v begins with a / or ~/, you’re creating a bindmount. Remove that, and you’re naming the volume.

-v /path:/path/in/container mounts the host directory, /path at the /path/in/container
-v path:/path/in/container creates a volume named path with no relationship to the host.

If you make any changes to the ~/nginxlogs folder, you’ll be able to see them from inside the Docker container in real time as well. In other words, the content of ~/nginxlogs and /var/log/nginx are synchronous.

Reference

Use bind mounts: Docker official documentation
How To Share Data Between the Docker Container and the Host
Docker Basics: How to Share Data Between a Docker Container and Host
Docker Volume - 目录挂载以及文件共享

Getting-Started | Haobin Tan

Getting Started

What is Docker?

Overview

Docker architecture

Docker engine

Docker images

Docker containers

Registries

Docker Vs. Virtual Machines (VMs)

Get Docker

Workflow overview

Tutorials

Reference

Container

TL;DR

What is Docker container?

Basic Usages

Pulling image

Running a container

Listing containers

Starting a container

Ruuning a command in a running container

Stopping a container

Removing a container

Reference

Image

TL;DR

Basic usages

Listing images

Downloading images

Searching images

Removing images

Building image

Tagging image

Reference

Dockerfile

TL;DR

What is Dockerfile?

Dockerfile commands

FROM

RUN

CMD

ENTRYPOINT

Understand how CMD and ENTRYPOINT interact

EXPOSE

ENV

ADD

COPY

VOLUMN

WORKDIR

ARG

ONBUILD

STOPSIGNAL

LABEL

SHELL

HEALTHCHECK

Example with common commands

Reference

Dockerfile Best Practice

General Guidelines

Write .dockerignore

Container should do one thing (Decouple applications)

Minimize the number of layers

Example

Sort multi-line arguments

Do not use ’latest’ base image tag

Remove unneeded files after each RUN step

Use proper base image

Set WORKDIR and CMD

Use ENTRYPOINT (optional)

Use “exec” inside entrypoint script

Prefer COPY over ADD

Use multi-stage builds

Leverage build cache

Specify default environment variables, ports and volumes

Add metadata to image using LABEL

Create ephemeral containers

Don’t install unnecessary packages

Dockerfile instructions

Write `.dockerignore`

Set `WORKDIR` and `CMD`

Use `ENTRYPOINT` (optional)

Prefer `COPY` over `ADD`

`apt-get`