Docker

What is Docker?

Docker is a tool that allows developers to package applications and all their dependencies in a container (sandbox) and deploy and run the application on host operating system (Linux).

What are Containers, Images?

A container is a discrete process that has been isolated from all other processes on the host machine. This is made possible by Kernel Namespaces and Cgroups. This decoupling allows applications to run consistently anywhere isolated from the rest of the applications. Containers run images, which contain everything required to run: application code, runtime environment, dependencies, scripts, etc. Containers run layers of stacked images (eg. a base linux image (mostly alpine due to its compact size), node image and application image on top).

Docker vs VMs

Note that docker containers do not run in their own virtual machines. VMs run on virtual hardware provided by host OS, whereas containers share the host kernel, thus avoiding computation overhead for virtualizing hardware. Also note that Docker requires Linux runtime environment, so running Docker on Windows/Mac requires a single Linux virtual machine that is shared among all the containers run in the system.

Installation

Download and install docker from Docker docs.

Check if installation was successful using

docker version

Docker Hub

Pull from docker repository

Pull image from repository (by default Docker hub unless specified otherwise).

docker pull redis

List all images

View all locally available images.

docker images

docker image ls

Run container

Creates a new container from an image, and execute the container. Pulls image from repository (Docker Hub) if container is not found locally. Notice that different layers are pulled separately from the repository. Splitting the layers provides the advantage that if some layers already exist locally, they are skipped from downloading again.

docker run postgres:9.6

docker run docker/whalesay cowsay Hello World!

Run in detached mode. Returns id of the container and starts running the container.

docker run -d postgres:9.6

Re-attach detached container. Requires id of detached container.

docker attach ee8f2eb0b433

docker ps

View all currently running containers.

docker ps

View all running and stopped containers (history). Containers can be restarted using the given container id.

docker ps -a

Stop container

Stops the running container. Requires id/name of container. Returns id/name of stopped container.

docker stop ee8f2eb0b433

Start container

Can be used to restart stopped container. Requires id/name of stopped container. Returns id/name of started container. It does not create a new container, unlike docker run. All configurations are restored which were set while running doc run command.

docker start ee8f2eb0b433

Port Mapping [Host port : Container port]

docker ps displays the port the application is listening inside the container. To connect to this port inside the container, we need the IP the application is running in. Every Docker container gets an IP assigned by default. This is an internal IP and can only be accessed from the host machine. Users outside of the docker host cannot access using this IP. For this, we need to use the IP of the docker host and map a free port from the docker host to the container port. You can also run multiple instances of application and map them to different ports on the host machine.

There should be a binding between the host port and container port. Conflicts will occur if you open two same ports on your host machine. Different containers can however listen on the same port because these two containers are completely isolated of one another.

docker run -p6000:6379 redis

Port 6000 of host machine is binded with port 6379 of container running redis. Redis runs on port 6479. This can be viewed using docker ps.

Display logs

View logs of docker container (eg. while using in detached mode). Requires id/name of running container.

docker logs 777f0e06c4c

docker logs 777f0e06c4c | tail

Custom naming containers

docker run -d --name my-custom-name redis

docker exec

Get the terminal of the running container as a root user. -it stands for interactive terminal. -i maps the standard input of the host to the Docker container. -t stands for pseudo terminal,which displays prompts from the terminal. Requires id/name of container. You can even navigate around the virtual file system, check logs, manage environment, etc.

docker exec -it 777f0e06c4c /bin/sh

docker exec 777f0e06c4c cat /etc/hosts

Docker network

Docker creates it's own isolated network where the containers are running in. When two containers are deployed in the same docker network, they can refer to each other using just the container name because they are in the same network. Applications running outside of docker connect to this network using localhost:port-number.

Display docker networks

Displays all docker networks.

docker network ls

Create new docker network

Creates new docker network

docker network create my-network

Docker run with network option

To run containers in custom docker network, it should be specified in the docker run command.

docker run -d \
-p27017:27017 \ 
-e MONGO_INITDB_ROOT_USERNAME=admin \
-e MONGO_INITDB_ROOT_PASSWORD=password \
--name mongodb \
--net my-network \
mongo

docker run -d \
-p8081:8081 \
-e ME_CONFIG_MONGODB_ADMINUSERNAME=admin \
-e ME_CONFIG_MONGODB_ADMINPASSWORD=password \
--net my-network \
--name mongo-express \
-e ME_CONFIG_MONGODB_SERVER=mongodb \
mongo-express

-e sets the required environment variables.

Docker inspect

View details of a container in JSON format such as mounts, configuration settings, network settings, etc. Requires id/name of container.

docker inspect dreamy_hellman

Docker Compose

With docker compose, docker commands can be mapped to a file, so that configurations are well structured. Note that format of Docker-compose yaml file depends on the version of docker-yaml.

version: '3'
services:
    mongodb:
        image: mongo
        ports:
            - 27017:27017
        environment:
            - MONGO_INITDB_ROOT_USERNAME=admin
            - MONGO_INITDB_ROOT_PASSWORD=password
    mongo-express:
        image: mongo-express
        ports:
            - 8080:8081
        environment:
            - ME_CONFIG_MONGODB_ADMINUSERNAME=admin
            - ME_CONFIG_MONGODB_ADMINPASSWORD=password
            - ME_CONFIG_MONGODB_SERVER=mongodb
        depends_on:
            - mongodb

The network configuration is not included in docker compose. Docker compose will automatically create a common network for all services listed in the docker compose file. Save it as a .yaml file.

Using Docker compose yaml file

Start a container

docker-compose -f my-docker-compose-file.yaml up

Stop a container

docker-compose -f my-docker-compose-file.yaml down

Dockerfile

A dockerfile is a document for building images. Filename must be Dockerfile.

FROM node

ENV MONGO_INITDB_ROOT_USERNAME=admin \
    MONGO_INITDB_ROOT_PASSWORD=password

RUN mkdir -p /home/app

COPY . /home/app

WORKDIR /home/app

CMD ["node","server.js"]

FROM image - build on a base image
ENV - set environment variables
RUN - execute any Linux commands on the container environment
COPY - executes on the host. Copies file from host to the container
WORKDIR - set current working directory so that you don't need to provide full path from /home.
CMD - executes an entry point. There can be multiple RUN commands, but only one CMD command. The command and parameters should be separate entries in the list.

FROM ubuntu

CMD sleep 5

Running docker run sleeper sleep 10, the sleep 10 will overwrite the command specified on CMD. To provide custom arguments, use the ENTRYPOINT command.

FROM ubuntu

ENTRYPOINT ["sleep"]

Running docker run sleeper 10 will append the argument 10 to the list of ENTRYPOINT.

To set a default value for argument, use the combination of both ENTRYPOINT and CMD. Both ENTRYPOINT and CMD should be specified in JSON format.

FROM ubuntu

ENTRYPOINT ["sleep"]

CMD ["5"]

If no arguments is provieded, argument from CMD will be appended to ENTRYPOINT. If arguments are provided, they will be appended to ENTRYPOINT and the values on CMD will be disregarded. You can also overwrite the whole entrypoint command using

docker run --entrypoint somethine-else sleeper 10

Building image from Dockerfile

docker build -t my-app:1.0 .

my-app - image name
1.0 - tag
. - path to Dockerfile

Delete image

Remove image from local host. Requires image id. Image id can be found running docker images. Image can be deleted if all containers built from the image are deleted.

docker rmi e455275b486c

Delete container

Remove container from local host. Requires container id/name. Container id/name can be found running docker ps.

docker rm silly_gagarin

Data persistence using Docker volumes

Data from virtual file system of container is lost when the container is restarted or removed. Docker volumes is used where changes needs to be persisted between restarts. Using Docker volumes, a folder in the physical file system is mounted in the virtual file system of the container. The data is automatically replicated between host file system and virtual file system. Thus, data is persisted even when the container is turned down.

Docker Volume types

Host volumes

Host volumes allows you to determine where on the host file system the data is replicated.

docker run -v /home/mount/data:/var/lib/mysql/data

This is in the form host-file-system-path:virtual-file-system-path.

Anonymous Volumes

Only path in the virtual file system is provided. Docker automatically maintains where in the host file system the replica should reside. For each container, a folder is generated in the host file system path /var/lib/docker/volumes/{hash}/_data.

docker run -v /var/lib/mysql/data

Named Volumes

This is similar to anonymous volumes, but you can specify the name of the folder on the host file system. This type of volume can be referenced simply using the name, so you don't have to know the path.

docker run -v name:/var/lib/mysql/data

Volumes in docker-compose

version: '3'
services:
    mongodb:
        image: mongo
        ports:
            - 27017:27017
        environment:
            - MONGO_INITDB_ROOT_USERNAME=admin
            - MONGO_INITDB_ROOT_PASSWORD=password
        volumes:
            - my-volume:/data/db
    mongo-express:
        image: mongo-express
        ports:
            - 8080:8080
        environment:
            - ME_CONFIG_MONGODB_ADMINUSERNAME=admin
            - ME_CONFIG_MONGODB_ADMINPASSWORD=password
            - ME_CONFIG_MONGODB_SERVER=mongodb
volumes:
    my-volume:
        driver: local

Note that here we have used named volumes. Also note that we can mount the reference of the same folder on the host file system to more than one container, for example if the containers need to share the same data.

Container Orchestration

Container orchestration contains of multiple docker hosts that can host containers, so that if one fails, the application is still available through the others. Some orchestration solutions can help automatically scale up the number of container instances when users increase and scale down when the demand decreases. They also help in load balancing requests across hosts. Some container orchestration solutions are docker swarm, kubernetes and mesos.

Happy Hacking!