Core Concepts

What is Docker?

Traditional virtualization relies on virtual machines, which solve cross-platform compatibility issues but are often too resource-intensive, slow to start, and demanding in terms of system resources. This limited their widespread adoption in the internet industry. Docker's container technology, while not fully cross-platform, offers rapid startup times and minimal resource usage, leading to its immediate popularity. Container technologies like Docker are expected to become the mainstream direction for future development.

Docker is more than just a virtualization or container technology—it's set to fundamentally transform the technical landscape:

• Revolution in Operations: Docker's quick startup and minimal resource footprint allow containers to be started and stopped on demand, enabling automated and intelligent operations to become the standard approach.
• Evolution in Design Patterns: The low cost of spinning up new container instances encourages a shift toward microservices architecture from the beginning of a project.

For example, a traditional web site might include user authentication, page rendering, and search functionality. Without containers, these features are typically integrated into a single system unless there's significant traffic. With container technology, these can be designed as separate services from the start, avoiding the need for system restructuring when traffic increases.

Installation and Configuration

Due to restrictions on Docker services in certain regions, alternative sources like Alibaba Cloud can be used for installation.

For Mac M1 chip users, check your system architecture with:

uname -m

Based on the output, select the appropriate version (arm64 for M1 chips).

Windows Environment Issues:

If you encounter the error: "error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.39/containers/json: open //./pipe/docker_engine: The system cannot find the file specified..."

Try this solution:

cd "C:\Program Files\Docker\Docker"
./DockerCli.exe -SwitchDaemon

Image Acceleration

Configure image acceleration from Alibaba Cloud's container registry service. Follow their documentation to set up the configuration file with the appropriate mirror URLs.

Docker Compose

Introduction to Docker Compose

Applications using microservices architecture typically consist of multiple services, each with potentially multiple instances. Manually managing these services would be inefficient and maintenance-heavy. Docker Compose simplifies this process by allowing you to define and run multi-container Docker applications easily.

Compose is a tool for defining and running multi-container Docker applications. With Compose, you configure your application's services in a YAML file, then create and start all services with a single command.

Using Docker Compose

Compose usage is straightforward: create a docker-compose.yml file to describe your container configurations, then use docker-compose commands to manage them.

Docker Compose Structure

Docker Compose organizes containers into three layers:

• Project: All files in the directory where docker-compose.yml is located form a project (named after the directory by default).
• Service: A project can contain multiple services. Each service defines the image, parameters, and dependencies for running containers.
• Container: A service can include multiple container instances.

Containers within the same Docker Compose setup can communicate using service names as hostnames. If a service is scaled to multiple containers, requests to the service name will be load-balenced across all containers using technologies like LVS.

Docker Architecture

Architecture Design

• Docker Daemon: A background process running on the host machine that communicates with the Docker client.
• Docker Client: The user interface that accepts commands and configuration identifiers, communicating with the Docker daemon.
• Docker Images: Read-only templates containing instructions for creating containers. Similar to system installation discs.
• Containers: Runnable instances of images. The relationship between images and containers is similar to classes and objects in object-oriented programming.

Containers are essentially processes, but unlike processes running directly on the host, container processes run in their own isolated namespaces. This allows containers to have their own root filesystem, network configuration, process space, and user ID space.

This isolation makes applications running in containers more secure than those running directly on the host. However, this isolation often leads beginners to confuse containers with virtual machines.

• Registry: A centralized service for storing and distributing images. After building a Docker image, you can run it on the current host, but to run it on other machines, you need a registry to avoid manual copying.

A Docker Registry can contain multiple repositories, each with multiple image tags, where each tag corresponds to a Docker image. This is similar to Maven repositories, where the registry is like the Maven repository, a Docker repository is like a JAR path, and image tags are like version numbers.

Docker Registries can be public or private. The most common public registry is Docker Hub, which hosts numerous high-quality images available for download and use.

Dockerfile

A Dockerfile is a text file containing instructions that describe the details of building an image. Common Dockerfile instructions include:

Command	Purpose
FROM	Base image file
RUN	Execute commands during the image build phase
ADD	Add files, copy from src to container's dest. Src can be a relative path, URL, or compressed file
COPY	Copy files, similar to ADD but doesn't support URLs or compressed files
CMD	Commands to execute after container startup
EXPOSE	Declare ports that the container will expose at runtime
WORKDIR	Specify the working directory in the container
ENV	Set environment variables
ENTRYPOINT	Container entry point. Similar to CMD but with different behavior
USER	Set the user or UID for running the image. Subsequent RUN, CMD, and ENTRYPOINT commands will use this user
VOLUME	Specify mount points for persistent storage

Note: RUN commands execute during the image build phase, and their results are packaged into the image. CMD commands execute after container startup. A Dockerfile can contain multiple RUN commands but only one CMD command.

Example Dockerfile:

FROM registry.example.com/java:8
ADD application.jar app.jar
CMD java -jar app.jar

Image Repositories

After creating microservice images, you typically need to publish them to an image repository for others to use. You can choose to build your own repository or use official Docker repositories. Here's how to set up a remote repository on Alibaba Cloud:

1. Modify the local image tag to match the remote repository format
2. Push the image to the remote repository
3. Test by deleting the local image and pulling it from the remote repository

Docker Client

Image Management

Searching Images

Use the docker search command to find images in Docker Hub:

docker search java

The results include:

• NAME: Image repository name
• DESCRIPTION: Repository description
• STARS: Number of stars, indicating popularity
• OFFICIAL: Whether it's an official repository
• AUTOMATED: Whether it's an automated build

Downloading Images

Use docker pull to download images from a Docker Registry:

docker pull java:8

Listing Images

Use docker images to list downloaded images:

docker images

The output shows:

• REPOSITORY: Image repository name
• TAG: Image tag (default: latest)
• IMAGE ID: Unique image identifier
• CREATED: Image creation time
• SIZE: Image size

Removing Images

Use docker rmi to remove an image (add -f for force removal):

docker rmi java

Building Images

Format: docker build -t image_name:tag Dockerfile_location

Example:

docker build -t myapp:latest .

Container Management

Starting Containers

Use docker run to create and start a container. Common options include:

• -d: Run in background
• -P: Random port mapping
• -p: Specify port mapping (hostPort:containerPort)
• --name: Assign a name to the container
• -e: Set environment variables
• -m: Set memory limit
• --restart=always: Automatically restart on Docker daemon restart
• --net: Specify network mode (bridge, host, container:NAME-or-ID, none)

Example:

docker run -d -p 91:80 nginx

This starts an Nginx container with port 91 on the host mapped to port 80 in the container.

For stopped containers, use docker start to restart them:

docker start container_id

Listing Containers

Use docker ps to list running containers. Add -a to list all containers:

docker ps -a

The output includes:

• CONTAINER_ID: Container ID
• IMAGE: Image name
• COMMAND: Command run when starting the container
• CREATED: Container creation time
• STATUS: Container status (Up or Exited)
• PORTS: Exposed ports
• NAMES: Container name

Stopping Containers

Use docker stop to stop a container:

docker stop container_id

For force stopping, use docker kill:

docker kill container_id

Removing Containers

Use docker rm to remove a stopped container (add -f for force removal):

docker rm container_id

To remove all containers:

docker rm -f $(docker ps -a -q)

Accessing Containers

Use docker exec to enter a running container:

docker exec -it container_id /bin/bash

Some containers might use sh instead of /bin/bash. Once inside, you can execute commands in the container's shell. Exit with the exit command.

Viewing Container Logs

docker container logs container_id

Container Resource Monitoring

docker stats

This command returns real-time resource usage, refreshing every second until you press Ctrl+C. The output includes:

• CONTAINER: Short container ID
• CPU %: CPU usage
• MEM USAGE / LIMIT: Current memory usage and maximum limit
• MEM %: Memory usage percentage
• NET I/O: Network I/O data
• BLOCK I/O: Disk I/O data
• PIDS: Process IDs

To limit container memory, use the -m parameter:

docker run -m 500M redis

Volume Mounting

When creating containers, you can map host directories to container directories. This allows you to affect the container by modifying files on the host, and vice versa. This bidirectional binding also provides backup functionality. When a container is deleted, the host content remains unaffected. If multiple containers mount the same directory, deleting one container doesn't affect the others.

Add the -v parameter when creating a container, with the format host_directory:container_directory:

docker run -v /host/path:/container/path image_name

For multiple directories:

docker run -v /host/path1:/container/path1 -v /host/path2:/container/path2 image_name

Example:

docker run -d --name myapp --restart=always \
        -v ${PWD}/logs/info.log:/logs/myapp/info.log \
        -v ${PWD}/logs/error.log:/logs/myapp/error.log \
        myapp:latest

You can also mount entire directories:

docker run -d --name myapp --restart=always \
        -v ${PWD}/logs/:/logs/myapp \
        myapp:latest

To view mount relationships, use docker inspect and look for the Mounts section in the JSON output.

CI/CD Automation with Docker

Overall Architecture

CI/CD pipelines with Docker typically involve several stages that automate the process from code commmit to deployment.

Workflow

The core steps in a Docker-based CI/CD pipeline include:

1. Build: Compile the code, create Docker images, and push them to a container registry
2. Test: Execute automated test scripts
3. Deploy: Pull images from the registry and start containers

Example .gitlab-ci.yml:

stages:
- build
- test
- deploy

build:
  stage: build
  tags:
  - build
  only:
  - develop
  - hotfix
  script:
  - echo "Compiling the code..."
  - echo "Building Docker image..."
  - docker build -t myapp:$CI_COMMIT_SHA .
  - echo "Pushing to registry..."
  - docker push myapp:$CI_COMMIT_SHA
  - echo "Build complete."

test:
  stage: test
  tags:
  - test
  only:
  - develop
  - hotfix
  script:
  - echo "Running tests..."
  - docker run myapp:$CI_COMMIT_SHA npm test
  - echo "Tests complete."

deploy:
  stage: deploy
  tags:
  - deploy
  only:
  - develop
  - hotfix
  script:
  - echo "Deploying application..."
  - docker pull myapp:$CI_COMMIT_SHA
  - docker stop myapp-container || true
  - docker rm myapp-container || true
  - docker run -d --name myapp-container -p 8080:8080 myapp:$CI_COMMIT_SHA
  - echo "Application successfully deployed."

Example build script:

#!/bin/bash

function push_to_registry() {
  docker tag "$1":latest registry.example.com/myorg/"$1"
  docker push registry.example.com/myorg/"$1"
  echo "Pushed to registry!"
}

image_name=myapp-$1
project_location=/workspace

# Check input parameters
if [ "$1" == '' ]; then
  echo "Parameter <image name> is missing..."
  exit
fi

cd "$project_location"/myapp-"$1" || exit
echo "$PWD"
echo "Starting to build image..."

# Update code
git fetch
git pull

# Build project
mvn clean package -DskipTests

# Build Docker image
docker build -t "$image_name":latest .

echo "Build successful."

push_to_registry "$image_name"

# Clean up unused images
docker rmi $(docker images | grep none | awk '{print $3}')

The build process consists of these main steps:

1. Git update
2. Maven build
3. Docker image build
4. Push to registry
5. Clean up obsolete images

After completing these steps, the latest image is available in the registry, ready for deployment in the deploy stage.