Understanding Docker Data Persistence

When you create a container from an image, it starts with a fresh filesystem containing only the image layers. Any data written during the container's lifetime disappears when the container is removed, unless proper measures are taken.

This presents a fundamental challenge: applications need persistent storage, but containers are designed to be ephemeral. Docker solves this through several storage mechanisms that decouple data from the container's lifecycle.

Filesystem Isolation in Docker

Eventhough Docker containers share the host kernel, each container maintains its own isolated filesystem. This isolation is essential for consistent environments across different hosts. However, this separation means data written inside a container is trapped within that container's layer—unless explicitly persisted through volumes or mounts.

Docker provides three primary approaches for persistent storage:

Bind mounts: Map host directories directly into the container Named volumes: Docker-managed storage regions tmpfs mounts: Memory-based temporary storage

Bind Mounts

Bind mounts allow you to attach a specific host filesystem path to a path within the container. This is useful for development workflows where you need the host's files immediately available inside the container.

The syntax uses the -v flag combined with the run command:

docker run -it -v <host-path>:<container-path>[:<options>] <image>

The optional flags include ro for read-only access or other mount-specific parameters.

Let's demonstrate this with a practical example using the Python image:

# First, create a directory on the host for persistence
~$ pwd
/home/developer
~$ mkdir appdata
~$ ls
apt-sources.sh  learn.code-workspace  packages.microsoft.gpg  appdata  work

# Start a container with the host directory mounted
~$ docker run -it -v /home/developer/appdata:/container/data python:3.6

# Python interactive shell
Python 3.6.15 (default, Dec 21 2021, 12:03:22) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.isdir(r'/container/data')
True

# Create a file in the mounted directory
>>> with open(r'/container/data/sample.txt', 'w', encoding='utf-8') as f:
...     f.write('persisted data\n')
... 
15
>>> exit()

After exiting the container, verify that the data exists on the host:

~$ ls /home/developer/appdata
sample.txt
~$ cat /home/developer/appdata/sample.txt
persisted data

The file created inside the container is now available on the host filesystem. This approach enables easy data sharing between containers by mounting the same host directory to different containers. Additionally, it solves the problem of duplicate data that would otherwise exist across multiple container instances.

One consideration: bind mounts depend entirely on the host filesystem structure. If the host directory is deleted or becomes unavailable, the mount fails and the container may encounter errors.

Docker Volumes

Docker-managed volumes provide an alternative to bind mounts, offering more control through the Docker CLI. Volumes are stored in Docker's managed storage area rather than the host's filesystem hierarchy.

Key characteristics:

Managed entirely by Docker Persist independently of container lifecycle Ideal for sharing data between containers Can be pre-populated before container start

Creating and Inspecting Volumes

The docker volume create command creates a new named volume:

~$ docker volume create my-data-volume
my-data-volume
~$ docker volume inspect my-data-volume
[
    {
        "CreatedAt": "2023-04-05T15:35:45+08:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/my-data-volume/_data",
        "Name": "my-data-volume",
        "Options": null,
        "Scope": "local"
    }
]

If no name is specified, Docker generates a unique identifier automatically. Using descriptive names helps with volume management in production environments.

Using Volumes with Containers

Mount a volume to a container using -v with the volume name:

docker run -it -v <volume-name>:<container-path>[:<options>] <image>

Create a container that writes data to the volume:

~$ docker run -it --name "data-writer" -v my-data-volume:/mnt/storage python:3.6
Python 3.6.15 (default, Dec 21 2021, 12:03:22) 
[GCC 10.2.1 20210110] on linux
>>> with open(r'/mnt/storage/output.txt', 'w', encoding='utf-8') as f:
...     f.write('shared content')
... 
14
>>> exit()

Now create a second container that reads from the same volume:

~$ docker run -it --name "data-reader" -v my-data-volume:/mnt/readonly:ro python:3.6
Python 3.6.15 (default, Dec 21 2021, 12:03:22) 
[GCC 10.2.1 20210110] on linux
>>> import os
>>> os.path.exists(r'/mnt/readonly/output.txt')
True
>>> exit()

The :ro flag mounts the volume as read-only, preventing modifications from this container. The --name flag assigns a friendly identifier to each container for easier management.

Volume Management Commands

docker volume ls - List all volumes docker volume rm <volume-name> - Remove a specific volume docker volume inspect <volume-name> - Display volume details docker volume prune - Remove unused volumes

tmpfs Mounts

The tmpfs mount type stores data in host memory rather than the filesystem. This provides extremely fast storage at the cost of volatility—the data disappears when the container stops.

tmpfs mounts are particularly useful for:

Temporary files that shouldn't persist Sensitive data that shouldn't be written to disk Caching with automatic cleanup

Note that tmpfs is available only on Linux hosts and cannot be used to share data between containers.

For detailed implementation guidance, refer to the official Docker tmpfs documentation.

The --mount Flag

The --mount flag provides a more explicit and verbose syntax for all mount types, making configuration clearer at the cost of brevity:

docker run -it --mount 'type=[volume|bind|tmpfs], src=<source>, target=<destination>[, volume-opt=<key=value>][, readonly]' <image>

Parameters include:

type: Mount category—volume, bind, or tmpfs src: Source path or volume name (optional for tmpfs) target: Mount point inside the container volume-opt: Driver-specific options readonly: Restricts write access

Use docker inspect <container-id> to verify mount configuration for any running container.

While the -v shorthand suffices for most use cases, --mount becomes valuable when you need multiple options or want explicit control over mount behavier.