Docker Volumes and Docker networking

1. Docker Volume

• A Docker volume is a way to store data outside the container filesystem, enabling data persistence across container restarts and deployments.

• Volumes are managed by Docker and can be easily created, used, and removed.

• Volumes are the preferred mechanism for persisting data generated by and used by Docker containers.

Volumes have several advantages

• Volumes are easier to back up or migrate than bind mounts.

• You can manage volumes using Docker CLI commands or the Docker API.

• Volumes work on both Linux and Windows containers.

• Volumes can be more safely shared among multiple containers.

• Volume drivers let you store volumes on remote hosts or cloud providers, encrypt the contents of volumes, or add other functionality.

• New volumes can have their content pre-populated by a container.

• Volumes on Docker Desktop have much higher performance than bind mounts from Mac and Windows hosts.

In addition, volumes are often a better choice than persisting data in a container's writable layer, because a volume doesn't increase the size of the containers using it, and the volume's contents exist outside the lifecycle of a given container.

---

2. Types of Docker Volumes

1. Named Volumes

Named volumes are created and managed by Docker, providing a simple way to persist data. They are not tied to the lifecycle of a specific container and can be shared among multiple containers.

# Create a named volume
linux$ docker volume create my_named_volume

# Run a container with the named volume
linux$ docker run -d -v my_named_volume:/data my_image:latest

In this example, the volume my_named_volume is mounted to the /data directory inside the container.

2. Anonymous Volumes

Anonymous volumes are similar to named volumes but do not have a specific name. They are often used for temporary data storage that does not need to be referenced by name.

# Run a container with an anonymous volume
linux$ docker run -d -v /data my_image

In this case, Docker automatically creates an anonymous volume and mounts it to the /data directory inside the container.

3. Host Volumes (Bind Mounts)

Host volumes, or bind mounts, map a directory on the host machine to a directory in the container. Changes made to the directory on the host are reflected in the container and vice versa.

# Run a container with a bind mount
docker run -d -v /path/on/host:/data my_image

This command mounts the /path/on/host directory from the host machine to the /data directory inside the container.

---

Create and manage volumes

Create a volume,

linux$ docker volume create my-vol

List volumes,

linux$ docker volume ls
DRIVER    VOLUME NAME
local     2f7e0f29a7e52e49a954a36343df03b0c042a3c165a21e65228842df72a68c7b
local     7d823dcb15c71c51759d817a440eeee9016e631d0c4c8080a3f76210fedd6227
local     198b06e91329fe18f616dcaf66949b3a0aaeff4700d075514b976b0b515f2fdc
local     30371c4d0976cbae02e2827388d0315b2611216b5613800fe9214cae0af705b5
local     ab62d0964b4aa8bc17d01565b9b3c97798a6beddbc09f584db864e16877c2508
local     cb728006c787b24b3d4740857e6d7cc3381bcbf9decaf63f2bf60a045cfe7e2c
local     mysql
local     mysql-data
local     two-tier-flask-app_mysql-data

Inspect a volume,

linux$ docker volume inspect mysql-data
[
    {
        "CreatedAt": "2024-07-21T12:07:29Z",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/mysql-data/_data",
        "Name": "mysql-data",
        "Options": null,
        "Scope": "local"
    }
]

Remove a volume,

linux$ docker volume rm volume_name

---

3. Docker Network

• Container networking refers to the ability for containers to connect to and communicate with each other, or to non-Docker workloads.

• Containers have networking enabled by default, and they can make outgoing connections.

• A container has no information about what kind of network it's attached to, or whether their peers are also Docker workloads or not.

• A container only sees a network interface with an IP address, a gateway, a routing table, DNS services, and other networking details. That is, unless the container uses the none network driver.

User-defined networks

You can create custom, user-defined networks, and connect multiple containers to the same network. Once connected to a user-defined network, containers can communicate with each other using container IP addresses or container names.

The following example creates a network using the bridge network driver and running a container in the created network,

To see the available networks,

linux$ docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
7411b696ab81   bridge    bridge    local
777de33b98ac   host      host      local
21f28e2c8913   none      null      local
6aa463fcc49d   twotier   bridge    local

# Creating custom network
linux$ docker network create -d bridge my-net
linux$ docker run --network=my-net -itd --name=container3 busybox

---

4. Types of Network drivers?

• bridge: The default network driver. If you don't specify a driver, this is the type of network you are creating. Bridge networks are commonly used when your application runs in a container that needs to communicate with other containers on the same host.

• host: Remove network isolation between the container and the Docker host, and use the host's networking directly.

• overlay: Overlay networks connect multiple Docker daemons together and enable Swarm services and containers to communicate across nodes. This strategy removes the need to do OS-level routing.

• ipvlan: IPvlan networks give users total control over both IPv4 and IPv6 addressing. The VLAN driver builds on top of that in giving operators complete control of layer 2 VLAN tagging and even IPvlan L3 routing for users interested in underlay network integration.

• macvlan: Macvlan networks allow you to assign a MAC address to a container, making it appear as a physical device on your network. The Docker daemon routes traffic to containers by their MAC addresses. Using the macvlan driver is sometimes the best choice when dealing with legacy applications that expect to be directly connected to the physical network, rather than routed through the Docker host's network stack.

• none: Completely isolate a container from the host and other containers. none is not available for Swarm services.

---

5. Task 1 - Docker compose

Create a multi-container docker-compose file that will bring up and bring down containers in a single shot (e.g., create application and database containers).

We are going to take a reference of our previous project two-tier-application and going to take a look on task.

• Use the docker-compose up command with the -d flag to start a multi-container application in detached mode.

  -> two-tier-application in this we have created two container's of frontend and database, Here is docker-compose.yml file of same project.

version: '3.9'
services:
  mysql: # Container 1 of database
    container_name: mysql
    image: 'mysql:5.7' # Base image
    ports:
      - '3306:3306'
    environment:
      MYSQL_ROOT_PASSWORD: root
      MYSQL_DATABASE: devops
      MYSQL_USER: admin
      MYSQL_PASSWORD: admin
    volumes:
      - 'mysql-data:/var/lib/mysql' # Binding of persistent volume
    networks:
      - twotier # user-defined bridge network
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      retries: 5
      start_period: 30s
      timeout: 5s

  flask-app: # Container 2 of frontend
    build:
      context: . # Building image using Dockerfile
    ports:
      - '5001:5000'
    environment:
      MYSQL_HOST: mysql # mysql is an database container name
      MYSQL_USER: admin
      MYSQL_PASSWORD: admin
      MYSQL_DB: devops
    depends_on: # Run only of mysql container is working fine
      mysql:
        condition: service_healthy
    networks:
      - twotier

volumes:
  mysql-data: # Define volume and it will create volume

networks:
  twotier: # Custome network name
    driver: bridge # Custome network driver type

Now, using docker compose up command to run container's using docker-compose.

• Use the docker-compose scale command to increase or decrease the number of replicas for a specific service. You can also add replicas in the deployment file for auto-scaling.

two-tier-flask-app$ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
two-tier-flask-app$ docker compose up --scale flask-app=3 -d
[+] Running 2/5
 ⠇ Network two-tier-flask-app_twotier        Created      11.8s
 ✔ Container mysql                           Healthy      11.1s
 ⠦ Container two-tier-flask-app-flask-app-3  Created      11.6s
 ⠦ Container two-tier-flask-app-flask-app-1  Starting     11.6s
 ✔ Container two-tier-flask-app-flask-app-2  Started      11.5s

two-tier-flask-app$ docker compose down
[+] Running 5/5
 ✔ Container two-tier-flask-app-flask-app-3  Removed    0.0s
 ✔ Container two-tier-flask-app-flask-app-2  Removed    12.2s
 ✔ Container two-tier-flask-app-flask-app-1  Removed    0.0s
 ✔ Container mysql                           Removed    12.2s
 ✔ Network two-tier-flask-app_twotier        Removed    0.1s

#Setting up replica set in docker-compose.yml file
 deploy:
      replicas: 3

• Use the docker-compose ps command to view the status of all containers, and docker-compose logs to view the logs of a specific service.

two-tier-flask-app$ docker compose ps
NAME                             IMAGE                          COMMAND                  SERVICE     CREATED         STATUS                    PORTS
mysql                            mysql:5.7                      "docker-entrypoint.s…"   mysql       2 minutes ago   Up 46 seconds (healthy)   0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp
two-tier-flask-app-flask-app-1   two-tier-flask-app-flask-app   "python app.py"          flask-app   2 minutes ago   Up 35 seconds             0.0.0.0:5001->5000/tcp, :::5001->5000/tcp

two-tier-flask-app$ docker compose logs flask-app
flask-app-1  |  * Running on all addresses (0.0.0.0)
flask-app-1  |  * Running on http://127.0.0.1:5000
flask-app-1  |  * Running on http://172.21.0.3:5000
flask-app-1  | Press CTRL+C to quit
flask-app-1  |  * Restarting with stat
flask-app-1  |  * Debugger is active!
flask-app-1  |  * Debugger PIN: 340-996-161

• Use the docker-compose down command to stop and remove all containers, networks, and volumes associated with the application.

two-tier-flask-app$ docker compose down
[+] Running 3/3
 ✔ Container two-tier-flask-app-flask-app-1  Removed   12.2s
 ✔ Container mysql                           Removed   12.2s
 ✔ Network two-tier-flask-app_twotier        Removed   0.1s

---

6. Task 2 - Docker volume

• Learn how to use Docker Volumes and Named Volumes to share files and directories between multiple containers.

• Create two or more containers that read and write data to the same volume using the docker run --mount command.

• Verify that the data is the same in all containers by using the docker exec command to run commands inside each container.

• Use the docker volume ls command to list all volumes and the docker volume rm command to remove the volume when you're done.

Create a Named Volume,

two-tier-flask-app$ docker volume create shared_data
shared_data

two-tier-flask-app$ docker volume ls
DRIVER    VOLUME NAME
local     mysql-data
local     shared_data
local     two-tier-flask-app_mysql-data

Run Two Containers Sharing the Same Volume,

docker run -d --name container1 --mount source=shared_data,target=/shared busybox
docker run -d --name container2 --mount source=shared_data,target=/shared busybox

Verify Containers are Running,

docker ps

Write Data to the Volume from container1,

docker exec container1 sh -c 'echo "Hello from container1" > /shared/hello.txt'

Read Data from the Volume in container2,

docker exec container2 cat /shared/hello.txt

Verify Data Consistency,

Hello from container1

List All Volumes,

docker volume ls

Remove the Volume,

docker volume rm shared_data

---

Connect with Me

LinkedIn | GitHub

Follow me for more amazing content :)