DevOps · Sub-chapter 1 · 8 min read

Docker

Sub-chapter 1 of DevOps · Ship the whole machine, not just the code

The DevOps primer gave you Docker in one screen. This is the deep dive. Docker is the single most leveraged tool you'll touch as a forward-deployed engineer, because it answers the oldest complaint in software: "it works on my machine." A container makes "my machine" portable - same filesystem, same dependencies, same entrypoint, whether it runs on your MacBook in Chandigarh, a customer's locked-down laptop, or an ap-south-1 EC2 box.

The instinct to build is this: a container image is a frozen, layered snapshot of a filesystem plus a default command to run inside it. Everything else - runtime, networking, volumes, Compose - is plumbing around that one idea. At Welzin, if a service isn't docker compose up-able from a fresh clone, it isn't done.

Outline

Container vs VM - shared kernel, namespaces and cgroups, not a tiny VM
Images, layers, caching - how a build is a stack of diffs, and why order matters
Multi-stage Dockerfiles - fat builder, thin runtime, the only sane way to ship compiled code
.dockerignore - keep build context (and secrets) out of the image
Tags & registries - never :latest in prod, how push/pull and digests work
docker run essentials - ports, volumes, env, networks, the flags you'll type daily
Docker Compose - multi-service local dev, dependencies, healthchecks
Image size & non-root - distroless/alpine, slim layers, USER discipline
Security basics - no secrets in layers, scan images, drop root
Debugging a live container - exec, logs, inspect, and common mistakes

101 Primer

A container is a process, not a VM

A VM virtualises hardware: it boots its own kernel, its own init system, its own everything. A container shares the host kernel and is really just a Linux process with three tricks applied:

Namespaces isolate what the process can see - its own PID tree, network interfaces, mounts, hostname. PID 1 inside the container is just some PID on the host.
cgroups limit what it can use - CPU, memory, IO.
A union filesystem gives it its own root (/) assembled from image layers.

That's why a container starts in milliseconds and a VM takes 30 seconds. There is no second kernel to boot. The flip side: containers share the host kernel, so isolation is weaker than a VM. For untrusted multi-tenant workloads you want gVisor, Firecracker, or actual VMs. For your code on your infra, containers are correct 95% of the time.

bash

docker run --rm alpine sh -c 'ps aux'   # PID 1 is your command, nothing else

Images, layers, and the build cache

An image is a stack of read-only layers. Each instruction in a Dockerfile that changes the filesystem (COPY, RUN, ADD) creates a new layer - a diff on top of the previous one. When you docker run, Docker adds a thin writable layer on top; that's where runtime changes go, and it's discarded when the container is removed.

The build cache is layer-by-layer: Docker reuses a cached layer if the instruction and everything it depends on is unchanged. This is why instruction order is the single biggest lever on build speed. Put the things that rarely change early, and the things that change every commit (your source) late:

dockerfile

# BAD - every source edit re-runs the (slow) dependency install
COPY . .
RUN npm ci

# GOOD - deps only reinstall when package files change
COPY package*.json ./
RUN npm ci
COPY . .

Inspect the layers and their sizes with:

bash

docker history welzin/api:dev
docker image inspect welzin/api:dev --format '{{.RootFS.Layers}}'

Multi-stage builds: the only way to ship compiled code

You need a fat toolchain to build (compilers, dev headers, node_modules), but none of it to run. Multi-stage builds let you compile in one stage and copy only the artifact into a clean final stage. The build tools never reach the shipped image.

A real Go example - final image is ~10 MB instead of ~800 MB:

dockerfile

# ---- build stage ----
FROM golang:1.23 AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /app/server ./cmd/server

# ---- runtime stage ----
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=build /app/server /server
USER nonroot:nonroot
EXPOSE 8000
ENTRYPOINT ["/server"]

For Node/Python, the same shape applies: install + build in the first stage, copy dist/ and production node_modules (or the Python venv) into a slim final stage. CGO_ENABLED=0 above produces a static binary so it runs on distroless with no libc. Pin base images by tag, and ideally by digest (golang:1.23@sha256:...) for reproducible builds.

`.dockerignore` - context discipline

docker build tars up the entire build context (the directory you point it at) and ships it to the daemon before the build even starts. Without a .dockerignore, that means node_modules, .git, and your .env all get uploaded - slow, and a great way to bake secrets into an image via a careless COPY . ..

text

.git
node_modules
dist
*.log
.env
.env.*
**/__pycache__
.DS_Store

Treat .dockerignore as mandatory in every repo, right next to .gitignore. They are not the same file and they protect against different mistakes.

Tags, registries, and digests

A tag is a mutable, human-friendly name. A digest (sha256:...) is the immutable content address.

bash

docker build -t welzin/api:1.4.2 -t welzin/api:latest .
docker tag welzin/api:1.4.2 123456789.dkr.ecr.ap-south-1.amazonaws.com/welzin/api:1.4.2
docker push 123456789.dkr.ecr.ap-south-1.amazonaws.com/welzin/api:1.4.2

Rules at Welzin:

Never deploy :latest. It's a moving target - you can't tell what's actually running. Deploy an immutable version tag (:1.4.2) or a Git SHA tag (:git-a1b2c3d).
Tag with the commit SHA in CI so any running container traces back to exact source.
Pin by digest for base images in production Dockerfiles.
We use ECR in ap-south-1; authenticate with aws ecr get-login-password | docker login --username AWS --password-stdin <registry>.

`docker run` - the flags you type every day

bash

docker run -d \
  --name api \                       # name it, don't use the random one
  -p 8000:8000 \                     # host:container port
  -e DATABASE_URL=$DATABASE_URL \    # env var (value from your shell)
  --env-file .env \                  # or a whole file
  -v $(pwd)/data:/data \             # bind mount: host dir -> container
  -v pgdata:/var/lib/postgresql/data \  # named volume: docker-managed
  --network welzinnet \              # attach to a user-defined network
  --restart unless-stopped \         # survive daemon restarts
  --memory 512m --cpus 1.5 \         # resource limits
  welzin/api:1.4.2

Two volume types, and the difference matters: a bind mount (./data:/data) maps a host path straight in - great for live-reloading source in dev. A named volume (pgdata:/...) is managed by Docker and is the right choice for database state. Anything not on a volume dies with the container - the writable layer is ephemeral.

On user-defined networks, containers reach each other by name via Docker's embedded DNS (api can curl http://db:5432). The default bridge network does not give you name resolution, so always create a network or use Compose (which makes one for you).

Docker Compose for local multi-service dev

Compose is how you run a realistic stack locally without memorising twelve docker run flags. One file, one command.

yaml

# compose.yaml
services:
  api:
    build: .
    ports: ["8000:8000"]
    env_file: .env
    depends_on:
      db:
        condition: service_healthy   # wait for db to be READY, not just started
    develop:
      watch:                          # live sync source on change
        - path: ./src
          target: /app/src
          action: sync

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_PASSWORD: dev
    volumes: ["pgdata:/var/lib/postgresql/data"]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      retries: 5

volumes:
  pgdata:

bash

docker compose up -d          # start everything in the background
docker compose logs -f api    # tail one service
docker compose exec db psql -U postgres   # shell into a running service
docker compose down -v        # stop AND delete volumes (full reset)

The depends_on + healthcheck combo fixes the classic flake where the API boots before Postgres is accepting connections. condition: service_started only waits for the container to exist; you almost always want service_healthy.

Image size and running as non-root

Smaller images pull faster, expose less, and start quicker. Discipline:

Pick a small base. alpine (musl libc, ~5 MB), *-slim Debian variants, or distroless (no shell, no package manager - just your app and its runtime). Distroless is our default for production binaries.
Combine RUN steps and clean up in the same layer - deleting a file in a later layer doesn't shrink the earlier one:
dockerfile
```
RUN apt-get update && apt-get install -y --no-install-recommends curl \
    && rm -rf /var/lib/apt/lists/*
```
Don't run as root. By default PID 1 in a container is root, and a container breakout then lands as root on the host. Create and switch to an unprivileged user:
dockerfile
```
RUN useradd --uid 10001 --no-create-home appuser
USER appuser
```
Distroless images ship a nonroot user for exactly this.

Security basics you don't get to skip

No secrets in layers. A secret COPYd in and RUN rm'd later still lives in the earlier layer - anyone with the image can docker history/extract it. Pass secrets at runtime via env/secrets manager, or use BuildKit secret mounts:
dockerfile
```
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm ci
```
Scan images. docker scout cves welzin/api:1.4.2 or Trivy (trivy image welzin/api:1.4.2) in CI. Fail the build on HIGH/CRITICAL with a known fix.
Drop capabilities and root. --user, --cap-drop ALL, --read-only, --security-opt no-new-privileges. Least privilege at runtime.
Pin base image digests so a re-pull can't silently swap in a compromised :latest.

Debugging a running container

bash

docker ps                          # what's running, ports, status
docker logs -f --tail 100 api      # stdout/stderr (log to stdout, not files!)
docker exec -it api sh             # shell in - bash may not exist on alpine
docker inspect api                 # full JSON: mounts, env, network, IP
docker inspect -f '{{.State.ExitCode}}' api   # why did it die?
docker stats api                   # live CPU/mem/IO
docker top api                     # processes inside
docker cp api:/app/config.json .   # pull a file out for inspection

If a container exits instantly, docker logs first - most of the time it printed the error and died. If the image is distroless and has no shell, debug it with an ephemeral debugging container sharing its namespaces:

bash

docker run --rm -it --pid container:api --network container:api nicolaka/netshoot

Common mistakes we see from new hires:

Writing logs/state to a file inside the container, then losing it on restart. Log to stdout; persist state on a volume.
COPY . . before RUN npm ci, nuking the layer cache on every edit.
Baking .env into the image because there was no .dockerignore.
Running as root "to make the permission error go away."
Using :latest and then being unable to say what's deployed.
docker compose down (keeps volumes) vs down -v (wipes them) - know which you typed before you panic about lost data.

Hands-on Checkpoints

Write a multi-stage Dockerfile for a Go or Node service; confirm the final image is < 50 MB with docker images.
Reorder a Dockerfile so a source-only edit rebuilds in seconds (cache hit on deps); prove it with two timed builds.
Add a .dockerignore; rebuild and confirm .git and .env are not inside via docker run --rm img ls -la.
Run the app non-root: add a USER, rebuild, and verify with docker exec api id.
Write a compose.yaml with app + Postgres using a healthcheck and depends_on: service_healthy; bring it up clean.
Scan your image with docker scout cves or trivy image; fix or document one CVE.
Break a container on purpose (bad env var), then diagnose it using only logs, inspect, and exec.

Docker

Docker

Outline

101 Primer

A container is a process, not a VM

Images, layers, and the build cache

Multi-stage builds: the only way to ship compiled code

`.dockerignore` - context discipline

Tags, registries, and digests

`docker run` - the flags you type every day

Docker Compose for local multi-service dev

Image size and running as non-root

Security basics you don't get to skip

Debugging a running container

Hands-on Checkpoints

Further reading

Knowledge check

Docker#

Outline#

101 Primer#

A container is a process, not a VM#

Images, layers, and the build cache#

Multi-stage builds: the only way to ship compiled code#

.dockerignore - context discipline#

Tags, registries, and digests#

docker run - the flags you type every day#

Docker Compose for local multi-service dev#

Image size and running as non-root#

Security basics you don't get to skip#

Debugging a running container#

Hands-on Checkpoints#

Further reading#

Knowledge check

Docker

Outline

101 Primer

A container is a process, not a VM

Images, layers, and the build cache

Multi-stage builds: the only way to ship compiled code

`.dockerignore` - context discipline

Tags, registries, and digests

`docker run` - the flags you type every day

Docker Compose for local multi-service dev

Image size and running as non-root

Security basics you don't get to skip

Debugging a running container

Hands-on Checkpoints

Further reading