Docker, Linux and networking: the refresher

This lesson is a refresher, not a tutorial. If every section feels familiar, skip ahead. If several do not, a short read here saves a lot of confusion later.

#Containers in five lines

An image is a packaged filesystem plus start-up settings. A container is a running instance of an image.
Containers share the host's kernel, so they start quickly and weigh much less than virtual machines.
A container's filesystem disappears when it is removed, unless you mount a volume.
Ports must be published to be reachable from outside.
GPUs are passed through with the vendor's container toolkit (for NVIDIA, the container toolkit and --gpus).

#The commands you will see

bash

docker pull vllm/vllm-openai:latest                  # download an image
docker run --rm -it ubuntu bash                      # run a container interactively, delete it on exit
docker run -d --name api -p 8000:8000 IMAGE          # run in the background, publish port 8000
docker run --gpus all -v /models:/models IMAGE       # give it GPUs and mount a folder
docker ps                                            # list running containers
docker logs -f api                                   # follow a container's output
docker exec -it api bash                             # open a shell inside it
docker stop api                                      # stop it
docker build -t myserver:1 .                         # build an image from a Dockerfile

#A Dockerfile

dockerfile

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["python", "server.py"]

Each line is a layer; layers are cached, so put what changes least (dependencies) first and what changes most (your code) last. Keep model weights out of the image: they are many gigabytes and change independently (packaging).

#Linux you will meet

Need	Command or idea
See processes	`ps aux`, `top`, `htop`
Memory and disk	`free -h`, `df -h`, `du -sh *`
GPU state	`nvidia-smi`
Read logs	`tail -f file`, `journalctl -u service`
Environment variables	`export NAME=value`, `printenv`
Permissions	`ls -l`, `chmod`, `chown`
Limits	`ulimit -a` (open files and processes)
Signals	`kill PID` sends a polite stop (SIGTERM); `kill -9` forces it
Exit codes	`echo $?` shows the last command's code: 0 is success

Services should handle SIGTERM by finishing in-flight requests and exiting. That is what makes rolling updates safe (reliability).

#Networking vocabulary

Term	Meaning
IP and port	The address of a machine and the door on it
DNS	Names (`api.example.com`) turned into addresses
TCP	The reliable connection protocols like HTTP run on
HTTP	Requests (method, path, headers, body) and responses (status, headers, body)
TLS	Encryption for HTTP (HTTPS); terminated at a load balancer or the service
Load balancer	Spreads requests across backends and checks their health
Reverse proxy or gateway	Sits in front of services: routing, auth, limits (gateways)
Health check	A URL or probe that says whether a replica should receive traffic
Timeout	How long a caller waits before giving up

Two LLM-specific points:

Streaming uses long-lived responses (server-sent events). Proxies and load balancers must not buffer them, and their idle timeouts must exceed the longest generation (networking and streaming).
Requests last seconds, not milliseconds. Default timeouts of 30 or 60 seconds, tuned for web pages, can cut off long answers.

#Exercise

Which command runs an image in the background, publishes port 8000 and gives it all GPUs?
Why is COPY . . placed after pip install in the Dockerfile?
A streamed answer appears all at once after a pause. Name two places to look.

Answers: 1. docker run -d --gpus all -p 8000:8000 IMAGE. 2. Dependencies change rarely, so their layer is cached across builds; your code changes often, so putting it last keeps rebuilds quick. 3. A proxy or load balancer buffering the response, and a client or CDN setting that waits for the full body.

Next: the anatomy of a production LLM service.