Architecture#

Kubernetes has two main parts: the Control Plane (brain) and Worker Nodes (muscle).

k8s architecture

Control Plane Components#

Component Role
kube-apiserver Single entry point for all operations. Validates requests, persists to etcd, serves the REST API
etcd Distributed key-value store. The source of truth — all cluster state lives here
kube-scheduler Watches for unscheduled Pods, assigns them to a Node based on resources and constraints
kube-controller-manager Runs control loops: Node controller, ReplicaSet controller, Endpoint controller, etc.
cloud-controller-manager Cloud-specific logic (load balancers, storage, node lifecycle) — only in cloud deployments

Worker Node Components#

Component Role
kubelet Agent on every node. Ensures containers described in PodSpecs are running and healthy
kube-proxy Maintains network rules (iptables/ipvs) to implement Service abstraction
container runtime Actually runs containers. k8s talks to it via CRI (e.g. containerd, CRI-O)

How They Interact (a request walkthrough)#

request walkthrough

Key Concepts#

Workload Resources#

workload resources

Resource Use Case
Pod Runs one or more tightly-coupled containers
ReplicaSet Keeps N replicas of a Pod running (usually managed by Deployment)
Deployment Stateless apps — rolling updates, rollback, scaling
StatefulSet Stateful apps (databases) — stable network identity, ordered operations
DaemonSet Run one Pod per node — log collectors, monitoring agents
Job Run to completion — batch tasks
CronJob Scheduled Jobs

Service & Networking#

A Service gives a stable IP/DNS to a set of Pods (selected by labels). Pods come and go; Service stays.

Service Type Scope Use Case
ClusterIP Internal only Default; Pod-to-Pod communication
NodePort External via NodeIP:Port Simple external access, dev/testing
LoadBalancer External via cloud LB Production external traffic (cloud)
ExternalName DNS alias Point to external service by DNS

Ingress = HTTP/HTTPS routing rules (path-based, host-based) sitting in front of Services. Requires an Ingress Controller (e.g. nginx, traefik).

ingress traffic path

Gateway API (GA since k8s 1.31, gateway.networking.k8s.io/v1) is the successor to Ingress — more expressive, supports TCP/UDP/gRPC, role-based split between infra and app teams. Ingress is still supported but new deployments should prefer Gateway API.

Config & Storage#

Resource Purpose
ConfigMap Non-sensitive config — env vars, config files
Secret Sensitive data — passwords, tokens (base64 encoded, not encrypted by default)
PersistentVolume (PV) A piece of storage provisioned in the cluster
PersistentVolumeClaim (PVC) A request for storage by a Pod — binds to a matching PV
StorageClass Defines how to dynamically provision PVs

Metadata & Organization#

  • Namespace — virtual cluster within a cluster, isolates resources by team/env
  • Label — key-value pairs on resources (app: nginx, env: prod)
  • Selector — query labels to target resources (app: nginx)
  • Annotation — non-identifying metadata (deployment tools, git SHA, etc.)

Security#

Resource Purpose
ServiceAccount Identity for a Pod to authenticate with the API server
Role / ClusterRole Defines permissions (verbs on resources)
RoleBinding / ClusterRoleBinding Grants a Role to a ServiceAccount/User

Key Design & Philosophy#

1. Declarative, Not Imperative#

You describe what you want (desired state), not how to get there.

# declarative: "I want 3 nginx pods"
spec:
  replicas: 3

# vs imperative: "start 3 nginx pods right now"
kubectl run nginx-1 ...
kubectl run nginx-2 ...
kubectl run nginx-3 ...

k8s continuously works to make actual state = desired state.

2. Control Loop (Reconciliation)#

Every controller runs an infinite loop:

control loop

This is why k8s is self-healing — if a Pod dies, the controller notices and creates a new one.

3. Everything is an API Resource#

k8s is fundamentally an API. Every object (Pod, Service, Deployment) is a REST resource. This makes it:

  • Extensible via CustomResourceDefinitions (CRD) — define your own resource types
  • Automatable — everything kubectl does, you can do via API

4. Loose Coupling via Labels & Selectors#

Components don’t reference each other by name. A Service doesn’t know which specific Pods it routes to — it selects by labels.

labels and selectors

Add/remove Pods freely — the Service automatically updates its endpoint list.

5. Immutable Infrastructure#

Don’t patch a running Pod. Replace it with a new one.

immutable update

6. Single Responsibility#

Each component does one thing: scheduler only schedules, kubelet only manages local pods, etc. This composability allows swapping components (e.g. different schedulers, CNI plugins).

Fundamentals#

Linux Primitives Under the Hood#

Every k8s abstraction maps to a Linux kernel feature. Understanding this makes k8s behavior much less magical.

Linux Namespaces → Pod Isolation#

A namespace partitions a global kernel resource so each process sees its own isolated view. When k8s creates a Pod, the container runtime creates a set of namespaces for it:

linux namespaces

Two containers in the same Pod share the net and ipc namespaces — that’s why they can talk via localhost and share memory. Each container gets its own mnt namespace.

namespaces inside a pod

Verify on the host:

# find the PID of a container process on the host
crictl inspect <container-id> | grep pid

# see its namespaces
ls -la /proc/<pid>/ns/
# net -> net:[4026531992]   ← same net ns as other containers in the pod
# mnt -> mnt:[4026532xxx]   ← unique per container

cgroups → resource.requests/limits#

cgroups (control groups) limit and account for resource usage. When you set resources in a Pod spec, kubelet translates them to cgroup config:

cgroups v2 is the default since k8s 1.25 (GA). File names differ from v1:

resources to cgroup files

cgroup hierarchy on the node (cgroupv2 + systemd): cgroup hierarchy

QoS class matters: when node is under memory pressure, BestEffort pods are OOM-killed first, then Burstable, never Guaranteed.

veth pairs + bridge → Pod networking#

Each Pod gets its own network namespace with a virtual ethernet interface. The CNI plugin wires it to the host:

veth and bridge

Step by step when a Pod is created: pod network setup

The pause container is the invisible container you never write — its only job is to hold namespaces so app containers can restart without losing the Pod’s IP.

overlayfs → container image layers#

Container images are stacked read-only layers. overlayfs merges them into one coherent filesystem:

overlayfs layers

This is why:

  • Multiple containers sharing the same image use almost no extra disk — they share lowerdir
  • Container writes are ephemeral — upperdir is deleted when container dies
  • PersistentVolumes mount into the container bypassing overlayfs entirely

iptables → Service (ClusterIP)#

kube-proxy programs iptables rules to implement Service load balancing. When you create a Service:

iptables chains

Dry run — packet flow for a ClusterIP request: clusterip packet flow

kube-proxy has three backend modes:

Mode Mechanism Lookup Status
iptables netfilter rules, random selection O(n) default, stable
ipvs kernel virtual server table O(1) stable, better for large clusters
nftables successor to iptables, atomic rule updates O(n→better) beta since k8s 1.31

Summary: k8s abstraction → Linux primitive#

k8s to linux primitives

Pod Lifecycle#

pod lifecycle

Probes that kubelet runs:

  • livenessProbe — is the app alive? Restart container if it fails
  • readinessProbe — is the app ready to serve traffic? Remove from Service endpoints if it fails
  • startupProbe — has the app finished starting? (for slow-starting apps)

Scheduling#

The scheduler picks a Node for each new Pod in two phases:

scheduling

Key scheduling controls:

Mechanism Purpose
resources.requests/limits Reserve and cap CPU/memory per container
nodeSelector Pin Pod to nodes with specific labels
affinity/antiAffinity Soft/hard rules for co-location or spreading
taints & tolerations Mark nodes as special (GPU, infra); Pods must tolerate to land there

Networking Model#

k8s mandates a flat network:

  • Every Pod gets its own IP
  • Any Pod can reach any other Pod directly (no NAT)
  • Implemented by a CNI plugin (Flannel, Calico, Cilium, etc.)

flat pod network

Service networking (how ClusterIP works): service routing

Rolling Updates#

rolling update

Rollback is instant — Deployment keeps previous ReplicaSet around:

kubectl rollout undo deployment/nginx

Learning Path#

learning path

References#

Official Kubernetes Docs

Linux Fundamentals

Recommended Reading

  • Kubernetes in Action (Marko Luksa) — best book for deep understanding of k8s internals
  • Container Security (Liz Rice) — Linux primitives (namespaces, cgroups, seccomp) under containers
  • Gateway API docs — successor to Ingress, GA since 2023
  • Kubernetes blog — release notes and deep-dive posts per version