Stateless Services on Kubernetes: Design, Deploy, and Scale Safely

Published

Jul 14, 2026

Updated

Jul 15, 2026

Reading time

15 min read

Kubernetes makes replacement normal: pods are restarted, rescheduled, upgraded, and scaled. A service that depends on one pod's memory, local disk, or network identity will eventually fail in ways that are difficult to reproduce. Stateless design turns those lifecycle events into routine operations — the same property a managed edge-orchestration layer depends on, since Performance and Security decisions made in front of the origin only stay correct if every replaceable instance behind them answers a request the same way.

Stateless does not mean a service has no state. It means durable business state lives in a suitable external system, and any instance can process a request using that state. A pod may cache data or hold an in-flight connection, but correctness must not depend on that transient data surviving. Separating durable state from disposable state this way is the core of stateless service design, and the rest of this guide works through how to apply it on Kubernetes without losing throughput or correctness along the way.

Identify the state before moving it

List every item that a process keeps beyond a single request: login sessions, upload progress, job ownership, rate-limit counters, idempotency keys, generated files, WebSocket routing data, and cache entries. For each item, decide whether it is durable, reconstructable, or disposable.

• Store durable records in a database, object store, or queue with an explicit retention and consistency model.
• Store shared ephemeral state, such as a distributed session or lease, in a purpose-built shared store when it is truly needed.
• Keep reconstructable caches local only if a cold cache does not harm correctness or overload the origin.
• Treat pod-local filesystems as temporary unless a workload deliberately uses a persistent volume and its operational constraints.

Avoid using sticky sessions to hide statefulness. Affinity can improve cache locality, but it does not protect against a pod restart, rollout, or zone failure. If a user action must reach the same instance to succeed, the system has an availability and scaling constraint that should be made explicit.

Moving state out of the pod introduces a second-order problem: several replicas can now race to read, modify, and write the same external record. A stateless service design does not eliminate concurrency control, it relocates it. Use an optimistic lock (a version column or ETag precondition) for read-modify-write updates, a database unique constraint for "claim this work once" semantics, and a short-lived distributed lease only when one owner genuinely must run a task at a time. Treat any in-memory rate limiter or deduplication cache as approximate unless it is backed by the same shared store every replica reads.

Build stateless services on Kubernetes that survive replacement

Start with an immutable image and a small, explicit runtime contract. The process should bind to the configured port, expose health endpoints, read configuration from the deployment environment, and emit logs to standard output or a collector. It should not expect a particular node, public IP, or writable application directory.

Use a Deployment for independently replaceable HTTP or gRPC services. Give it resource requests based on measured steady-state demand and limits chosen to prevent one workload from harming others. A CPU limit can cause throttling and long tail latency; measure before applying it universally. Memory limits are essential protection, but an out-of-memory restart is not a capacity plan.

Readiness, liveness, and startup probes

Configure probes to reflect distinct questions:

• Startup: has the application completed its bounded initialization?
• Readiness: can this instance accept its assigned traffic now?
• Liveness: is the process unrecoverably stuck and worth restarting?

Keep probe endpoints inexpensive and independent of user authentication. Do not make them perform expensive queries or call every optional downstream dependency. During an outage, a dependency-sensitive liveness check can cause a restart storm; readiness can remove an instance from traffic when it cannot safely serve, while circuit breakers and fallbacks protect dependency calls.

Kubernetes ships conservative probe defaults: periodSeconds: 10, timeoutSeconds: 1, failureThreshold: 3, checked from initialDelaySeconds: 0. A startup probe holds liveness and readiness inactive until it first succeeds, which is what makes a slow-booting JVM or cache-warming process survivable without a false restart. But the liveness math has a consequence worth stating explicitly: at the defaults, a liveness probe declares a container dead after failureThreshold * periodSeconds, or 30 seconds — a number that must stay smaller than terminationGracePeriodSeconds, or a stuck process can be killed before its own shutdown sequence gets a fair chance to run.

Spread replicas across zones and nodes

A Deployment with several replicas offers no protection if the scheduler happens to place them all on one node or in one zone. topologySpreadConstraints bounds that imbalance directly:

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector: { matchLabels: { app: checkout-api } }

maxSkew caps how unevenly pods matching labelSelector may be distributed across the values of topologyKey; whenUnsatisfiable: DoNotSchedule treats it as a hard requirement rather than a preference. Pair this with a Pod Disruption Budget, and set unhealthyPodEvictionPolicy deliberately — the default, IfHealthyBudget, can stall a node drain behind a pod that is running but will never pass readiness on its own, because evicting it would drop the guarded application below its desired healthy count.

Drain requests and work correctly

Kubernetes removes a container after its termination grace period, but its lifecycle order matters: kubelet starts that grace period, runs preStop, and then sends SIGTERM if the container is still running. The application must cooperate. On termination, fail readiness immediately, stop accepting new connections, and allow in-flight requests a bounded time to finish. Align load-balancer draining, pre-stop behavior, application shutdown timeout, and terminationGracePeriodSeconds so they do not contradict one another.

For asynchronous workers, use acknowledged queues and idempotent handlers. A worker can be terminated after receiving a message but before recording its result. Make duplicate delivery safe with a durable idempotency key or transactional outbox pattern. Extend a message lease only while the worker is healthy, and route repeatedly failing work to a dead-letter process with an owner.

Long-lived connections deserve a separate plan. WebSockets, streaming responses, and large uploads cannot be drained like a 50 ms API call. Set maximum connection age, communicate reconnect behavior to clients, and roll out gradually. For global audiences, consider where connection termination and application state live; an edge connection layer may reduce round-trip latency, but the origin protocol still needs safe reconnect and state recovery behavior.

If the pod runs a sidecar, its shutdown order matters as much as the main container's. Native sidecar containers — declared as initContainers entries with restartPolicy: Always, stable since Kubernetes 1.33 — start before the main container and stop after it: the kubelet sends SIGTERM to main containers first, waits for them to exit, and only then terminates sidecars in reverse startup order. That ordering is what makes a service-mesh proxy or log shipper reliable during shutdown; a plain second container with no defined ordering offers no such guarantee, and its drain behavior should be treated as untested until observed under load.

Horizontal pod scaling without moving the bottleneck

Horizontal pod scaling with the HorizontalPodAutoscaler can add replicas based on CPU, memory, or custom metrics. Pick a signal related to useful capacity. Queue depth, queue age, active requests, or concurrency may be more meaningful than CPU for an I/O-bound service. Validate the complete loop: metric freshness, scale-up delay, node capacity, image pull time, application warm-up, and downstream connection limits.

The autoscaling/v2 API separates the scaling decision from the rate at which it is applied. By default, the controller ignores a computed change inside a ±10% tolerance band around the current replica count, which avoids thrashing on metric noise. The behavior field then lets you shape scale-up and scale-down independently:

behavior:
  scaleDown:
    stabilizationWindowSeconds: 300
    policies: [{ type: Percent, value: 25, periodSeconds: 60 }]
  scaleUp:
    stabilizationWindowSeconds: 0
    policies: [{ type: Pods, value: 4, periodSeconds: 15 }]

A short or zero stabilization window on scale-up gets capacity in place quickly during a spike; a long window on scale-down (five minutes by default) avoids releasing pods traffic will need again a minute later. Tune the two independently, and remember that horizontal pod scaling only helps if a new pod can reach ready state, and pass its downstream connection budget, before the traffic that triggered the scale-up arrives.

Set a concurrency limit at the application or proxy layer. Otherwise, more traffic can create unbounded goroutines, threads, promises, or database calls inside each replica. Backpressure is a correctness feature: queue, shed low-priority work, return a retryable response, or serve a cached/degraded result before the service reaches failure.

Capacity planning must include shared resources. If each new pod opens twenty database connections, an autoscaler that grows from ten to one hundred pods requests two thousand connections. Cap per-pod pools, use a database-aware proxy where appropriate, and alert on pool wait time, not only pod CPU.

Keep latency visible from edge to dependency

Measure user-facing latency as well as cluster metrics. A healthy pod can still produce a slow experience because of DNS, TLS, cache misses, regional routing, a serial API chain, or an overloaded database. Propagate W3C trace context from the ingress or edge to the service and dependency clients, and attach route, release version, region, and outcome to telemetry.

Track p50, p95, and p99 latency, error rate, saturation, queue age, connection-pool wait, and cache hit ratio. Correlate them with rollout and autoscaling events. The OpenTelemetry Kubernetes guidance is a useful starting point for collecting signals without making logs the only source of truth.

Be careful with the pod's own identity as a metric label: a pod name or IP creates a new time series every time a replica is replaced, and a fleet scaling between ten and a hundred pods can quietly turn a modest metric set into a cardinality problem that slows dashboards and inflates storage cost. Keep pod identity in logs and trace exemplars, where it is genuinely useful for debugging one request, and leave it out of the label set every replica writes on every scrape.

For cacheable public traffic, use correct Cache-Control and cache-key rules at the edge. Keep authenticated, personalized, and mutation routes explicit. Edge caching reduces origin load only when invalidation, variation, and authorization semantics are correct; it should never cache a response merely because the origin was slow.

Test the replacement contract

Before calling a service stateless, run controlled tests:

1. Delete one pod during live-like traffic and confirm requests drain or retry safely.
2. Roll out a new version while holding long-running requests and queue work.
3. Scale replicas up and down while watching database connections and tail latency.
4. Simulate a dependency timeout and verify deadlines, circuit breakers, and fallback behavior.
5. Remove a node or zone in a non-production environment and confirm topology and capacity assumptions.
6. Cordon and drain a node with kubectl drain, or use a chaos-engineering tool, to confirm the Pod Disruption Budget and unhealthyPodEvictionPolicy behave the way you expect under real eviction pressure, not only in the manifest.

Also test a cold start. An image that takes minutes to pull or initialize cannot respond quickly to a traffic spike, even if the autoscaler chooses the right replica count.

Apply it: Acme Shop checkout API

checkout-api records an idempotency key and order intent in PostgreSQL before publishing fulfillment work to a queue. It may cache product data locally, but its filesystem, memory, and connection identity are never the only record of an order.

State inventory and migration

Migrate one item at a time: dual-read the old and new record, write the durable record, verify reconciliation counts, then remove the local path only after the rollback window closes.

Drain contract

Kubelet begins terminationGracePeriodSeconds before it calls preStop; it sends SIGTERM only after the hook returns if the process is still running. The /drain endpoint must make the pod unready, stop accepting new work, and wait for in-flight work within that same grace period. SIGTERM is the fallback shutdown path if the hook returns early or cannot complete, not the event that begins the drain.

apiVersion: apps/v1
kind: Deployment
metadata: { name: checkout-api, namespace: acme-shop-sandbox }
spec:
  selector: { matchLabels: { app: checkout-api } }
  template:
    metadata: { labels: { app: checkout-api } }
    spec:
      terminationGracePeriodSeconds: 45
      containers:
        - name: checkout-api
          image: registry.example.invalid/acme/checkout-api@sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc
          lifecycle: { preStop: { httpGet: { path: /drain, port: 8080 } } }
          readinessProbe: { httpGet: { path: /readyz, port: 8080 }, periodSeconds: 3 }

checkout-api preStop=/drain ready=false accepting_new_work=false inflight=2
checkout-api preStop=/drain inflight=0 queue_lease_released=1
checkout-api sigterm=received fallback_shutdown=true exit=0

Zone spread and disruption policy

checkout-api runs across three zones in the sandbox cluster, and its Pod Disruption Budget is deliberately paired with a spread constraint so the two policies reinforce each other instead of fighting during a node drain:

spec:
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: DoNotSchedule
      labelSelector: { matchLabels: { app: checkout-api } }
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata: { name: checkout-api, namespace: acme-shop-sandbox }
spec:
  minAvailable: 2
  unhealthyPodEvictionPolicy: AlwaysAllow
  selector: { matchLabels: { app: checkout-api } }

minAvailable: 2 keeps two ready replicas through a voluntary disruption; unhealthyPodEvictionPolicy: AlwaysAllow means an already-failing checkout-api pod cannot block a node drain waiting to become healthy on its own, so one stuck pod cannot hold up maintenance on every node.

Validation, rollback, and failure behavior

In a sandbox, submit one test checkout, begin kubectl rollout restart deployment/checkout-api -n acme-shop-sandbox, and verify /drain marks the pod unready before it stops new work, then verify the order remains exactly once in PostgreSQL while queue work completes or redelivers safely. If the drain endpoint cannot finish within 45 seconds, or its SIGTERM fallback cancels unsafe work, halt promotion, return to the previous ReplicaSet, and leave the durable record and queue message intact for reconciliation.

Related guides

• Modern Twelve-Factor Apps
• Kubernetes Autoscaling
• Dependency Resilience
• Safe Container Releases

Authoritative references

• The Twelve-Factor App: Processes
• Kubernetes: Deployments
• Kubernetes: Pod lifecycle and termination
• Kubernetes: Horizontal Pod Autoscaling
• Kubernetes: Pod topology spread constraints
• Kubernetes: Disruptions and Pod Disruption Budgets
• Kubernetes: Sidecar containers
• OpenTelemetry on Kubernetes