Kubernetes SRE guide

Kubernetes Autoscaling: HPA, KEDA, Capacity Guardrails

Autoscaling works when demand signals, pod startup time, cluster capacity, and downstream limits are designed as one operating system.

Published: Jul 14, 2026
Updated: Jul 14, 2026
Reading time: 14 min read

On this page

Kubernetes autoscaling can add replicas, but replicas alone do not guarantee lower latency or more throughput. A service may be limited by an unscalable database, connection pool, external API quota, slow image pull, pending pod, or cold cache. Treat autoscaling as a feedback loop with explicit signals, limits, and validation, not a substitute for capacity planning.

Scale on work, not on a convenient chart

CPU can be a useful proxy when it tracks saturation. When it does not, choose a signal closer to constrained work, such as in-flight requests, queue depth, consumer lag, or a safely aggregated custom metric.

Overview

Outcome and prerequisites

Outcome: Acme Shop can scale its stateless catalog API for a measured traffic ramp without overrunning its database connection budget. Prerequisites: a non-production namespace, measured CPU and startup behavior, a ready Deployment, Metrics Server or an equivalent resource-metrics pipeline, node headroom, and a database owner-approved connection ceiling.

Running scenario: Acme Shop catalog API

Acme Shop's catalog-api serves public product reads and uses a small database pool for cache misses. Load testing shows a pod is useful through roughly 70% average CPU, takes 45 seconds to become ready, and opens at most eight database connections. The sandbox database allows 80 application connections, so the team chooses a maximum of eight replicas, reserving capacity for other clients. HPA is a guardrail, not permission to scale beyond the dependency's tested limit.

Acme Shop catalog autoscaling loop

Demand signal
Prometheus-compatible resource metrics report sustained pod CPU.
HPA recommendation
HPA stays within minimum and maximum replica guardrails.
Scheduling
Nodes must have allocatable resources and IP capacity for pending pods.
Readiness
New pods warm safely before accepting catalog traffic.
Dependency protection
Per-pod pools keep aggregate database connections within the approved ceiling.

Figure 1. Scaling is successful only when a requested replica schedules, becomes ready, and can complete useful work without saturating a dependency.

Configure HPA behavior for the workload

Set requests from measured behavior. Requests inform scheduling; copying values from another service does not create capacity. A CPU limit can increase tail latency through throttling, while too-low memory limits can create restarts. Validate them under representative load. The HPA below is a sandbox starting point. It uses the autoscaling/v2 resource metric and deliberately limits scale-up and scale-down changes.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: catalog-api
  namespace: acme-shop-sandbox
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: catalog-api
  minReplicas: 2
  maxReplicas: 8
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Pods
          value: 2
          periodSeconds: 60
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 60
      selectPolicy: Max

With eight connections per pod, eight catalog replicas can request up to 64 database connections. The remaining 16 connections are headroom for migrations, administration, and other approved consumers. This is a planning calculation, not proof that the database can sustain the resulting query rate; validate pool wait, query latency, cache miss rate, and database saturation together.

KEDA can suit queue workers that scale from backlog or consumer lag, including scale-to-zero where startup and first-message delay are acceptable. Choose its trigger threshold from arrival rate, processing rate, and an accepted queueing delay. Test delayed metrics, poison messages, broker authentication failures, uneven partitions, and a downstream service that cannot absorb more consumers. Handlers must be idempotent and concurrency-bounded.

Representative output: Acme Shop catalog HPA status

NAME          REFERENCE                 TARGETS     MINPODS  MAXPODS  REPLICAS  AGE
catalog-api   Deployment/catalog-api     82%/70%     2        8        4         18m

event=SuccessfulRescale reason=NewSize message="New size: 4; reason: cpu resource utilization above target"
pending_pods=0 ready_replicas=4 db_pool_active=28 db_pool_wait_p95=4ms

Validate the complete loop

Measure time from HPA decision to ready replica: scheduling, node provisioning if required, image pull, initialization, readiness, and load-balancer propagation. Compare it to the fastest credible traffic rise. If it arrives too late, maintain measured headroom, reduce startup work, improve image distribution, or apply a product-approved traffic-shaping policy. Do not assume a load test that begins with warm pods validates a burst.

Validation

Validation, rollback, and failure behavior

In the sandbox, establish a baseline and run a gradual catalog traffic ramp followed by one bounded step increase. Observe SLOs, HPA recommendations, ready and pending replicas, node availability, database pool wait, cache misses, and recovery after load falls. Stop the test if errors, queueing, or dependency saturation breach the agreed guardrail. Restore only the prior reviewed HPA manifest or Deployment revision through the normal change path, then let the stabilization window settle. If metrics are stale, missing, or implausible, hold the current safe replica count and follow the runbook; do not raise maxReplicas or disable dependency limits as an emergency default.

Troubleshooting

Symptom	Likely cause	Safe check	Recovery
Desired replicas rise but pods stay Pending	Node capacity, quota, affinity, IP, or volume constraint	Inspect pod conditions and namespace quota	Pause the load increase and restore capacity or placement rules before retrying.
More replicas increase errors	Database pool, third-party quota, or retry storm is the real limit	Compare dependency saturation and per-pod concurrency	Reduce the approved replica ceiling or caller concurrency; fix the dependency bottleneck.
HPA oscillates	Noisy metric or scale-down behavior is too aggressive	Compare metric freshness with recommendation history	Correct aggregation and use a tested stabilization window.
New pods do not improve latency	Startup, readiness, image pull, or cache warming takes too long	Measure each interval from scale decision to traffic admission	Preserve warm headroom or reduce startup work before changing targets.
Worker backlog grows after scale-out	Poison messages, uneven partitions, or non-idempotent retries	Inspect a bounded sample and consumer lag by partition	Pause unsafe consumers, quarantine poison work, and repair handler behavior.

Authoritative references

Align traffic delivery with origin scaling

Talk to Optimi about edge and delivery performance architecture that complements, rather than replaces, safe Kubernetes scaling controls.

Discuss scaling architecture

Running scenario: Acme Shop catalog API

Configure HPA behavior for the workload

Validate the complete loop

Related guides

Authoritative references

Align traffic delivery with origin scaling