Kubernetes SRE guide

Kubernetes Autoscaling: HPA, KEDA, Capacity Guardrails

Autoscaling works when demand signals, pod startup time, cluster capacity, and downstream limits are designed as one operating system.

Published: Jul 14, 2026
Updated: Jul 15, 2026
Reading time: 9 min read

On this page

Kubernetes autoscaling can add replicas, but replicas alone do not guarantee lower latency or more throughput. A service may be limited by an unscalable database, connection pool, external quota, slow image pull, pending pod, or cold cache. Treat autoscaling as a feedback loop with explicit signals, limits, and validation, not a substitute for capacity planning.

That discipline matters even more once a managed orchestration layer steers traffic in front of the origin: if HPA and KEDA scale a backend faster or slower than the edge expects, Performance, Security, and Visibility all inherit the mismatch, so scaling guardrails belong in the same system view as the edge, not reviewed alone.

Scale on work, not on a convenient chart

CPU can be a useful proxy when it tracks saturation. When it does not, choose a signal closer to constrained work, such as in-flight requests, queue depth, consumer lag, or a safely aggregated custom metric.

Overview

Outcome and prerequisites

Outcome: Acme Shop can scale its stateless catalog API for a measured traffic ramp without overrunning its database connection budget, and scale a queue-backed order-events consumer without breaching the same guardrails. Prerequisites: a non-production namespace, measured CPU and startup behavior, a ready Deployment, Metrics Server or an equivalent metrics pipeline, node headroom, an approved connection ceiling, and a reachable backlog metric source with an agreed activation threshold.

Running scenario: Acme Shop catalog API

Acme Shop's catalog-api serves public product reads and uses a small database pool for cache misses. Load testing shows a pod is useful through roughly 70% average CPU, takes 45 seconds to become ready, and opens at most eight database connections. The sandbox database allows 80 application connections, so the team chooses a maximum of eight replicas, reserving capacity for other clients. HPA is a guardrail, not permission to scale beyond the dependency's tested limit. Acme Shop also runs an order-events consumer drained by queue depth, not CPU; the guide returns to it later to show how HPA and KEDA divide that work.

Acme Shop catalog autoscaling loop

Demand signal
Prometheus-compatible resource metrics report sustained pod CPU.
HPA recommendation
HPA stays within minimum and maximum replica guardrails.
Scheduling
Nodes must have allocatable resources and IP capacity for pending pods.
Readiness
New pods warm safely before accepting catalog traffic.
Dependency protection
Per-pod pools keep aggregate database connections within the approved ceiling.

Figure 1. Scaling is successful only when a requested replica schedules, becomes ready, and can complete useful work without saturating a dependency.

Configure HPA behavior for the workload

Set requests from measured behavior; copying values from another service does not create capacity. A CPU limit can increase tail latency through throttling, while too-low memory limits create restarts. Validate them under representative load. The HPA below is a sandbox starting point, using the autoscaling/v2 resource metric and deliberately limiting scale-up and scale-down changes.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: catalog-api
  namespace: acme-shop-sandbox
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: catalog-api
  minReplicas: 2
  maxReplicas: 8
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Pods
          value: 2
          periodSeconds: 60
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 60
      selectPolicy: Max

With eight connections per pod, eight catalog replicas can request up to 64 database connections. The remaining 16 connections are headroom for migrations, administration, and other approved consumers. This is a planning calculation, not proof that the database can sustain the resulting query rate; validate pool wait, query latency, cache miss rate, and database saturation together.

Representative output: Acme Shop catalog HPA status

NAME          REFERENCE                 TARGETS     MINPODS  MAXPODS  REPLICAS  AGE
catalog-api   Deployment/catalog-api     82%/70%     2        8        4         18m

event=SuccessfulRescale reason=NewSize message="New size: 4; reason: cpu resource utilization above target"
pending_pods=0 ready_replicas=4 db_pool_active=28 db_pool_wait_p95=4ms

Tune scaling guardrails: tolerance, stabilization, and missing metrics

The behavior block is only half of what shapes a decision. The controller also applies a default tolerance, skipping a scaling action when the current-to-desired metric ratio stays within roughly 10% of 1.0. On an eight-replica ceiling like catalog-api, that band is close to one replica of slack — why 82%/70% triggered a rescale but 74%/70% would not. Kubernetes 1.33 added an alpha, gated HPAConfigurableTolerance feature letting one HPA set different scaleUp/scaleDown tolerances; treat it as forward-looking, not something to depend on before it graduates past alpha.

The controller is also conservative when metrics are incomplete: a pod missing a reading counts as 100% of target for scale-down but 0% for scale-up, and not-yet-ready pods are excluded the same way — dampening both directions during exactly the rollout or metrics hiccup when a wrong guess would be costliest, including the default CPU initialization grace period right after a rollout (see troubleshooting below).

HPA and KEDA: how the two mechanisms divide the scaling work

KEDA does not replace HPA; it extends it. A ScaledObject watches an event source — queue depth or consumer lag — and owns the 0-to-1 "activation" decision itself. Once one replica should exist, KEDA creates a standard autoscaling/v2 HPA behind the scenes for the 1-to-N range, so the tolerance and missing-metric behavior above applies exactly as it does to catalog-api.

Acme Shop's order-events consumer fits that split: idle at an empty queue, but must not fall behind once backlog appears. A representative ScaledObject for the sandbox:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-events-consumer
  namespace: acme-shop-sandbox
spec:
  scaleTargetRef:
    name: order-events-consumer
  pollingInterval: 30
  cooldownPeriod: 300
  minReplicaCount: 0
  maxReplicaCount: 6
  triggers:
    - type: rabbitmq
      metadata:
        queueName: order-events
        mode: QueueLength
        value: "50"

pollingInterval (30s default) sets how often KEDA checks the trigger; cooldownPeriod (300s default) sets how long backlog must clear before scaling toward minReplicaCount. minReplicaCount defaults to 0, enabling scale-to-zero, and maxReplicaCount defaults to 100 if unset — the same connection reasoning that capped catalog-api at eight replicas applies here, so Acme Shop sets maxReplicaCount: 6 rather than trusting the default.

Scale-to-zero has a real cost: the first message after a cold start typically adds a couple of seconds to well over ten, depending on image and startup work — fine for order-events, where queueing is invisible to the buyer, but not for a synchronous, latency-sensitive path. Handlers must also stay idempotent and concurrency-bounded, since a scale-out event can hand the same backlog to more workers than existed a moment earlier.

KEDA activates queue workers and HPA scales the active range

KEDA owns the zero-to-one activation decision; the generated HPA applies standard scaling behavior once the consumer is active.

Download:PNG SVG

Correlate scaling decisions where the edge already sees them

HPA and KEDA only see origin-side signals; neither tells you whether a scale-up resolved what a user felt at the edge. A managed orchestration layer such as MYO lines up origin replica counts and consumer lag against edge-side error rate and latency for the same window, so a scaling guardrail breach reads as one correlated event, not scattered readings reconciled by hand.

Validate the complete Kubernetes autoscaling loop

Measure time from HPA decision to ready replica: scheduling, node provisioning if required, image pull, initialization, readiness, and load-balancer propagation. Compare it to the fastest credible traffic rise. If it arrives too late, maintain measured headroom, reduce startup work, improve image distribution, or apply a traffic-shaping policy. A load test starting with warm pods does not validate a burst — and for the queue path, one starting with warm consumers does not validate a genuine cold start.

Validation

Validation, rollback, and failure behavior

In the sandbox, establish a baseline and run a gradual catalog traffic ramp followed by one bounded step increase, and separately drive order-events backlog from zero through the activation threshold. Observe SLOs, HPA recommendations, ready and pending replicas, node availability, database pool wait, cache misses, cold-start latency, and recovery after load falls. Stop the test if errors, queueing, or dependency saturation breach the agreed guardrail. Restore only the prior reviewed manifest or revision through the normal change path, then let the stabilization window and cooldown period settle. If metrics are stale or implausible, hold the current safe replica count and follow the runbook rather than raising maxReplicas, maxReplicaCount, or dependency limits.

Troubleshooting

Symptom	Likely cause	Safe check	Recovery
Desired replicas rise but pods stay Pending	Node capacity, quota, affinity, IP, or volume constraint	Inspect pod conditions and namespace quota	Pause the load increase and restore capacity or placement rules before retrying.
More replicas increase errors	Database pool, third-party quota, or retry storm is the real limit	Compare dependency saturation and per-pod concurrency	Reduce the approved replica ceiling or caller concurrency; fix the dependency bottleneck.
HPA oscillates	Noisy metric or scale-down behavior is too aggressive	Compare metric freshness with recommendation history	Correct aggregation and use a tested stabilization window.
Replica count barely moves under real load swings	Default ~10% HPA tolerance masks a genuine trend	Compare the trend across several sync periods	Confirm the metric, then narrow tolerance rather than the target.
New pods do not improve latency	Startup, readiness, image pull, or CPU initialization grace period hides a real reading	Measure each interval from scale decision to traffic admission	Preserve warm headroom or reduce startup work before changing targets.
Worker backlog grows after scale-out	Poison messages, uneven partitions, or non-idempotent retries	Inspect a bounded sample and consumer lag by partition	Pause unsafe consumers, quarantine poison work, and repair handler behavior.
KEDA never scales past zero despite real backlog	Scaler cannot reach the event source, or activation threshold sits too high	Check the ScaledObject and generated HPA conditions	Fix source connectivity or auth before assuming demand is absent.

Authoritative references

Scale with guardrails the edge can trust

Talk to Optimi about correlating HPA and KEDA decisions with edge Performance, Security, and Visibility through MYO, so autoscaling and edge orchestration move as one system.

Discuss scaling architecture