Imagine: a service with 20 pods steadily handles 500 RPS, but every evening at 7:00 PM a peak of 2000 RPS hits. You've configured the Horizontal Pod Autoscaler based on CPU — and it stays silent. Under load, the CPU doesn't grow because the bottleneck is database locks or waiting for an external API response. Pods don't scale, latency grows, alerts fire. Sound familiar? This is the classic autoscaling trap: the tool works, but it measures the wrong thing.
In this article, we'll break down how autoscaling in Kubernetes actually works, compare three approaches — HPA, VPA, and KEDA — and show with an example where each of them saves money and where it creates problems. And we'll also discuss why learning these mechanisms on your own often hits a wall of no feedback and costly mistakes.
Why autoscaling is not just "enable HPA"
Autoscaling in Kubernetes is not a single button but a set of controllers, each with its own logic and limitations. HPA (Horizontal Pod Autoscaler) changes the number of replicas, VPA (Vertical Pod Autoscaler) — the resources (CPU/memory) of a container, KEDA (Kubernetes Event-driven Autoscaling) adds scaling based on events from queues, Kafka, cron, and other sources. All of them work through the metrics API (metrics-server, Prometheus Adapter, custom metrics API), and that's where the difficulties begin.
The key idea: an autoscaler is a feedback control loop. It periodically polls metrics, compares them with the target value, and changes the desired state (replicas or resources). The latency of this loop, the quality of the metrics, and the correct choice of the target indicator determine whether the system will be stable or go into oscillation.
Real case breakdown: traffic peaks and order queue
Let's take a typical online store. There is an API service that processes orders, and a RabbitMQ queue where tasks for reserving goods land. In normal mode, the API consumes 200m CPU per pod, at peak — up to 800m. HPA is configured with targetCPUUtilizationPercentage: 70. It seems simple.
But at peak, CPU doesn't have time to grow: the API spends most of its time waiting for a response from the payment gateway. HPA sees average utilization of 40% and doesn't scale. The queue grows, orders are processed with delays of minutes.
What a practitioner does:
1. Switches HPA to a custom metric — for example, http_requests_per_second from Prometheus Adapter. The target value is 100 RPS per pod. Now scaling responds to real load, not CPU.
2. For the order queue, connects KEDA with a trigger based on RabbitMQ queue length. KEDA can scale to zero when the queue is empty and quickly spin up pods during a spike.
3. Configures VPA in Off mode (recommendations only) to understand real resource requests and avoid conflicting with HPA.
Result: latency at peak drops from several seconds to hundreds of milliseconds, and at night KEDA scales pods to zero, saving on unused resources.
If they had left HPA on CPU, the problem would only worsen: pods wouldn't be added, and VPA in Auto mode could suddenly restart pods, changing requests, leading to temporary unavailability.
Three approaches: comparing in practice
HPA: horizontal scaling based on metrics
HPA increases the number of replicas based on observed metrics. It supports CPU, memory, custom metrics (e.g., RPS), and external metrics (from Kafka, cloud services). The main parameter is the target value. The algorithm is simple: desiredReplicas = ceil(currentReplicas * (currentMetricValue / targetMetricValue)).
Where it breaks: metric latency. If Prometheus collects data every 30 seconds and HPA polls every 15, the reaction to a peak can take a minute or more. During that time, users will already see timeouts.
Tip: use behavior in HPA (available since Kubernetes 1.18) to configure scaleUp/scaleDown policies to avoid sharp jumps and oscillation.
VPA: vertical scaling and recommendations
VPA automatically selects requests/limits for containers based on historical consumption. It works in three modes: Off (recommendations only), Initial (applies when a pod is created), Auto (applies and restarts pods when changed).
The main pitfall: VPA in Auto mode restarts pods when resources change. In production, this can lead to failures if PodDisruptionBudget is not configured. Also, VPA conflicts with HPA if both use CPU or memory as a metric. The correct scheme: HPA scales on a custom metric (RPS), VPA manages only memory (or vice versa), or VPA runs in Off mode to collect recommendations.
KEDA: event-driven scaling
KEDA extends HPA by adding event-driven triggers: queue length, Kafka lag, cron schedule, metrics from a database. It can scale to zero (scale-to-zero), which is critical for saving during off-peak hours.
Limitation: scale-to-zero implies that the application can start quickly (cold start). If your service initializes in 30 seconds, the first request after idle may time out. The solution is to use minReplicaCount: 1 for critical services or configure warm-up.
Where the mechanism breaks: three failure scenarios
1. Custom metric latency
You've configured HPA on a metric from Prometheus Adapter, but the adapter caches data or Prometheus itself can't keep up. As a result, HPA makes decisions based on stale data. How to notice: compare the time between the load spike and the moment replicas increase. If the delay exceeds 1-2 minutes, consider more frequent metric collection or using external metrics directly from the application.
What to do: reduce the scrape interval in Prometheus (e.g., to 15 seconds), configure HPA with behavior.scaleUp.stabilizationWindowSeconds: 0 for quick reaction, but with caution.
2. HPA and VPA conflict
If HPA and VPA simultaneously manage CPU, they start "pulling the blanket": HPA increases replicas, VPA increases requests, which lowers utilization and provokes further replica growth. The cluster goes into a storm of pod recreations.
How to notice: unstable number of replicas, frequent pod restarts without load changes. What to do: separate areas of responsibility. For example, HPA — on RPS, VPA — on memory. Or use VPA only in Off mode to collect recommendations and manually update manifests.
3. Scale-to-zero and cold start
KEDA scales to zero, but the application is not ready for quick startup. The first request after idle takes 10-20 seconds, users leave. How to notice: monitor response time for the first request after a period of low load. What to do: set minReplicaCount: 1 for user-facing APIs, or use KEDA with advanced settings, e.g., horizontalPodAutoscalerConfig with behavior for smooth startup.
Learning paths: self-study, courses, practice
Autoscaling in Kubernetes is an area where theory without practice is useless. You can read the documentation, but configuring HPA, VPA, and KEDA together without feedback is difficult: mistakes don't show up immediately but in production under peak load.
Self-study path: Kubernetes documentation, articles, experiments in minikube or kind. Pros — free, cons — no structure, easy to miss important details (e.g., interaction between VPA and HPA), and mistakes in production can be costly.
Courses and structured learning: provide sequence, case studies, and feedback. For example, the course Kubernetes in Production covers exactly such scenarios: how to configure HPA on custom metrics, avoid conflicts with VPA, use KEDA for event-driven scaling. Personalized lessons tailored to the student's level allow immediate application of knowledge to your cluster.
Practice in real projects: the most valuable experience, but requires time and responsibility. If you work in a team with a mentor, this is the best option.
Honest trade-off: self-study saves money but can take 2-3 times longer and lead to mistakes that are expensive to fix. Courses accelerate the process and provide proven schemes but require financial investment.
Who this path suits and who it doesn't
Suits: DevOps engineers, SREs, developers who manage production clusters and want to automate scaling without surprises. If you are already familiar with Kubernetes basics (pods, deployments, services) and want to move to an advanced level — this is your path.
Doesn't suit: beginners who haven't yet launched a cluster and don't understand how requests/limits work. Autoscaling is the next step after mastering basic abstractions. Also, don't go here if you're looking only for theoretical knowledge without practice — the material requires application.
In conclusion, I want to say: autoscaling in Kubernetes is a powerful tool, but it requires understanding metrics, latency, and component interaction. If you're ready to master these mechanisms in practice, the course Kubernetes in Production provides a structured path with real case studies and feedback on your solutions. Start by configuring HPA in a test cluster, then move on to complex scenarios — and production will thank you.