Kubernetes Resource Requests and Limits That Don’t Throttle You

Almost every resources block I read was written once, by guesswork, and never touched again. That single copied stanza is why one team’s pods sit idle holding half the cluster hostage while another team’s service crawls under CPU throttling nobody can see.

Requests and limits are the two numbers that decide how your workload schedules and how it behaves under pressure. Get them from data and the cluster packs tightly and runs predictably. Get them from a template someone pasted in 2021 and you pay for it twice: in wasted capacity you are billed for, and in latency spikes you cannot explain because the throttling never shows up as an error. This is how to set them from actual usage, and which of the four numbers you should usually leave off entirely.

Requests schedule, limits constrain

The two words do genuinely different jobs, and conflating them is the root of most of the trouble. A request is what the scheduler reserves for your pod on a node: it is the number that decides where the pod lands and how much headroom the node keeps. A limit is the ceiling the kubelet enforces at runtime through the container’s cgroup. The scheduler never looks at limits. The runtime enforcement never looks at requests. So a pod can request 100m of CPU, be scheduled onto a busy node on that basis, and then be told it may burst to a 2000m limit that the node has no spare capacity to honour. The request got you a seat; the limit was a promise about a room that may already be full.

The combination you choose also assigns the pod a Quality of Service class, and that class decides who gets killed first when a node runs out of memory. If every container in the pod sets both a CPU and a memory limit, and each limit equals its request, the pod is Guaranteed — evicted last. Set some values but not that matched pair and you are Burstable — evicted in the middle, by how far you have overshot your request. Set nothing at all and you are BestEffort — the first thing thrown overboard when the node comes under memory pressure. For anything you actually care about, you want to be Guaranteed on memory. The reason why is the next section.

Memory is a hard kill, CPU is a soft throttle

This is the single most important asymmetry, and it is why the two resources want opposite treatment. Memory is incompressible: once a process has allocated a page, you cannot politely ask for it back. So when a container exceeds its memory limit, the kernel’s OOM killer terminates it and Kubernetes records the container’s last state as OOMKilled. There is no grace, no slowdown, just a dead process and a restart. Set a memory limit too low and you are not saving memory — you are arming a tripwire.

CPU is compressible, and the enforcement is entirely different. A CPU limit is translated into a Completely Fair Scheduler (CFS) quota — a slice of time the container may run within each roughly 100-millisecond period. Exceed it and the container is not killed; it is throttled, held off the CPU until the next period begins. That sounds harmless and often is not. A latency-sensitive service that briefly needs a burst of CPU to clear a request queue hits its quota, gets parked for the rest of the period, and every in-flight request stalls. Your p99 latency climbs and nothing in your dashboards says “error”, because from Kubernetes’ point of view nothing went wrong. This is why a low CPU limit is frequently worse than none at all.

Read the real numbers first

You cannot set these from first principles; you have to measure. The cheapest instrument is kubectl top, which reads from metrics-server — a lightweight component that scrapes the kubelet’s resource metrics and that you have to install, because it is not in a cluster by default. Confirm it is running, then watch a representative pod through a real load cycle, not a quiet Sunday.

# metrics-server present? (installed separately; not in-cluster by default)
kubectl get deployment metrics-server -n kube-system

# live CPU (cores) and memory (bytes) per pod, busiest first
kubectl top pod -n payments --sort-by=cpu

# per-container, so a sidecar's usage doesn't hide behind the app
kubectl top pod api-7f9c8b6d5-abcde -n payments --containers

# sample every 15s across a real load window to catch peaks, not the mean
while true; do
  date -u +%H:%M:%S
  kubectl top pod -n payments --no-headers
  sleep 15
done

kubectl top gives you a coarse live view; it is enough to size a first pass but not to see throttling. For that you need the container CPU throttling counters your monitoring stack already collects — the ratio of throttled periods to total periods for the container. If that ratio is meaningfully above zero on a service that is not maxing its request, the CPU limit is hurting you, not protecting you. Measuring what production actually does, rather than trusting the template, is the same discipline that makes the DORA delivery metrics worth instrumenting: the number on the dashboard beats the number you assumed.

A starting formula, then tune

With a few days of observed usage, here is a defensible starting point. Set the memory request to your observed peak plus roughly 25 per cent headroom, and set the memory limit equal to it. Equal memory request and limit gives you a Guaranteed-class pod on the dimension that kills, removes the OOM tripwire caused by a limit set below real usage, and makes scheduling honest. Set the CPU request to your typical steady-state usage — this is what earns the pod its fair share and its scheduling decision. And on CPU limits: for most latency-sensitive services, leave the limit off. A pod with a CPU request but no CPU limit still gets its guaranteed share under contention, and is free to burst into idle capacity instead of being throttled against an arbitrary ceiling.

resources:
  requests:
    # steady-state CPU: the pod's fair share under contention
    cpu: "250m"
    # observed peak memory + ~25% headroom
    memory: "512Mi"
  limits:
    # memory limit == request: Guaranteed on the dimension that kills
    memory: "512Mi"
    # no cpu limit: burst into idle capacity instead of being throttled.
    # add one only for genuinely noisy neighbours or hard tenancy caps.

The honest exceptions: set a CPU limit when you have a genuinely antisocial workload that will starve its neighbours, or a hard multi-tenant billing boundary you must enforce — and if you do, set it well above the request so the ceiling is a guard rail, not a leash. Everything else runs better without one.

Two operational points close this out. First, these values are not write-once. Usage drifts as traffic and code change, so re-read kubectl top and the throttling counters on a schedule and adjust — the same principle behind allocating cluster cost back to the teams that cause it, because a pod requesting four times what it uses is a bill someone is paying for nothing. Second, stop the guessed template at the door: a pre-commit and CI check that fails any workload missing a memory request-equals-limit pair costs an afternoon and ends the entire class of problem. The numbers that throttle you are always the ones nobody measured.