Kubernetes Production Checklist

Chapter 2 of 9 · 5 checks · 3 min

Requests, limits and capacity

Whether every container tells the scheduler what it needs, cannot take a node down with it, and has room to grow.

Show
  1. Every container sets CPU and memory requestsMust

    Why it matters and how to check

    The scheduler places Pods by their requests. A container with no requests can land on a node that is already full, and it is among the first evicted under memory pressure. Requests are also what HorizontalPodAutoscaler utilisation targets are measured against.

    To verify: list containers with no requests across the workload's namespace. The output should be empty. Then compare requests with real usage from kubectl top pods or your metrics system over a busy day; requests far above usage waste nodes, far below it causes noisy neighbours.

    Example

    Shell
    kubectl get pods -n checkout -o json | jq -r '.items[] | .metadata.name as $p
      | .spec.containers[] | select(.resources.requests == null) | "\($p) \(.name)"'
    kubectl top pods -n checkout --containers
  2. Every container has a memory limit equal to its requestMust

    Why it matters and how to check

    The security checklist asks for a memory limit on workloads with a limit equal to the request, because a memory limit above the request can expose the whole node to out-of-memory problems. With a matching limit, a leaking container is killed and restarted on its own, instead of the kernel picking victims across the node.

    To verify: check each container's resources.limits.memory against resources.requests.memory. Then look for recent OOM kills: kubectl get pods shows restarts, and kubectl describe pod shows OOMKilled as the last state. Repeated OOM kills mean the limit is too low for real usage, not that the limit should go.

  3. CPU limits are a deliberate choice, not a copied defaultShould

    Why it matters and how to check

    The security checklist says a CPU limit might be set on sensitive workloads, and warns that CPU limits throttle usage and can have unintended effects on autoscaling. A latency-sensitive service with a tight CPU limit can show slow responses at modest load while its CPU graph looks fine.

    To verify: for each container with a CPU limit, find the reason in the manifest or its review. Check throttling in your metrics (cAdvisor exposes container_cpu_cfs_throttled_periods_total); sustained throttling on a latency-sensitive service is a reason to raise or drop the limit.

  4. The namespace has a ResourceQuota and a LimitRangeShould

    Why it matters and how to check

    On a shared cluster, one runaway deployment or a mistyped replica count can consume every node. A ResourceQuota caps what a namespace can request in total; a LimitRange gives containers default requests and limits so nothing lands without them.

    To verify: run kubectl describe resourcequota and kubectl describe limitrange in the workload's namespace. The quota should leave headroom above the workload's peak (including the HPA maximum and a rollout surge), or the next scale-up will be rejected.

    Example

    Shell
    kubectl describe resourcequota -n checkout
    kubectl describe limitrange -n checkout
  5. The cluster has room for the workload at peak, plus a node failureShould

    Why it matters and how to check

    Replicas that cannot be scheduled are not replicas. Check capacity for the HPA maximum plus a rolling update's surge, with one node (or one zone) gone. The project's large cluster guidance designs for no more than 110 Pods per node and 5,000 nodes per cluster, so very dense node pools need checking against that too.

    To verify: look at kubectl describe nodes under "Allocated resources" for each node pool and add up free requestable CPU and memory. If you rely on node autoscaling, confirm the pool's maximum size and your cloud quotas allow the extra nodes.