← Problem Library
K8s L6 K8S 018 · 29 min

GPU workloads stay Pending even though the cluster autoscaler is on

A situation where the autoscaler is working but the workload waits because of a special node-group condition.

K8sPlatform ReliabilityLevel 6Pro29 min
Scenario

A scenario that narrows down the root cause, centered on reviewing the special-resource node group together with the autoscale criteria, in the situation of GPU workloads staying Pending even though the cluster autoscaler is on.

What to check first
  • Identify the primary failure signal in the Autoscaler Capacity scenario.
  • Separate visible symptoms from the underlying technical dependency.
  • Describe the safest recovery path and the follow-up prevention work.
Checking checklist
  1. Summarize the current impact and the last known change.
  2. Collect direct evidence from logs, runtime state, and configuration before changing anything.
  3. Separate immediate recovery from permanent prevention work.
Recovery and prevention

Choose the smallest safe recovery action first, then record the prevention work that reduces repeat incidents.

Questions worth viewing together
Should I translate commands, logs, or product names?

No. Keep commands, logs, file names, APIs, and product names unchanged, then explain the reasoning in the selected UI language.

What should the answer focus on?

State the root cause, the evidence that supports it, and the safest recovery direction.

Why can the scenario remain in the original language?

The source scenario is treated as an incident artifact. Guidance, checklist, hints, and explanations can be localized around it.