Scenario
Operators trust node disk dashboards and still get unexpected pod evictions after a burst of image pulls or log growth.
A kubelet eviction storm begins (eviction-threshold-hit-on-imagefs-not-dashboard-rootfs)
Dashboards looked safe, yet evictions start because the metric everyone watched was not the same filesystem the kubelet used for image pressure decisions.
What to check first
- Identify the primary failure signal in the The Node Was Full on the Filesystem Kubelet Cared About and the Dashboard Measured Another One scenario.
- Separate visible symptoms from the underlying technical dependency.
- Describe the safest recovery path and the follow-up prevention work.
Checking checklist
- Summarize the current impact and the last known change.
- Collect direct evidence from logs, runtime state, and configuration before changing anything.
- Separate immediate recovery from permanent prevention work.
Recovery and prevention
Map kubelet eviction signals to the actual filesystem source before tuning Prometheus alerts.
Questions worth viewing together
Community-field Kubernetes problem inspired by Kubernetes and Grafana community patterns where evictions were triggered by imagefs pressure that dashboa... Storage health dashboards can be misleading if they do not align with kubelet's imagefs and rootfs distinctions.
Teams often blame kubelet aggressiveness when the monitored filesystem was simply not the one under pressure.
Node storage observability should mirror kubelet eviction dimensions, not only generic disk usage.
Similar cases seen in the field