← Problem Library
K8s L6 K8S 1351 · 16 min

A Cilium upgrade keeps policy green and one namespace loses east-west traffic

The rollout completes, yet one node sees endpoints through outdated map state that no longer matches the new pod range.

K8sPlatform ReliabilityLevel 6Pro16 min
Scenario

A cluster networking upgrade is green overall and later workloads on one node alone cannot reach peers after the pod CIDR was reallocated.

What to check first
  • Identify the primary failure signal in the The Upgrade Finished and One Node Kept Enforcing Yesterday's Pod Map scenario.
  • Separate visible symptoms from the underlying technical dependency.
  • Describe the safest recovery path and the follow-up prevention work.
Checking checklist
  1. Summarize the current impact and the last known change.
  2. Collect direct evidence from logs, runtime state, and configuration before changing anything.
  3. Separate immediate recovery from permanent prevention work.
Recovery and prevention

Inspect node-local BPF map state before broadening policy or recycling entire namespaces.

Questions worth viewing together
What should you verify first when A Cilium upgrade keeps policy green and one namespace loses east-west traffic appears?

Community-field Kubernetes problem inspired by Cilium community cases where stale BPF map state kept enforcing the old pod CIDR after upgrade. Node-local dataplane state can outlive a healthy-looking Cilium rollout.

What usually causes A Cilium upgrade keeps policy green and one namespace loses east-west traffic in production?

Teams often blame policy YAML when only one node is still applying a previous endpoint map.

What should you document after resolving A Cilium upgrade keeps policy green and one namespace loses east-west traffic?

Dataplane cleanup verification is critical after upgrades that also change pod addressing assumptions.