Scenario
A network optimization rollout completes and later one node alone intermittently drops service traffic.
A kube-proxy migration to IPVS mostly works and one node blackholes service traffic because stale iptables rules from the old mode were never cleaned up there
The rollout looks healthy, yet one node still carries residual service rules from the previous proxy model and misroutes traffic.
What to check first
- Identify the primary failure signal in the The Cluster Migrated Proxy Modes and One Node Kept Yesterday's Rules scenario.
- Separate visible symptoms from the underlying technical dependency.
- Describe the safest recovery path and the follow-up prevention work.
Checking checklist
- Summarize the current impact and the last known change.
- Collect direct evidence from logs, runtime state, and configuration before changing anything.
- Separate immediate recovery from permanent prevention work.
Recovery and prevention
Audit the effective rules on the failing node before changing Service manifests or the CNI.
Questions worth viewing together
Community-field Kubernetes problem inspired by kube-proxy migration discussions where one node retained stale iptables rules after switching to IPVS. Mode migrations often fail asymmetrically on one node even when the daemonset rollout is green.
Teams often blame service selector drift when a node is still enforcing the old proxy rules.
Proxy-mode migrations should validate residual rule cleanup on every node.
Similar cases seen in the field