Cluster Maintenance
81 incident problems about Cluster Maintenance. Start with the reviewed ones.
Read first
When the ConfigMap changed but the Pod keeps using the old valuesAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when values don't refresh after a ConfigMap update because of envFrom, subPath, volume projection, or rollout trigger differences.Kubernetes3 min readWhen CoreDNS is Running but only DNS lookups failAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when the CoreDNS Pod looks healthy but failures occur in the kube-dns path, upstream, node-local-dns, or policy.Kubernetes3 min readWhat to check first when CrashLoopBackOff appearsAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when a Pod restarts repeatedly and the real cause is left in the previous logs and events.Kubernetes3 min read
Recommended problems
Reviewed problems first, then problems with detailed scenarios.
All problems (81)
A node goes NotReady after a rebootAfter an OS security update and reboot, one node stays NotReady. SSH works, but kubelet keeps failing to start.ReviewedKubernetesIntermediate16 minFreeSECURITY-1369An Ubuntu bastion patch run completes and SSH trust breaksRoutine patching succeeds and later automated SSH clients start rejecting the bastion even though no access policy was intentionally changed.SecurityAdvanced13 minProSECURITY-1359An Ubuntu bastion patch window completes and SSH trust breaksAn Ubuntu bastion patch window completes and SSH trust breaks focuses on Identity And Access and asks the reader to isolate the key signal in Linux. Patch windows can rotate machine identity, not just patch packages.SecurityAdvanced14 minProSECURITY-1349An Ubuntu bastion patch window succeeds and SSH trust breaksRoutine patching is completed and later automated SSH clients reject the bastion despite no intended access control changes.SecurityAdvanced14 minProSECURITY-1327An admission policy rollout breaks only one namespaceAn admission policy rollout breaks only one namespace focuses on Identity And Access and asks the reader to isolate the key signal in Kubernetes. Distributed trust injection systems rarely converge everywhere at the same instant.SecurityAdvanced15 minProNETWORK-1397An MLAG pair carries normal traffic and a disaster-recovery test failsAn MLAG pair carries normal traffic and a disaster-recovery test fails focuses on cluster-maintenance and asks the reader to isolate the key signal in Cisco. Recovery paths can drift silently because they are invisible during steady-state operation.NetworkAdvanced15 minProSECURITY-1318A Kubernetes admission webhook serves valid TLS and requests still failA cluster rotates webhook serving certs in place and later admission failures appear even though the pod and service remain healthy.SecurityAdvanced16 minProSECURITY-1316An SSH CA rollout signs host certificates correctly and engineers still get...An SSH CA rollout signs host certificates correctly and engineers still get... focuses on Identity And Access and asks the reader to isolate the key signal in Linux. SSH CA incidents often hide in stale host cert issuance rather t...SecurityIntermediate13 minProNETWORK-1369A pfSense HA pair syncs cleanly and one package VPN remains brokenAn HA firewall upgrade looks successful and later one package-managed VPN fails only after failover to the secondary node.NetworkAdvanced14 minProNETWORK-1359A pfSense HA sync reports success and one package VPN never returns on the...A pfSense HA sync reports success and one package VPN never returns on the... focuses on Identity And Access and asks the reader to isolate the key signal in netgate. HA success on the base firewall does not guarantee package-owned...NetworkAdvanced14 minProLINUX-1323A rootless Docker service works until rebootA node image refresh is followed by rootless container services staying down after reboot until someone logs in manually.LinuxAdvanced14 minProSECURITY-1335An Ubuntu unattended upgrade secures packages and leaves one bastion inaccessibleA hardened bastion receives unattended upgrades and later downstream systems that pin host identity stop trusting it.SecurityIntermediate14 minProLINUX-1340A boot upgrade leaves one host without DNSA boot upgrade leaves one host without DNS focuses on cluster-maintenance and asks the reader to isolate the key signal in Linux. DNS failures after upgrades can come from two healthy components disagreeing on which interface is primary.LinuxAdvanced15 minProK8S-1361A Cilium rollout leaves one namespace unreachableA platform team standardizes labels and later one namespace loses east-west traffic only on a subset of nodes.KubernetesAdvanced15 minProK8S-1371A Cilium upgrade leaves one namespace unreachableA label cleanup is rolled out and later one namespace loses traffic only on a subset of nodes.KubernetesAdvanced15 minProK8S-1327A DaemonSet appears healthy and one node family never gets the agentA fleet adds a new node pool image and later one DaemonSet quietly skips those nodes while remaining healthy elsewhere.KubernetesAdvanced15 minProK8S-1392A Gatekeeper policy looks unchanged and one namespace starts failing admissionA Gatekeeper policy looks unchanged and one namespace starts failing admission focuses on Identity And Access and asks the reader to isolate the key signal in Kubernetes. Admission drift can hide in CRD conversion...KubernetesAdvanced15 minProLINUX-1334A kernel update fixes a driver issue and a custom nftables service still failsA host kernel upgrade is followed by sporadic early boot network exposure or missing firewall enforcement until services settle.LinuxAdvanced15 minProK8S-1399A kube-proxy replacement migration works on worker nodes and one control...A kube-proxy replacement migration works on worker nodes and one control... focuses on cluster-maintenance and asks the reader to isolate the key signal in Kubernetes. Pod path success does not prove host-network exception beha...KubernetesAdvanced15 minProK8S-1357A kubeadm worker join passes and pods cannot pull imagesA kubeadm worker join passes and pods cannot pull images focuses on cluster-maintenance and asks the reader to isolate the key signal in Kubernetes. Successful node join does not prove the node runtime inherited the same registry assumptio...KubernetesAdvanced15 minPro