Runner Hygiene
143 incident problems about Runner Hygiene. Start with the reviewed ones.
Read first
When the CI cache restores the wrong dependency graphAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when package.json, the lockfile, and artifacts no longer agree even after a cache hit.CI/CD3 min readWhen GitHub Actions succeeds but only the rollout failsAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when the build logs look fine but the failure surfaces only on the target deploy cluster or runtime.CI/CD3 min readThe checking order when GitHub Actions succeeds but only the deploy failsAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when the workflow is green but the failure happens only in the rollout, promotion, or health check stage.CI/CD3 min read
Recommended problems
Reviewed problems first, then problems with detailed scenarios.
All problems (143)
Self-hosted runner: No space left on device in the middle of buildsSelf-hosted runner: No space left on device in the middle of builds is a hands-on troubleshooting drill. Find what fills a long-lived runner's disk. GitHub Runner Hygiene needs to be checked by narrowing scope, recent change, and the current live signal before rollback. 실무에서는...ReviewedCI/CDBeginner3 minFreeCICD-1614A GitLab runner job stays pendingOne GitLab job stays pending after tag cleanup.CI/CDBeginner6 minFreeCICD-1655A Jenkins credential exists and one pipeline still failsOne Jenkins stage still fails after a credential alias cleanup.CI/CDBeginner7 minFreeCICD-1635A Jenkins credential is valid and one stage still failsOne Jenkins stage still fails after a credential rename.CI/CDBeginner7 minFreeCICD-1645A Jenkins job is online and one credential step still failsOne Jenkins credential step still fails after cleanup.CI/CDBeginner7 minFreeCICD-1665A Jenkins secret exists and one pipeline still stopsOne Jenkins pipeline still stops after a credential alias cleanup.CI/CDBeginner7 minFreeK8S-1569An IRSA token file rotates and one DaemonSet still loses AWS accessA DaemonSet loses AWS access only on nodes where an init copy path still runs.KubernetesAdvanced10 minProCICD-1487A rootless GitHub runner survives reboots and loses Docker accessSelf-hosted runners fail only after host reboots on hardened images with rootless Docker.CI/CDAdvanced11 minProCICD-1558A self-hosted runner image adds rootless Docker and one job still failsOnly restored runners fail image builds after moving to rootless Docker.CI/CDAdvanced11 minProK8S-1559An IRSA migration completes and one DaemonSet still loses cloud accessA DaemonSet loses AWS access only on older nodes after IRSA migration.KubernetesAdvanced11 minProCICD-1408A CodeBuild local cache speeds most jobs and one privileged image build...A CodeBuild local cache speeds most jobs and one privileged image build... focuses on Artifact Promotion and asks the reader to isolate the key signal in AWS. Local cache reuse can inherit ownership assumptions from the most privil...CI/CDAdvanced13 minProLINUX-1392A rootless container runtime works on one host and fails on anotherA rootless container runtime works on one host and fails on another focuses on Identity And Access and asks the reader to isolate the key signal in Linux. User namespace failures often come from host identity allocation conflicts rather than f...LinuxAdvanced14 minProLINUX-1336A rootless Podman or Docker service survives upgrades and failsA rootless Podman or Docker service survives upgrades and fails focuses on Runner Hygiene and asks the reader to isolate the key signal in Linux. Rootless startup is a contract between systemd user units and the user lifecycle itself.LinuxAdvanced14 minProCICD-1356A BuildKit cache import appears healthy and later every merge build goes coldA BuildKit cache import appears healthy and later every merge build goes cold focuses on Artifact Promotion and asks the reader to isolate the key signal in docker. A present cache tag does not guarantee it still points to a cache artifact.CI/CDAdvanced15 minProCICD-1366A Buildx multiarch release publishes a manifest list and one architecture...A Buildx multiarch release publishes a manifest list and one architecture... focuses on Artifact Promotion and asks the reader to isolate the key signal in docker. Temporary-looking tags can still be part of the publish chain for...CI/CDAdvanced15 minProCICD-1376A Buildx multiarch release shows green and one architecture tag is absentA Buildx multiarch release shows green and one architecture tag is absent focuses on Artifact Promotion and asks the reader to isolate the key signal in docker. Temporary refs can still be live dependencies in the publish chain of...CI/CDAdvanced15 minProCICD-1320A deploy stage reuses the wrong Helm valuesA self-hosted runner deploys reliably until cross-environment contamination appears after a compose-based local prep step was added.CI/CDAdvanced15 minProCICD-1410A remote buildx cluster handles amd64 and arm64 and one node keeps producing...A remote buildx cluster handles amd64 and arm64 and one node keeps... focuses on Artifact Promotion and asks the reader to isolate the key signal in docker. Remote builder pools can hide architecture-specific emulation drift on a single node b...CI/CDAdvanced15 minProCICD-1420A remote buildx farm stays green and reproducibility breaksA remote buildx farm stays green and reproducibility breaks focuses on Runner Hygiene and asks the reader to isolate the key signal in docker. Reproducibility failures often come from node-level toolchain skew hidden inside an otherwise healt...CI/CDAdvanced15 minProLINUX-1372A rootless Docker service works until rebootA rootless Docker service works until reboot focuses on Runner Hygiene and asks the reader to isolate the key signal in docker. Rootless failures after reboot often come from XDG runtime path lifecycle drift, not from the...LinuxAdvanced15 minPro