Symptom384 problems· 9 reviewed

Rollout Stuck

384 incident problems that show up as “Rollout Stuck”.

All problems (384)

K8S-139HPA scale-down stabilization holds extra replicas during a rollout, and the Deployment budget never frees enough capacity for the next revisionNothing is obviously broken, but overlapping controller safety windows create a deadlock on available resources.KubernetesIntermediate16 minProNETWORK-131One switch in the ring has the wrong MST region name, and the topology becomes a boundary only after a reboot reloads the saved configThe live network was temporarily aligned, but the persisted spanning-tree identity says otherwise.NetworkIntermediate16 minProCICD-154The monorepo release graph marks a shared schema package as optional, and one dependent service never rebuilds after a contract changeDependency topology is encoded in automation, but one edge was weakened until the next schema evolution exposed it.CI/CDIntermediate16 minProK8S-318A ConfigMap hot reload works for the main container while a sidecar cached the previous schema and never reopens the mounted files during a failover rehearsalThe pod has new config, but not every process inside the pod has new understanding. Normal traffic masked the issue until the standby or alternate path became active under rehearsal conditions.KubernetesIntermediate17 minProK8S-198A conservative rollout budget still loses availability when cluster pressure starts evicting old pods during a failover rehearsalDeployment math is correct, yet platform pressure changes the effective availability story. Normal traffic masked the issue until the standby or alternate path became active under rehearsal conditions.KubernetesIntermediate17 minProCICD-336A deployment lock is cleared manually, but downstream cleanup logic still assumes the old lock owner is active and deletes the wrong environment during a failover rehearsalThe lock seems gone, yet ownership metadata from the previous run still drives destructive automation. Normal traffic masked the issue until the standby or alternate path became active under rehearsal conditions.CI/CDIntermediate17 minProCICD-246A monorepo release train publishes service code before the shared contract package reaches the private registry mirror during a failover rehearsalApplication code moves ahead of the contract library and consumers resolve inconsistent dependency sets. Normal traffic masked the issue until the standby or alternate path became active under rehearsal conditions.CI/CDIntermediate17 minProCICD-174A multi-architecture promotion pipeline validates only one platform manifest during a failover rehearsalThe tag exists and one worker pool succeeds, yet another architecture never receives the complete promoted artifact. Normal traffic masked the issue until the standby or alternate path became active under rehearsal conditions.CI/CDIntermediate17 minProCICD-497A pipeline pins the runner image (Rollout Stuck)A pipeline pins the runner image (Rollout Stuck) focuses on ci-cd-workflow-debugging and asks the reader to isolate Rollout Stuck in GitHub. 실무에서는 ci-cd-workflow-debugging 문제를 볼 때 실패 단계만 보지 말고 최근 변경, 이미지 태그, 시크릿 주입, 롤백 가능 여부를 먼저 함께 확인하는 편이 빠릅니다.CI/CDIntermediate17 minProCICD-499A pipeline pins the runner image (Rollout Stuck)A pipeline pins the runner image (Rollout Stuck) focuses on ci-cd-workflow-debugging and asks the reader to isolate Rollout Stuck in GitHub. 실무에서는 ci-cd-workflow-debugging 문제를 볼 때 실패 단계만 보지 말고 최근 변경, 이미지 태그, 시크릿 주입, 롤백 가능 여부를 먼저 함께 확인하는 편이 빠릅니다.CI/CDIntermediate17 minProCICD-501A pipeline pins the runner image (Rollout Stuck)A pipeline pins the runner image (Rollout Stuck) focuses on ci-cd-workflow-debugging and asks the reader to isolate Rollout Stuck in GitHub. 실무에서는 ci-cd-workflow-debugging 문제를 볼 때 실패 단계만 보지 말고 최근 변경, 이미지 태그, 시크릿 주입, 롤백 가능 여부를 먼저 함께 확인하는 편이 빠릅니다.CI/CDIntermediate17 minProK8S-288A PodDisruptionBudget is satisfied in aggregate while one topology domain has no remaining healthy replicas for a targeted drain during a failover rehearsalThe global count looks safe, but the local maintenance blast radius is not. Normal traffic masked the issue until the standby or alternate path became active under rehearsal conditions.KubernetesIntermediate17 minProK8S-127A very generous startup probe masks a crash loop during rollout, and autoscaling expands the broken ReplicaSet before anyone noticesThe app never becomes truly healthy, but the rollout budget is consumed on pods that are still within the startup grace window.KubernetesIntermediate17 minProCICD-143The deployment job waits for image replication, but only the amd64 manifest arrives before the timeout and mixed-arch clusters fail progressivelyThe release gate checks one success condition, yet the fleet needs a broader replication contract than the workflow currently verifies.CI/CDIntermediate17 minProK8S-142The service mesh sidecar starts after the application, and the startup probe passes once before all outbound traffic begins failingInitialization timing hid the networking dependency because the first health check ran before the sidecar took control.KubernetesIntermediate17 minProCICD-515A broken-glass deploy skips manual approvalA broken-glass deploy skips manual approval focuses on ci-cd-release-safety and asks the reader to isolate 404 and Rewrite Mismatch. 실무에서는 ci-cd-release-safety 문제를 볼 때 실패 단계만 보지 말고 최근 변경, 이미지 태그, 시크릿 주입, 롤백 가능 여부를 먼저 함께 확인하는 편이 빠릅니다.CI/CDIntermediate18 minProCICD-517A broken-glass deploy skips manual approvalA broken-glass deploy skips manual approval focuses on ci-cd-release-safety and asks the reader to isolate 404 and Rewrite Mismatch. 실무에서는 ci-cd-release-safety 문제를 볼 때 실패 단계만 보지 말고 최근 변경, 이미지 태그, 시크릿 주입, 롤백 가능 여부를 먼저 함께 확인하는 편이 빠릅니다.CI/CDIntermediate18 minProCICD-519A broken-glass deploy skips manual approvalA broken-glass deploy skips manual approval focuses on ci-cd-release-safety and asks the reader to isolate 404 and Rewrite Mismatch. 실무에서는 ci-cd-release-safety 문제를 볼 때 실패 단계만 보지 말고 최근 변경, 이미지 태그, 시크릿 주입, 롤백 가능 여부를 먼저 함께 확인하는 편이 빠릅니다.CI/CDIntermediate18 minProK8S-168A sidecar takes over networking after the application already passed its first startup probe during a failover rehearsalHealth looks good once, but the final runtime path is not the one the probe validated. Normal traffic masked the issue until the standby or alternate path became active under rehearsal conditions.KubernetesIntermediate18 minProCICD-560A pipeline pins the runner image (Rollout Stuck)A pipeline pins the runner image (Rollout Stuck) focuses on ci-cd-workflow-debugging and asks the reader to isolate Rollout Stuck in GitHub. 실무에서는 ci-cd-workflow-debugging 문제를 볼 때 실패 단계만 보지 말고 최근 변경, 이미지 태그, 시크릿 주입, 롤백 가능 여부를 먼저 함께 확인하는 편이 빠릅니다.CI/CDIntermediate19 minPro