Runtime Configuration
147 incident problems about Runtime Configuration. Start with the reviewed ones.
Read first
How to split Linux failures by systemd, permission, and filesystem signalsAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when a service fails but you need to isolate whether the cause is a systemd unit, permission, inode, mount, or process.Linux3 min readWhen sudo works in the shell but fails under automationAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when only automation jobs fail because of TTY, sudoers, environment reset, or service user differences.Linux3 min readWhen permissions are fixed but only new directories are created with the wrong groupAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when existing file permissions are correct but group inheritance breaks for newly created files and directories.Linux3 min read
Recommended problems
Reviewed problems first, then problems with detailed scenarios.
All problems (147)
LINUX-1377A cron-triggered deploy script works manually and fails overnightA maintenance script is promoted to cron and later the scheduled run fails with a binary not found error no one can reproduce manually.ReviewedLinuxBeginner10 minFreeSECURITY-1398An Elastic detection rule misses admin abuseAn Elastic detection rule misses admin abuse focuses on incident-response and asks the reader to isolate the key signal in Opensearch. Detection quality can degrade when schema evolution changes relationships between actor and target fie...SecurityAdvanced13 minProSECURITY-1388An Elastic detection rule still misses one attacker pathAn Elastic detection rule still misses one attacker path focuses on incident-response and asks the reader to isolate the key signal in Opensearch. Detection drift often follows schema normalization changes rather than event loss.SecurityAdvanced13 minProSECURITY-1371A Grafana SSO login succeeds and users land as ViewersAn IdP claim cleanup lands and later every Grafana login works but the expected team roles stop being assigned.SecurityAdvanced15 minProSECURITY-1315An AWS STS AssumeRoleWithWebIdentity call works from one workload and fails...An AWS STS AssumeRoleWithWebIdentity call works from one workload and fails... focuses on Identity And Access and asks the reader to isolate the key signal in AWS. STS web identity failures often come from audience mismatch even when issuer and subje...SecurityAdvanced15 minProSECURITY-1397A fail2ban jail matches the right event and bans nothingA fail2ban jail matches the right event and bans nothing focuses on incident-response and asks the reader to isolate the key signal in Linux. Security automation often breaks on log-shape migrations that preserve human reada...SecurityIntermediate10 minProSECURITY-1387A fail2ban rule triggers on the right log lines and never blocks the clientA fail2ban rule triggers on the right log lines and never blocks the client focuses on incident-response and asks the reader to isolate the key signal in Linux. Detection and enforcement can drift apart when one consumes par...SecurityIntermediate10 minProSECURITY-1382A Grafana auth proxy setup works and admin privileges disappearA Grafana auth proxy setup works and admin privileges disappear focuses on Identity And Access and asks the reader to isolate the key signal in NGINX. Identity regressions can come from harmless-looking canonicalization changes like lowercasing...SecurityIntermediate11 minProSECURITY-1392A Grafana team sync still works and one admin loses elevated accessA Grafana team sync still works and one admin loses elevated access focuses on Identity And Access and asks the reader to isolate the key signal in grafana. SSO regressions often come from type changes in claims, not just from missing values.SecurityIntermediate11 minProSECURITY-1385A Prometheus exporter secret is rotated and alerting still uses the old...A Prometheus exporter secret is rotated and alerting still uses the old... focuses on Identity And Access and asks the reader to isolate the key signal in Kubernetes. Hot-reload automation can succeed technically while watching the...SecurityIntermediate11 minProLINUX-1362A rootless Docker host survives reboot and later loses outbound connectivityA rootless Docker host survives reboot and later loses outbound connectivity focuses on network-segmentation and asks the reader to isolate the key signal in Linux. Rootless restore issues after reboot often come from helper lif...LinuxAdvanced14 minProLINUX-1323A rootless Docker service works until rebootA node image refresh is followed by rootless container services staying down after reboot until someone logs in manually.LinuxAdvanced14 minProLINUX-1336A rootless Podman or Docker service survives upgrades and failsA rootless Podman or Docker service survives upgrades and fails focuses on Runner Hygiene and asks the reader to isolate the key signal in Linux. Rootless startup is a contract between systemd user units and the user lifecycle itself.LinuxAdvanced14 minProSECURITY-1322A Vault policy looks permissive and KV reads still failA service token is updated and later can enumerate secrets but cannot actually read the intended values from Vault.SecurityAdvanced14 minProCICD-1334A Compose deployment deletes generated migration assetsA local compose workflow looks fine until one container starts missing generated migration or asset files after a directory is mounted for live editing.CI/CDAdvanced15 minProK8S-1327A DaemonSet appears healthy and one node family never gets the agentA fleet adds a new node pool image and later one DaemonSet quietly skips those nodes while remaining healthy elsewhere.KubernetesAdvanced15 minProK8S-1357A kubeadm worker join passes and pods cannot pull imagesA kubeadm worker join passes and pods cannot pull images focuses on cluster-maintenance and asks the reader to isolate the key signal in Kubernetes. Successful node join does not prove the node runtime inherited the same registry assumptio...KubernetesAdvanced15 minProLINUX-1327A logrotate change saves disk and starts truncating live uploadsA high-volume service rotates logs successfully and later upload or replay jobs show gaps that correlate with rotation windows.LinuxAdvanced15 minProK8S-1333A Loki ingestion path stays healthy and dashboards still miss one tenantA Loki ingestion path stays healthy and dashboards still miss one tenant focuses on Identity And Access and asks the reader to isolate the key signal in grafana. Multi-tenant observability bugs often come from query path identity, not from ing...KubernetesAdvanced15 minProK8S-1325A projected service account token looks valid and a custom API aggregation...A projected service account token looks valid and a custom API aggregation... focuses on Identity And Access and asks the reader to isolate the key signal in Kubernetes. In Kubernetes auth, a valid token can still be unusable if its aud...KubernetesAdvanced15 minPro