Linux Storage and Filesystems
64 incident problems about Linux Storage and Filesystems. Start with the reviewed ones.
Read first
How to split Linux failures by systemd, permission, and filesystem signalsAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when a service fails but you need to isolate whether the cause is a systemd unit, permission, inode, mount, or process.Linux3 min readWhen sudo works in the shell but fails under automationAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when only automation jobs fail because of TTY, sudoers, environment reset, or service user differences.Linux3 min readWhen permissions are fixed but only new directories are created with the wrong groupAn InfraTree guide that lays out the first signals to check, the CLI verification order, common misdiagnoses, and a safe recovery path when existing file permissions are correct but group inheritance breaks for newly created files and directories.Linux3 min read
Recommended problems
Reviewed problems first, then problems with detailed scenarios.
LINUX-001An inode problem where files cannot be created even though disk space is freeA problem for identifying and organizing inode exhaustion, which is easy to miss when looking only at df output.ReviewedLinuxBeginner16 minFreeLINUX-033Deleted log file keeps disk fullOperators remove the giant log file, but the filesystem usage never drops because the service keeps writing to an already deleted file descriptor.ReviewedLinuxIntermediate18 minFreeNo space left on device: the disk is full and files cannot be writtenNo space left on device: the disk is full and files cannot be written is a hands-on troubleshooting drill. Use df and du to find what filled the disk. Linux Storage and Filesystems needs to be checked by narrowing scope, recent change, and the current live signal before rollba...ReviewedLinuxBeginner3 minFreeLINUX-002Narrowing a /var partition surge with du and lsofA hands-on disk analysis scenario that extends into a file-handle problem where space is not reclaimed after deletion.LinuxIntermediate24 minFree
All problems (64)
No space left on device: the disk is full and files cannot be writtenNo space left on device: the disk is full and files cannot be written is a hands-on troubleshooting drill. Use df and du to find what filled the disk. Linux Storage and Filesystems needs to be checked by narrowing scope, recent change, and the current live signal before rollba...ReviewedLinuxBeginner3 minFreeLINUX-001An inode problem where files cannot be created even though disk space is freeA problem for identifying and organizing inode exhaustion, which is easy to miss when looking only at df output.ReviewedLinuxBeginner16 minFreeLINUX-033Deleted log file keeps disk fullOperators remove the giant log file, but the filesystem usage never drops because the service keeps writing to an already deleted file descriptor.ReviewedLinuxIntermediate18 minFreeLINUX-002Narrowing a /var partition surge with du and lsofA hands-on disk analysis scenario that extends into a file-handle problem where space is not reclaimed after deletion.LinuxIntermediate24 minFreeLINUX-091Tmpfs mount survives the deploy but vanishes after rebootThe application worked during the change window, then failed after restart because the critical mount existed only as a temporary runtime action.LinuxIntermediate14 minFreeLINUX-012A tmp directory cleanup policy that intermittently breaks only upload processingA tmp directory cleanup policy that intermittently breaks only upload... is a hands-on troubleshooting drill. A situation where a batch cleanup task collides with the application's temp files, breaking only a specific feature. Azure Linux Storage and Filesystems needs to be ch...LinuxIntermediate20 minFreeLINUX-074Boot stalls on a dead NFS mountThe host itself is healthy, but boot takes far too long because a remote mount is treated like a local dependency and blocks the sequence.LinuxBeginner13 minFreeLINUX-004An open log-file problem where space is not freed even after logrotate runsCovers a common operational trap where swapping the log file without restarting the service still does not free space.LinuxBeginner18 minFreeLINUX-052Application helper failsThe binary exists and permissions look correct, but the helper script or unpacked runtime tool cannot execute because the temporary mount has a stricter policy than the application expects.LinuxBeginner15 minFreeLINUX-037Tmpfs exhaustion in /run prevents a daemon socket from startingDisk usage on persistent filesystems looks normal, but the service socket cannot be created because the tmpfs used for runtime state is full.LinuxBeginner15 minFreeLINUX-034LVM extend succeeds but XFS filesystem size never changesThe volume group and logical volume show the new capacity, yet the mounted filesystem still reports the old size because the final grow step never ran.LinuxBeginner16 minFreeLINUX-100Filesystem is remounted read-only after transient storage errors but the app hides it as generic 500sThe application logs are noisy, yet the real cause is the kernel remounting the filesystem read-only after I/O faults the app never surfaces clearly.LinuxAdvanced18 minProLINUX-077Rsync restore with numeric-ids shifts file ownershipThe backup looks intact, but the restored application loses access because numeric UID preservation no longer matches the target system account layout.LinuxAdvanced20 minProLINUX-068tmpfs-backed runtime path fillsThe service outage looks like a simple restart loop, but the deeper issue is runtime storage pressure from large repeated coredumps on a limited tmpfs path.LinuxAdvanced20 minProLINUX-085XFS metadata corruption warning is stale but the on-call plans a destructive repair on the mounted volumeAn old alert resurfaces during an incident, and the real risk now is operator action because the filesystem is still mounted and serving production writes.LinuxAdvanced22 minProLINUX-062Missing bind mount at boot makes rsync --delete clean the host path instead of the volumeThe sync command is valid, but the intended mount point never appeared and the cleanup step now targets the underlying directory on the root filesystem.LinuxAdvanced24 minProLINUX-053rsyslog disk queue fills /var after the remote collector becomes unreachableCentral logging is configured correctly for healthy periods, but when the collector disappears the buffered queue grows until the local partition reaches pressure.LinuxAdvanced24 minProLINUX-009An rsync deploy filling the root partitionA scenario that narrows down the root cause, centered on improving the deploy tool's temp-file strategy and the partition policy, in the situation of an rsync deploy filling the root partition because of temp files.LinuxAdvanced26 minProLINUX-057RAID rebuild targets the wrong device after a degraded mdadm recoveryThe array is degraded but recoverable, and the biggest risk now is re-adding or marking the wrong member while interpreting device names under pressure.LinuxAdvanced27 minProLINUX-098Rsync transfer is complete but hard links were expanded and the backup target fills twice as fastThe restore path works, yet storage usage explodes because the transfer options did not preserve link semantics the source tree relied on.LinuxAdvanced16 minPro