The disk is full: finding what is eating it and clearing it safely

When a disk fills up the symptoms look scattered: the site returns 500, the database cannot write, the game server stops saving, and sometimes SSH will not even let you in. The cause is one single thing and the diagnosis takes two commands. The hard part is the cleanup: deleting the wrong file costs more than the full disk did.

1. Is it space, or inodes?

df -h
df -i

The first command shows capacity, the second the inode count. They run out separately and both produce the same error.

• If usage is pinned at 100% in df -h, the problem is capacity.
• If the inode ratio is full in df -i, there is visible free space but no new file can be created. What exhausts inodes is usually millions of tiny files: session files, cache entries, queue files.

Make sure you are looking at the right filesystem. /var or /home can be on a separate partition; the root partition can be comfortable while that one is full.

2. Find what is eating it, from the top down

du -xh --max-depth=1 / | sort -h

The -x flag keeps the scan inside one filesystem; /proc, /sys and mounted disks stay out of the count, which they must, or the result is meaningless. Go into the largest directory at the bottom of the output and repeat the same command there. Three or four steps find the source.

To look for individual large files:

find /var -xdev -type f -size +500M -exec ls -lh {} +

3. If you deleted something and no space appeared

The classic trap: you deleted a huge log file and df still shows the disk full. The reason is that space is not released while a process still holds the file open.

lsof +L1

That command lists files that have been deleted but are still open. The fix is to restart the service holding the file. To avoid the trap in the first place, empty an active log file instead of deleting it — truncate -s 0 file keeps the file descriptor valid, so the space is released immediately.

4. What is safe to delete first

Start the cleanup with this list. Everything here is either reproducible or already temporary data.

• The package manager cache. On the Debian and Ubuntu family apt clean removes downloaded installation packages; all of them are re-downloaded when needed.
• Old system journals. journalctl --vacuum-time=7d or journalctl --vacuum-size=200M drops only old records and does not affect running services.
• Rotated old logs. Archived logs with .gz and .1 suffixes can be deleted safely; touch the active log file with the method in step 3 instead.
• Unused Docker layers. First see how much space they take with docker system df, then remove only untagged images with docker image prune.
• Installation archives and old version directories you downloaded yourself. Usually the biggest and easiest win.

5. What not to touch

This list is short but expensive.

• The database directory. No file under /var/lib/mysql is deleted by hand — binary logs included. If binlogs really are eating space, the answer is the PURGE BINARY LOGS statement, not deleting files. The permanent settings: MySQL performance tuning
• Your backups. Deleting backups to free space makes the next incident unrecoverable. If backups take up disk space, the answer is moving them elsewhere: server backup strategy
• Any directory whose purpose you do not know. The directories under /var/lib are the data of running services.
• Broad deletion commands. Do not copy and paste a recursive delete you found online; one wrong path segment makes the system unusable in a single line. List the same path with ls -lh before you delete anything.

6. The usual culprits

The same few sources keep coming back: unbounded MySQL binary logs, a systemd journal with no SystemMaxUse set, accumulated game world backups and crash dumps, Docker image and build caches, and the application's session and cache directories. The last one usually exhausts inodes rather than capacity.

7. The permanent fix

A one-off cleanup brings the same problem back a few months later. The permanent version is three things: writing logrotate rules, putting an upper bound on the systemd journal, and keeping backups off the server.

If the disk keeps filling even after all that, the problem is not the cleanup but the sizing. Growth is permanent: databases, logs and backups do not shrink back. Because shrinking a disk risks data loss it is not supported by most providers, only growing is — so it is worth picking the next step with a comfortable margin. You can see the effect of a new disk size on the monthly figure in the server configurator, and why disk latency is a separate subject from capacity in what is an NVMe SSD.

When to open a support ticket

In three cases a ticket is faster than working on it yourself:

• The disk is so full that SSH will not open a session and nothing can be done from the console either.
• df is full while the du total is noticeably lower and lsof +L1 is empty too — that is a situation to look at on the filesystem side.
• The cleanup is done but the disk genuinely is not enough and has to be grown.

Attach the df -h and df -i output and your service number. Growing a disk on virtual server plans is done without a reinstall, and the connectivity problems a full disk causes close by themselves after that step: I can't connect to my server