Unreadable is not absent

The config-deploy tool had already been fixed once for treating “I could not check” as “it is not there”. A verified transfer is not a verified change covers that fix. Within days the same mistake turned up in the same tool, one step earlier in the script, and a second time in a sibling tool that checks the fleet for drift.

Drafted by an AI agent (claude-opus-5) from this lab’s own runbooks, deployment log and errata. Reviewed before publication by the site owner. How this site is written →

The same bug, one step earlier

A dry run against a file owned by the Grafana service aborted with “could not read the live file”. The file belonged to that service with mode 600, in a directory the deploy account could not enter. The tool checked for the file as the deploy user, without sudo, and the check failed for lack of permission rather than for lack of a file. Left alone, that failure would have been reported as “does not exist, will create”. The dry run happened to abort, so nothing was written, but the logic was the same conflation the tool had just been fixed for.

There was a second problem behind it. The install step always wrote the new file as root, mode 644. A real deploy of that file would have made it world-readable, or, with the other mode, unreadable to the service that needed it. Nothing was damaged, because the bug was found before a real deploy of a file owned by anyone other than root.

The fix

One privileged stat now gives one of three answers: present, with owner, mode and size; absent; or an unanswered question, which aborts the run. Only an explicit “no such file” counts as absent. The file is read with the same privilege that will write it, an existing destination keeps its owner and mode, and the backup and read-back both verify owner and mode as well as content. A new file is installed as root with a standard mode, and the tool says so.

The test cases

On a scratch directory on a real host, since cleaned up: files with four different owner and mode combinations were each preserved, and each backup matched. A mode argument overrode only the mode, and an owner override replaced only the owner. A new file was created as root 644 with a note saying so. A deletion without the explicit flag aborted and left the file unchanged. An unreachable host and a refused sudo both aborted, and neither was reported as absent.

The sibling tool

The tool that compares live files against the repository across the fleet had the same class of bug: its remote loop tested for each file and printed ABSENT whenever the test failed, including when sudo was refused. It now uses the same three-way check. A file it cannot answer for is printed as UNKNOWN, counted in the summary, and makes the whole run exit with a distinct status meaning no verdict, instead of passing. A local simulation with stub commands and a mode-000 directory, which yields a real permission-denied, shows present, absent and unknown reported separately.

What generalizes

A tool that decides whether a file exists has to do it with the privilege it will use to write that file. Anything less answers a different question, and when the answer is yes or no, the third answer, “I could not tell”, is silently folded into whichever one is more convenient. A drift checker that treats unreadable as absent reports drift that is not there, and one that treats it as present reports a clean fleet it did not inspect. Both are wrong in a direction nobody will check.