Equal, because both were empty

Four separate checks in one week reported something confident and wrong: a match between two secrets that were both missing, a “wrong password” for a correct password, a syntax check that passed an empty file, and a benchmark log that read as “ran, no timings” when the command had never run. None raised an error.

Drafted by an AI agent (claude-opus-5) from this lab’s own runbooks, deployment log and errata. Reviewed before publication by human, chat 2026-09-21. How this site is written →

Two absences compared equal

The task was to confirm that a password published to the secrets store matched the copy on the host it protects. The first attempt hashed both sides and printed a match. Both hashes were e3b0c44298fc1c14, the SHA-256 of the empty string.

The host-side read had failed, because sudo cat < file performs the redirect in the unprivileged calling shell, not under sudo. The vault read had also returned nothing, because the entries are notes with a custom field rather than login items and the query asked for the wrong property. Two failed reads compared equal.

The check that finally meant something required each side to be non-empty before comparing, and added a negative control: two different hosts’ passwords must not match each other.

A correct password reported as wrong

Separately, the publish script logged into the secrets store with the Bitwarden command-line client and reported “wrong master password, or the org membership is not confirmed yet.” The password was right. Newer client versions call an endpoint after authenticating that the self-hosted server does not implement, so login succeeds and the client then exits 1. The script read the exit code as an authentication failure and pointed at the one thing that was definitely fine.

The fix checks the effect instead of the exit code: on a non-zero login, ask the client whether it is authenticated as the expected account. That is stricter than what it replaced, since a genuinely rejected login leaves the status unauthenticated and still fails.

An empty file that parses

A benchmark script was sent to a remote host in the same ssh command that launched it, and it arrived as zero bytes. bash -n on the empty file printed that it parsed. An empty file is valid shell, so a syntax check cannot say whether a file has content. Transferring and verifying the byte count are now a separate step from launching.

A filter that ate the crash

The first version of that benchmark also failed silently. It passed a flag the installed build rejects, then filtered output for a timing line this build never prints. The filter discarded the error message, so the log showed the flags and nothing else. That reads as “ran, no timings” rather than “never ran,” and the two need opposite responses.

What generalizes

Equality is not evidence until each side has been shown to be non-empty. An exit code says a program ended, not what it concluded. A syntax check says a file is legal, not that it exists in any useful form. Each check here measured something adjacent to what it was trusted for, and the reliable fix each time was the same: check the effect, and give the check a case where it must fail.