Category: Lessons Learned
-
The certificate challenge that was never enabled
This one never failed. It was found by reading, before anything ran, and that is the only reason it is a short story rather than a long one.
-
The notification that killed the script
Two scripts sit on the lab’s gateways waiting for the day the primary one dies. One moves the public DNS records to the standby. The other moves them back. Both were, until this week, incapable of telling anyone they had…
-
The punctuation that would not parse
A PowerShell script failed to parse with seven errors. Not one of them mentioned the cause, and every one of them pointed somewhere else.
-
The sampler that was late
A virtual machine kept freezing for minutes at a time. Remote sessions dropped and took several minutes to come back. On one occasion the hypervisor’s console showed the guest pinned at 100% CPU for the duration; on an earlier occasion…
-
The check that punished uptime
A test asserted that the tunnel between two servers was up. It failed on both tunnels, while both were demonstrably carrying traffic — seventeen established connections, health endpoints answering, and a full request succeeding through each one.
-
A firewall rule that could never match
A firewall rule was added to both gateways to allow traffic on a new port. The rule was syntactically correct, applied cleanly, appeared in the live ruleset, and was never evaluated once.
-
The step that was not about DNS
The runbook step was about joining a VPN. One command, on each machine, to enroll it in the private mesh.
-
If the watcher dies, its last words are reassuring
The monitoring system was switched off for eighteen minutes. Every monitor reported UP for the entire period.. Status 1 is UP. Both rows were real, correct, and eighteen minutes old.
-
Every light was green and DNS was dead
Two new DNS resolvers went live. The configuration validated. Both services reported active. Every internal name resolved correctly, on both machines.
-
The config was read, and ignored anyway
An earlier post covered a setting that never applied because another file set it first. sshd_config is first-match-wins, a hardening file was numbered 99-, and the value lost.