• Why Authentik

    An identity provider is the thing that knows who you are, so that eighteen other things do not have to. You log in once, to it, and every service in the lab accepts that login instead of maintaining its own username and password.

    Read on →

  • Why Rathole

    The services in this lab run on machines at home, behind a consumer internet connection. Some of them need to be reachable from the public internet. The home network has no ports open to the internet and no port forwarding configured on the router, and…

    Read on →

  • Validation is not a read-only operation

    A web server refused to start. The error named a permission problem on its own log file, on a machine where the log directory was owned correctly and the permissions had been set exactly as the runbook specified.

    Read on →

  • The certificate challenge that was never enabled

    This one never failed. It was found by reading, before anything ran, and that is the only reason it is a short story rather than a long one.

    Read on →

  • Why Caddy

    Every web request that reaches this lab from the internet arrives at a reverse proxy first. That proxy terminates the encrypted connection, works out which internal service the request is for, and forwards it there. It is also responsible for obtaining and renewing the certificates…

    Read on →

  • Why Proxmox

    Everything in this lab that is not a rented gateway runs as a virtual machine, and something has to run those machines. That job goes to Proxmox: an open source hypervisor platform that turns one physical server into many isolated ones, with a web interface…

    Read on →

  • The notification that killed the script

    Two scripts sit on the lab’s gateways waiting for the day the primary one dies. One moves the public DNS records to the standby. The other moves them back. Both were, until this week, incapable of telling anyone they had run.

    Read on →

  • Status: the identity tier is up

    Single sign-on is running. That is the tier everything above it has been waiting on since the architecture was written, and the one the runbooks warn against skipping in the strongest terms available.

    Read on →

  • The punctuation that would not parse

    A PowerShell script failed to parse with seven errors. Not one of them mentioned the cause, and every one of them pointed somewhere else.

    Read on →

  • The sampler that was late

    A virtual machine kept freezing for minutes at a time. Remote sessions dropped and took several minutes to come back. On one occasion the hypervisor’s console showed the guest pinned at 100% CPU for the duration; on an earlier occasion it had not.

    Read on →