Writing / Series

Checks that lie

Seven posts about checks that lie and failures that stay silent. Start with the first, which is the idea; the other six are the cases.

7 posts · read in order

  1. Events

    Part 1: Walk the Room: What Live Events Taught Me About Checks That Lie

    At a live show the console can say everything is fine, and you still walk to the back of the room to listen. I spent sixteen years building Endless Events, and this post carries that habit into my DJ, music and home builds: the checks that passed without looking, the one question that catches them, and why job alerts now go through triage instead of my phone.

    4 min read
  2. Builds

    Part 2: A Check That Can't Fail Is Worse Than None

    A canary built to catch one bad state passed exactly that state, because grep -c prints 0 and also exits non-zero. The post ends with copyable self-tests that run a known-bad and a known-good input through a check in the same run.

    6 min read
  3. Music

    Part 3: A Healthy Container and a Blind DJ App

    For 1 hour and 42 minutes, the app that tells my stream what I'm playing didn't see a single track. The whole time, its container reported healthy and its overlay page answered every request. I had power-cycled my decks between sets, and the app only looks for decks once, when it starts.

    7 min read
  4. Builds

    Part 4: The Radio That Lied About Dying

    The radio in my new Meshtastic base station was dead for ten days, and the service manager recorded every one of its fourteen restart attempts as a clean, successful exit. Read at face value, that pointed at the hardware, and the first evaluation called the new station "not a candidate today." The real cause was a full node database crashing the radio software, behind a start script that reported each crash as a success. I found it by building the whole station the way I wanted it, with transmit turned off, and watching. That inert build also caught a bug that would have given my node a new identity on every reboot.

    7 min read
  5. Home

    Part 5: Half a Light Curtain Said “Success” and Did Nothing

    For months, a smart LED light curtain in my house wouldn't reliably do what Home Assistant told it. In August 2026 I stopped patching it and took it apart, one hypothesis at a time. Settings, a power cycle and a new local control path didn't solve it. Then I learned the one curtain was two devices, and that one of them answered "success" to every command while its lights didn't change.

    6 min read
  6. Home

    Part 6: A Switch That Feeds Smart Bulbs Is Not a Light

    For four days in September 2026, a room-wide "lights off" automation in my Home Assistant setup cut the power to a set of smart bulbs 75 times. Every one of those runs reported success, and every time anyone looked, the lights were on. It won for about ten seconds at a time.

    5 min read
  7. Home

    Part 7: The Veto That Never Retries

    In September a bathroom in my house stayed lit for nearly nine hours because a humidity sensor read 51 when the rule wanted it under 50. Thirteen minutes later it read 48, and nothing happened. The automation that turned the lights off had already used its one chance, and in Home Assistant a condition that turns true again does not start anything.

    7 min read