The Report That Lied for Five Days and Never Threw an Error

Every morning an assistant I built sends me a short brief: what’s on today, what to watch out for, what needs a reply. For five mornings in a row it opened with some version of “nothing scheduled, free day at home.” I was on the road the entire time with a packed schedule. The brief was cheerful, confident, and completely wrong, and not a single alarm went off anywhere in the system.

TL;DR

• An automated daily report ran “successfully” for five days while quietly producing garbage: a data source returned “success, no results” instead of an error, so nothing looked broken.
• “It ran without errors” is not the same as “it worked.” Build checks for the empty answer, not just the crash.
• Give an AI the surrounding context and let it interpret, instead of hard-coding one rigid rule.

The report that looked fine

The brief is a boring, useful little thing. It reads a few sources, one of them my calendar, and lands every morning without me doing anything. It had been right for months. That is the first quiet trap of automation: a thing that has been reliable for long enough becomes invisible, and you stop checking the one output you’ve decided to trust.

So when it started telling me my days were empty, I shrugged and assumed I’d forgotten to put things in the calendar. The honest version is less flattering: I trusted my own code because it wasn’t complaining. No red logs, no crash, no alert. The job exited clean every morning, so as far as every dashboard was concerned, everything was fine.

What actually broke

A few days earlier, an unrelated change to the system underneath had knocked out the piece that reads my calendar. It did not fail. It returned “success, zero events.” An empty answer with a thumbs-up attached. To the rest of the program that is indistinguishable from a genuinely empty calendar: no events today, great, free day, write that up.

A tool that returns “success, no data” is more dangerous than one that crashes. The crash gets noticed. The empty success gets believed.

This is the silent-failure problem, and it lives everywhere automation does. Loud failures get seen and fixed. The expensive failures are the quiet ones, where a component hands back something technically valid and completely wrong, and every layer above it nods and passes it along as fact.

Why nothing caught it

Two reasons, and both bite real teams too.

First, I never checked for the empty case. My code asked “did the calendar read fail?” and the answer was always no, because an empty success is not a failure. The question I should have been asking is “does this result even make sense?” Zero events for five business days running is not a normal reading; it is a sign the sensor is broken. I had written logic to catch the crash and nothing to catch the nonsense.

Second, the report was too rigid to be suspicious. It decided “free day” from a single field: is there an event on today’s exact date, yes or no. That one narrow rule had no way to notice that the surrounding week was full of travel. It wasn’t reading the situation. It was reading one box, and the box happened to be empty.

The fix, in two parts

Part one was to make silence loud. An empty result from that calendar read is now treated as suspicious by default. When the data looks implausible, a full work week with nothing in it, the report says so out loud rather than smoothing it into a tidy sentence. The goal was never to build something that can’t fail, but to fail in a way I can actually see.

Part two was to stop encoding judgment as a rule. Instead of feeding the report a single yes-or-no field, I now hand the assistant a window of the surrounding days and let it read the week the way a person would. “Away yesterday, nothing today, away tomorrow” reads as “still traveling,” not “free day at home.” The harder you try to hard-code every edge case into rigid logic, the more brittle the whole thing gets. Hand the model the real context and a clear question, and you get judgment instead of a lookup.

What I’d do differently

I would have written the suspicious-empty check on day one. It costs about ten minutes, and it is the single most useful habit I know of in any automation. Decide, up front, what an impossible-but-valid result looks like, and make the system shout the moment it sees one. Not “did it error,” but “is this answer even plausible.” Wrong answers wearing a success badge will sit there and look healthy for as long as you let them.

The manager version of this lesson has nothing to do with calendars. Any automated report, dashboard, or AI summary your team leans on can do exactly what mine did: keep running, keep looking healthy, and quietly feed you numbers that drifted into nonsense after some upstream change nobody connected to it. “The pipeline ran” is not “the pipeline was right.” The question to put to whoever owns a recurring automated output is not “does it run?” It is “how would we know if it started lying to us?”

So here is the one worth chewing on: what is the one automated report your team reads every week that nobody double-checks anymore? That is the one most likely to be quietly wrong right now.