When It Fails, Will I Know?

On Thursday, ten teachers will sign in to a professional development session using a tool I built. If it holds up, the other hundred follow.

I'm not an engineer. I'm the director of programs for an Arkansas district — professional development is one of the things I'm over — and I've spent the last few years building tools our district actually uses. I built this one with an AI agent. I described what I needed and it produced a working system on top of Google Sheets. I load the staff list and the sessions, teachers sign in, and I get admin controls and a review step before anything is final. When I finalize a session, each building principal gets a report of who from their campus attended. And I get the clean list I need to enter hours into the state system.

The tool itself isn't an agent. An agent built it. Either way, the question I care about is the same one, and it isn't the question these systems get sold on: not "does it work," but "when it fails, will I know?"

PD hours go to the state, and I'm the one who enters them. That puts me first in line to catch a bad number — which sounds reassuring until you look at what I can actually catch.

If the system credits a teacher three times for one session, I see it while I'm inputting. Nobody gets fourteen hours for a four-hour day without it jumping off the page. I fix it and move on.

If the system doesn't credit a teacher at all, I see a shorter list — and no way to know it's short. I wasn't at the door counting. The record says twenty-two attended, and twenty-two is a perfectly plausible number. That teacher loses hours they earned and finds out months later, if ever.

Too much credit is a nuisance. Missing credit costs somebody something, and it leaves no trace. That asymmetry is what I'd want another administrator to take from this.

So before Thursday I wrote out the ways a system doing this job can fail. There are five, and I didn't come up with them. A benchmark published this July called GuardianAgentBench sorts agent failures into exactly these five categories: missing a required call, picking the wrong tool, passing bad or incomplete parameters, repeating a call, and running calls out of order. (Worth applying the same scrutiny to my source that I'm asking you to apply to vendors: five of the paper's eight authors work at Vectara, one of the three platforms tested — and the top score in the paper belongs to a model running on Vectara.) Two things in their results are worth an administrator's attention. The best-performing setup they tested got about three-quarters of tasks fully right. And the strongest models fail in a specific direction: they skip required steps rather than picking the wrong ones. That's the failure I'm most worried about here.

Here's what those five look like in my district.

It skips a step. A teacher signs in and the sign-in never gets written. Nothing appears wrong. The teacher saw a confirmation. The absence of a record looks exactly like the absence of a teacher, and I can't tell them apart after the fact. This is the one that worries me most.

It does the wrong thing. Instead of appending a row to Thursday's attendance tab, it edits that teacher's row on the master staff roster — the sheet that feeds every other sheet. Nobody asked it to write there. And this is a different category of problem than a wrong answer. A wrong answer is just wrong. An unrequested write is a change to a record I'm accountable for, in a place I wasn't watching.

It does the right thing with the wrong details. Tuesday's session gets recorded as Thursday. Every screen shows a successful sign-in. The teacher is credited. The hours exist. Nothing surfaces until someone reconciles dates against the calendar, which is never.

It does the right thing twice. Duplicate entries, inflated hours.

It does things in the wrong order. It records attendance before the roster is loaded, matching names against an empty list. This is the rarest of the five in their results — under five percent of failures — and I'm including it for completeness, not because I expect it.

Here's the part I keep coming back to. Not one of those five throws an error. There is no red screen for any of them. The only thing standing between a wrong record and the state is whether a human happens to look at it and think that can't be right — and that depends entirely on whether the wrong record looks implausible.

Fourteen hours in a four-hour day is implausible, so I catch it. Twenty-two attendees is plausible, so I don't. Tuesday recorded as Thursday is plausible. An edit to a roster cell I have no reason to open is plausible until the day it isn't. The duplicate entry is only harmless because a person is looking, and the other four are the ones where looking doesn't help.

A clean screen is not evidence that anything worked. It's evidence that nothing threw an error, and those are different claims.

What I'm doing about it: ten people instead of a hundred, and a paper list Thursday so I have something to reconcile against. I'll publish what the reconciliation shows, including the parts that don't flatter the tool.

These tools will keep arriving in your district, and the pitch is always the same shape: here's the workflow, here's the time it saves. That's the version where everything goes right. Ask instead which failures throw an error and which produce a clean-looking wrong record. Ask what happens when a step gets skipped. If nobody can answer, nobody has checked.

You don't have to be technical to ask that. You just have to be the person whose name is on the state report.