Dispatch № 007July MMXXVI

Human in the loop: why AI can't grade its own homework.

AI saying 'done' isn't proof. What human-in-the-loop AI actually means, how checked work reaches your customers, and what happens when the check fails.

The short answer

Human in the loop means the AI does the work and something independent checks it before it reaches anyone: a separate checklist for every task, a human for anything that needs judgment. The AI can't just say "done." It has to show "done."

Say done vs show done

Ask an AI if it did the job and it will say yes. It says yes when the prices were wrong, when it skipped a rule, when it half-read the request. Saying done costs it nothing. So every helper writes down what it did: the task, the input, the output, the laws it followed. That's the proof, and it exists whether anyone asks for it or not.

The checker is not the worker

A separate checklist then checks the work against the laws. The checker is not the worker, the same way the person who cooked isn't the one who tastes the plate. One system does, another verifies. Merge the two and you're back to trusting the answer.

What gets checked

The work matches the laws. Tone, rules, the never-do list.
The facts match the brain. Prices, dates, names, links.
The leash was respected. Nothing sent, spent, or changed beyond what you allowed.

A miss on any of these is a failed proof. Failed proofs never reach your customers.

When the check fails

The work stops. The task goes back to the helper with a note about what failed, or up to a human when the failure needs judgment. Either way it's written into the brain, so the same failure gets rarer over time. And no, no system catches everything. Anyone who says otherwise is lying. That's why proof comes with humans on call, around the clock.

Trust is earned. The proof trail is how it's earned.