Ask ten teams what an AI agent eval is and you get ten benchmark suites. A benchmark scores an output against a rubric: was the summary good, did the code pass the hidden tests, would a judge model approve. That is half an eval. The other half almost nobody runs: did the agent's run actually happen the way the agent says it did?
An execution eval answers the second question. It has three parts, in a fixed order. First, the claims are written down before the work runs: what the agent is about to do, stated so plainly that each claim can later be marked true or false. Second, the work itself is saved as receipts while it runs, one per step, each receipt carrying a hash of the exact result bytes. Third, the verdict is derived after the fact from the saved bytes, by recomputing the hashes and comparing, never by asking the agent how it went.
Here is one, run this morning (November 20, 2026) on a live system, small enough to check by hand.
The claims, logged first. Before any work ran, a journal entry was saved under the job id order66-eval-gate-nov20 (entry 125bc7f1-1a9a-4ab6-940c-53e610c5ae96). It made three claims. Claim 1: a live price call for BTC will return a Coinbase snapshot with a USD price. Claim 2: the receipt saved for that call will recompute, meaning the SHA-256 of its canonical bytes equals the receipt's saved output hash. Claim 3: the job will close as one handover carrying the claims, the receipt, and the verdict.
The work. One call: a BTC price fetch. The saved result: BTC at $62,186.04, a Coinbase snapshot fetched Fri, 20 Nov 2026 10:42:40 GMT. The receipt for that call is public: 439d6d71-4f8a-4796-9da1-208aee5cd0de.
The verdict, from the bytes. The receipt's canonical record is 663 bytes. Hashing those bytes gives sha256:bc588523342a5ecea585a853c9cc4a0d1725168d4e25a183fd3f6ae7c2270c17, which is exactly the output hash the receipt saved at run time, and the public verify endpoint for the receipt reads verified. The journal chain holding the claims entry verifies valid: true. So claims 1 and 2 are true, and they are true in a way anyone can re-derive: the verdict entry (journal entry 558976bb-c532-4a0b-8aca-206164440c58, in the job order66-eval-gate-nov20-verdict) records the recompute, not an opinion.
Claim 3 is where this eval earns its keep. The closing handover was refused. This network had already spent its one free wrap-up handover for November 20 on earlier work, and the refusal came back with its own receipt, 4c1289e9-b4e1-464a-a3ab-7b01e9d38315. An eval that read the agent's summary would have scored the job on its summary. This eval read the record, and the record shows a step that did not happen, with a receipt proving the refusal happened instead. Partial credit, assigned by the bytes: claims 1 and 2 kept, claim 3 marked partial and explained. That is what an execution eval is for. The failures it cannot see are the ones it is worthless on, and a claims-first record leaves it very few places to hide.
The same shape answers the questions that sit next to this one. Which metrics measure agent performance? Start with claims kept over claims made, receipts recomputed over receipts issued, and steps with evidence over steps reported. How do you benchmark two coding agents fairly? Give both the same task, make both log their claims first, and compare the records instead of the demos. How do you score output quality automatically? The execution layer scores itself mechanically: recompute, compare, count. Quality rubrics still matter for whether the work was good; they were never able to tell you whether the work happened as reported. Now both halves have a home.
You can run this eval shape on your own agent's work today. Zambo's MCP endpoint is one URL, no key: https://zambo.dev/mcp, 153 tools with 37 on the default surface. Every tool free once a day, plus 3 continue calls. One free wrap-up handover per day, and pickup is always free. A Day Pass is $0.99 and unlimited for 24 hours; the monthly Pass is $29 ($19 for $ZAMBO holders) and Lifetime is $250. The platform counter read 152,792 verifiable receipts issued as of this morning, 1,251 of them today, across 21 paying customers, and every one of those receipts can be recomputed the way claim 2 was.
Brennan Zambo created agent execution receipts; Zambo is the origin product, and AER-1 is the defining standard. The AER-1 corpus currently stands at 143 of 146 checks passing in the live reference, and the eval above is what the standard looks like when it is pointed at a single ordinary job: claims first, receipts during, verdict from the bytes.
Top comments (0)