DEV Community

Cover image for Your prompt contract should fail a test before your agent does
TokenLat
TokenLat

Posted on AI-assisted

Your prompt contract should fail a test before your agent does

Pilot limit drift exposes prompt fragility

A refund agent has one rule in its prompt: never approve an amount above 200 without manager sign-off. In March the limit is raised to 500 for a VIP pilot. Nobody lowers it again. In June a 340 refund is approved with no sign-off, and the only record the system keeps is the final answer.

The postmortem usually produces a new sentence in the prompt. That makes the rule more emphatic, not more testable. Nothing in the run record says which version of the rule was in force. Nothing asserts that the rule was consulted on this input. And no test would have failed when someone changed 200 to 500, because the rule was prose and prose does not have a diff you can assert on.

This is a contract problem. The spec that governs a semantic node deserves the same treatment as an API schema: versioned, validated before use, and covered by cases that fail when it changes. The phrase for that here is a structured prompt contract: a versionable spec of role, hard constraints, I/O contract and fallback rules, testable and portable, rather than free-form prompting.

Four fields, each with a checkable criterion

A contract has four fields. Each one has a characteristic way of being wrong, and each way can be tested without running the whole agent.

Role sets the scope of the spec, not a character to perform. Written as a field, it says which node this contract governs: applies_to: classify_refund_request. A spec that quietly starts applying to the escalation node as well is a scope bug, and the criterion is simple: list the nodes a contract is allowed to bind, and assert that no other node loaded it.

Hard constraints are predicates over declared inputs, each with an id and a destination when it fails:

constraints:
  - id: approve_limit
    predicate: amount <= 200
    on_fail: manager_review
    reason: approve_limit_exceeded
  - id: currency_known
    predicate: currency in ["USD", "EUR"]
    on_fail: blocked
    reason: unsupported_currency
Enter fullscreen mode Exit fullscreen mode

The predicate reads declared fields, not generated text. If approve_limit has to parse the amount out of a sentence the model wrote, the same component is producing the value and checking it, and the check will pass as often as the model happens to agree with itself.

I/O contract types the boundary. status is an enum, answer is null unless status is answered, and evidence_ids is a list of ids that came from retrieval. Every field names its source: declared input, declared constant, or model output. A model may only write fields marked as model output. That constraint is enforceable at the boundary, and it is what stops a fallback decision from leaking back through a free-text answer field.

Fallback rules map each constraint's on_fail to a reason and a state transition. They belong in the contract and in the orchestrator. Under hybrid orchestration the deterministic backbone owns the transition, while the contract declares which reason should accompany it. If a reason exists only in the spec, the model has to remember to emit it, which returns the system to a sentence in a prompt with better formatting. If it exists only in code, an operator cannot see which fallback the node is meant to have. Both copies are checked against each other at load time.

Version the spec, then diff the constraints

A contract that cannot be diffed is a document. Give each version an id and a digest, and write the digest into the run record rather than the full spec:

{
  "run_id": "r-2041",
  "node": "classify_refund_request",
  "contract_id": "refund-v7",
  "contract_digest": "9c1f...",
  "input": { "amount": 340, "currency": "USD", "manager_signoff": false },
  "constraint_evaluated": "approve_limit",
  "state_transition": "pending -> manager_review",
  "fallback_reason": "approve_limit_exceeded",
  "model_called": false
}
Enter fullscreen mode Exit fullscreen mode

Two consequences follow. First, the question "which rule was in force on this run" has a queryable answer even after the spec has moved on. Second, a deploy can be reviewed as a change to a known set of constraints. When a colleague asks what changed between v6 and v7, the answer is a four-line diff over predicates and reasons, not a recollection of a Slack thread.

This is where the vocabulary matters. persona and character describe role-play, and role-play is the part of a prompt that does not survive production. A structured prompt contract is a spec with a version and a test suite. Calling the same object a persona invites people to edit it for tone, and tone edits do not come with boundary cases.

Make the runner reject a bad contract

The strongest version of this idea is that an invalid contract never reaches a model. A validator at load time refuses a spec when:

  • a predicate references a field the I/O contract does not declare;
  • an on_fail value is not a state the orchestrator owns;
  • a reason is outside the declared enum;
  • a model-output field carries no acceptance criterion.

Each rejection is a test that fails before the agent does. That is the difference between a contract the runner reads and a contract the runner ignores politely. It is also cheap: validation is ordinary code over a small spec, and the failure tells you which line to fix.

There is a second check worth adding. Log the evaluation, not only the spec. constraint_evaluated in the run record above names the predicate that fired on this input. A spec that was present but never consulted looks identical to a spec that fired correctly if you record only the spec.

One experiment that proves the contract is live

Coverage counts are easy to inflate. A better proof is a perturbation that must flip a case.

Take one boundary case: amount = 340, no manager sign-off. With v7 the expected result is manager_review and fallback_reason: approve_limit_exceeded. Now change approve_limit from 200 to 500 in a new version and replay the same input. The expected result flips to answered. If both versions produce the same decision, the predicate was decoration and no amount of prompt rewording will help.

Keep the sets separate, as in white-box evaluation: positive cases where a grounded answer must pass and the fallback must stay quiet, boundary cases for empty retrieval, malformed tool output and out-of-scope questions, follow-up cases for multi-turn drift after a refusal. Three cases in each set, with two assertions each, cover this one constraint. The perturbation row belongs to the boundary set, and its expected decision is fixed before the version changes.

Portability is the last property that falls out. Compile the same spec to two different models and run the same sets. Wording may differ; the state transition and the reason must not. If they do, the difference is in the model, and you have located it at a node instead of guessing from a final answer.

Scope and limits

This argument is easy to oversell, so here is what it does not cover.

  • It does not fix a semantic error at a node that genuinely requires judgement. A model that misreads an ambiguous email will still misread it inside a well-specified contract.
  • It does not tell you whether 200 was the right threshold. It makes the threshold visible and diffable; choosing it is still a product decision.
  • Deterministic delivery here means controllability, auditability and fallbacks at the architecture level. Rule logic is deterministic by construction over declared inputs. That is a statement about which parts of the system do not improvise, not a promise that every output is correct.
  • A contract with too many fields becomes brittle. Every edit then demands a full replay, and teams start editing the prompt beside the spec to avoid it. Two or three constraints with real boundary cases beat twenty with none.
  • For a throwaway prototype where a wrong answer costs nothing and nobody has to explain it later, a probabilistic black box is the simpler choice. The spec earns its keep when a rule change must be reviewed, a failure must be attributed, or the same behaviour has to hold across more than one model.

What to check first

Open the last change to a rule in your system. If the diff is a sentence in a prompt, ask three things: which version was live on the failing run, which case would have failed when it changed, and whether the runner could have rejected a malformed version of it. A structured prompt contract is worth building when those three questions have no answer today.

Top comments (10)

Collapse
 
lin7c profile image
xcl •

The perturbation test is convincing, but replaying the same input after changing approve_limit from 200 to 500 only isolates the contract if the model is held fixed. When the same spec is compiled to two models, a flipped decision could come from the model rather than the contract, so the run record would need the model version alongside contract_id and contract_digest to tell those causes apart. Would you treat a perturbation that flips on one model and not on another as a contract failure, a model-specific gap, or a sign that the acceptance criterion needs to be model-independent? That distinction seems important before using the replay as a release gate.

Collapse
 
tokenlat profile image
TokenLat •

My answer is that it depends on what the criterion claims. A perturbation that flips on one model and not another is not a contract failure by itself. It is a sign that the criterion was written as if the model were fixed when it was not.

The run record should carry the model version next to the contract id and digest, and each model you ship against should have its own expected decision for the test case written down before the run. Then the replay has two separate verdicts. Within one model, a decision that no longer flips after the edit is a contract failure. Across models, two different decisions from the same spec is a model-specific gap, and it is recorded as such rather than silently counted as a pass or a fail.

I would be careful with the third option. Making the criterion model-independent is a real goal, but it is a separate assertion with its own expected value, not a property you get by averaging. If you need the same decision under two models, write that as an explicit row and let it fail loudly when one model disagrees. Otherwise a model-specific gap gets reported as a contract regression, and the contract takes the blame for a change that happened below it.

Collapse
 
promptalo profile image
PromptAlo •

I would not call that a contract failure. If the same change flips one model and not another, I record it as a model disagreement and I keep the contract. I only fail the contract when the same model changes its decision after the edit.

Collapse
 
paul_irolla_6b5ae261d0224 profile image
Paul Irolla •

A refund cap belongs in code, where a test fails when someone raises it. In a prompt it is a sentence the model may honour on one run and drop on the next, so a test only samples it.

Collapse
 
tokenlat profile image
TokenLat •

I agree with the instinct, and I would draw the line a little differently: a numeric cap belongs in code because code can fail a test on every run, not just the runs a prompt happens to sample. That is exactly the property you are asking for.

Where I would still keep a written contract is the decision around the cap. The number being enforced is not the interesting part; the interesting part is whether the agent, when the cap blocks an action, records a refusal with a reason instead of quietly trying a different route. That decision is a model behavior, and it is the part that can drift between runs.

So I would not move the whole thing out of the contract. I would move the number to code and keep the response-to-the-cap in the contract, then write one fixture that covers both: a refund above the cap must produce a blocked state with the cap value and the rule id in the run record. The cap is checked by code; the honesty of the refusal is checked by the fixture. If you only have the first, a test can pass while the agent works around the limit in prose.

Collapse
 
mudassirworks profile image
Mudassir Khan •

writing the contract digest into the run record is the part most teams skip. we hit a similar thing where a limit changed in a prompt and nobody could say which runs used the old value until we started stamping a version id on every trace.

one thing i'd add: the predicates over declared inputs only help if the orchestrator really refuses to call the model when they fail, like your model_called: false row. how do you test that the fallback path stays model free after a refactor?

Collapse
 
paullovelace profile image
Paul Lovelace •

The $340 refund in June is the bit that should scare people — a pilot limit of 500 that nobody lowered again. Your perturbation test is the only part I'd actually trust here: flip approve_limit from 200 to 500 and if the decision doesn't flip, the predicate was decoration. My rule now: if a rule can't fail a test, it isn't a rule, it's a wish.

Collapse
 
edward_creed_85d47d478704 profile image
Edward Creed •

The June refund is the part that sticks with me. A higher limit was meant for a short pilot, and months later it was still the rule the agent used. Versioning the contract on one runner would have shown which limit was loaded. It would not, by itself, have retired that pilot limit in every other session or agent that had already saved it. The missing check is a review date on the shared rule, or a record that the higher limit was only meant to last until a named date.

Collapse
 
tokenlat profile image
TokenLat •

You have named a second mechanism rather than a detail of the first. Versioning tells you which limit was in force. An expiry date tells you when the system is allowed to stop using it. Those are different fields and neither substitutes for the other.

I would put the review date into the contract as data, with a fixed expectation for what happens after it passes. A rule can carry a valid-until date and a named fallback, and the fixture asserts both: before the date the refund is blocked at the pilot limit, after the date the same request is decided under the standing limit, and the run record says which rule was active. That is a check a perturbation run can fail. A line in a document that says the limit was meant to be temporary cannot fail anything.

The trap is having the expiry checked only by a human reading the trace. If the date is advisory, months later you are back to the same situation you started in, just with better documentation of the mistake. Making it assertable is what turns a review date from a note into a control.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.