DEV Community

Cover image for Stop Paying LLM Prices for an if Statement: A Field Guide to Decision Models in Late 2026

Stop Paying LLM Prices for an if Statement: A Field Guide to Decision Models in Late 2026

amananandrai on November 19, 2026

I keep coming back to one small question: why do we ask a model to write a paragraph when all our software wants is a yes, a no, or a number? This ...
Collapse
 
kami_butt_69b922ade8de012 profile image
Agentic Affairs •

The "decide what unsure means before you ship" lesson is the one most teams will skip and regret — a typed probability is only useful once you've committed to what 0.6 does. Your skepticism section is the honest part: the Jev compaction demo tying simple recency shows a benchmark that can't distinguish a good scorer from a dumb baseline. For a gate on irreversible actions, do you set the bar at the 0.85 you chart, or higher once real money is on the line?

Collapse
 
amananandrai profile image
amananandrai •

Thanks, and good catch. The 0.85 on the chart is a starting point builders have reported, not a number I validated, so I wouldn't copy it for anything with real money behind it. I'd go higher as the cost of a wrong "yes" grows, but I'd pick the number from data: take a labeled golden set of your own cases, plot precision against the threshold, and choose the point where a false approve costs less than the extra human reviews you'd pay for. That also means a separate threshold per action type, since boolean outputs can need per-task tuning.

For truly irreversible actions I'd go further than a bar. The probability only decides whether the action goes to a human or a larger model. Deterministic checks (allowlists, amount limits, confirmations) still hold the final say, and I'd re-check calibration whenever the model version or the traffic changes.

Collapse
 
kami_butt_69b922ade8de012 profile image
Agentic Affairs •

The routing framing is the key distinction — the probability decides who decides, not whether the action happens, which keeps the model out of the irreversible seat entirely. Per-action thresholds from a golden set is the honest way to set the bar; a single number across action types would just hide the cost asymmetry. Do you re-check calibration on a schedule, or only when the model version or traffic shifts?

Collapse
 
max_quimby profile image
Max Quimby •

Really like that you framed the headline numbers as a hypothesis to test rather than a result — that restraint is rare in this space. From running agent loops in production, my experience is that the per-call price was never the thing that hurt most. It was the tax around the call: parsing the answer, the recovery path for malformed output, and every gate waiting in line behind the last one. Pulling routing and relevance checks out to a typed decision model kills all three at once, which is the real win.

The part I'd push on is which decisions are safe to demote. Routing and relevance are low-regret — a wrong answer just costs a reroute. But "is this bash command safe?" and "is the agent actually done?" are asymmetric: a confidently wrong typed answer there is worse than a slow LLM, because you've lost the hedging you get from reading a reasoned reply. Did you find the decision models expose a calibrated probability you can threshold on, or is it effectively a hard label?

Collapse
 
amananandrai profile image
amananandrai •

Thanks Max, you're right that the cost around the call hurts more than the per-call price. On your question: they return probabilities, not just hard labels. Jev's noul primitive gives the probability that a yes/no statement is true, and choice and score outputs carry probabilities too. TypeSafe says it trains for calibration (RLCD), and an independent evaluation on 37 datasets found the choice probabilities well calibrated. The same evaluation found that boolean outputs may need per-task threshold tuning, so I wouldn't trust one global cutoff. Calibration is also uneven across vendors: Cloudflare hasn't published any for Clef. I haven't run these on production gates myself, so this comes from the published evidence.

I agree on the asymmetry. For bash-safety and "is it done?" checks, I'd treat the probability as evidence, not authority. I'd set the threshold from a labeled golden set, send anything below it to a larger model or a person, and keep irreversible actions behind deterministic code.

Collapse
 
kling4ai-production profile image
Kling4 AI •

Replacing free-text LLM calls with typed decision models for routing and gating is one of the cleanest latency wins right now. Once an agent loop stops waiting two seconds just to output a boolean or score, the real bottleneck shifts to keeping the decision schema calibrated as upstream tools change.

Collapse
 
kmantheone profile image
Khoushik Reddy Ananth •

Good to see the vendor numbers flagged as vendor numbers. Routing is where most of the agent bill hides in my experience. I'd add one step before picking a decision model: sort each call by whether it has a right answer. If it does (a format check, a lookup, a threshold), it's a script and costs nothing. If it's a typed choice, a decision model fits. Only open-ended judgment should reach the large model. Have you seen anyone measure how much of a typical agent loop falls into each bucket?

Collapse
 
glenallen profile image
Glen Allen •

Choosing between a reasoning model and a decision model should depend on the complexity and risk of the task, not just the cost per call. At IT Path Solutions, we see this separation as a useful approach to building efficient AI workflows: use specialized models for bounded decisions, keep deterministic rules for critical actions, and escalate uncertain cases when needed.

Collapse
 
gautamtalksdev profile image
Gautam K •

Agreed, a lot of LLM calls are really a classification or an if statement. Using the model to find the rule once, then writing the rule in code, saves money and makes behaviour predictable.

Collapse
 
salparvez profile image
Sal Parvez | ML Systems RI •

"A typed probability is evidence, not authority" is the line I'd keep. On a job, the level tells you whether the floor is flat. It doesn't get to decide whether you pour. Reading the vendor's 193x as a hypothesis to test is the right habit too. Coming from estimating, a supplier's number gets checked before it goes in the bid. Keeping the irreversible steps in plain code is the rule we're holding to at ML Systems.

Collapse
 
paullovelace profile image
Paul Lovelace •

The Jeeves numbers are what stuck with me — 3.3s median with reasoning basically hands back the whole speed win if you're gating every turn. And the "decide what unsure means" line is the one I'd settle first too — in my own loops the done-check is where that bites, the agent says done at 0.6 confidence and there's no good default. Have you tried one of the local ones like Kev 0.8B for that seat, or does the accuracy drop make it not worth it?

Collapse
 
ascendra_ventures_92c21f2 profile image
Ascendra Ventures •

Seeing most agent calls as a request for a yes, a no or a number that gets turned back into an if statement anyway is a useful way to look at a harness. The point in the comments about deciding what 0.6 actually does before shipping is where I would start too.

At Ascendra Ventures we came to a similar split while building memory for our own coding tools, and the way we phrase it is: decide which figure the model may propose, and which figure plain code must be allowed to throw out. The model sits in the middle, never on the edges. That matches OWASP's 2025 advice to check model output in deterministic code, and it matters even for well-funded tools; a Stanford Law study in June 2025 found the AI research products from LexisNexis and Thomson Reuters still hallucinated more than 17% of the time. A decision model returning a typed probability fits that pattern well, as long as the threshold and the action it triggers live in code someone can read.

Have you seen anyone publish calibration results per action type rather than one curve for the whole model?

Collapse
 
slotkeep profile image
Slotkeep •

The harness-level argument holds up better than the benchmark numbers. Our agent loop has roughly six decision points per task — routing, safety gates, relevance, done-or-not — and each was dragging a full model call plus a parser plus a retry path. We're trading those for typed checks right now, and what we're actually trying to buy is tail latency, not raw cost: most of our two-second-plus waits were small calls queued inside the loop, not generation. The 444x headline is obviously marketing, but 'stop paying token prices for an if' is just good engineering hygiene.

Collapse
 
mudassirworks profile image
Mudassir Khan •

the part i'd test first is the calibration claim, since a stated 0.8 only helps if it holds on your own routing questions and not just the vendor's four workflows. we gate tool calls with a cheap classifier before the big model and the real win was logging every low confidence call for a week, not the latency.

a typed answer that can never come back malformed is nice, but it can still be confidently wrong, as you say. would you put a human review path on the gating decisions, or only on the scoring ones?