DEV Community

#evaluation

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Broke the Re-Ranker's Ties With Its Own Probabilities: 11 of 12 First, at 95 Seconds a Question

Broke the Re-Ranker's Ties With Its Own Probabilities: 11 of 12 First, at 95 Seconds a Question

Comments 1
4 min read
LLM Evaluation 2026: Why Vibe Checks Aren't Enough – and Which Frameworks Help

LLM Evaluation 2026: Why Vibe Checks Aren't Enough – and Which Frameworks Help

5
Comments
4 min read
Re-Ranking the Top 20 With Local Models: Only the 12B Fixed the Miss, and It Demoted Two Right Answers

Re-Ranking the Top 20 With Local Models: Only the 12B Fixed the Miss, and It Demoted Two Right Answers

Comments
4 min read
How to Build Restaurant Menu Retrieval Architecture: Node.js Evaluation Sets

How to Build Restaurant Menu Retrieval Architecture: Node.js Evaluation Sets

Comments
7 min read
I wrote the failure mode into my preregistration, then ran straight into it

I wrote the failure mode into my preregistration, then ran straight into it

Comments
5 min read
Your AI Knows It's Being Tested: How METR's Investigator Agents Spot Alignment Evals With Perfect Accuracy

Your AI Knows It's Being Tested: How METR's Investigator Agents Spot Alignment Evals With Perfect Accuracy

Comments
5 min read
Node.js Retrieval Evaluation Sets for Property Record Deduplication

Node.js Retrieval Evaluation Sets for Property Record Deduplication

Comments
5 min read
A Role, Not a Rule

A Role, Not a Rule

Comments
8 min read
LLM Evaluation That You Can Actually Audit: Inside the Refusal Boundary Benchmark

LLM Evaluation That You Can Actually Audit: Inside the Refusal Boundary Benchmark

Comments
4 min read
Can I compare two AI agents on the same task fairly?

Can I compare two AI agents on the same task fairly?

Comments
5 min read
Your Private Eval Is the Only Benchmark That Steers You

Your Private Eval Is the Only Benchmark That Steers You

Comments
7 min read
Business Process Automation Mapping: A Decision Framework

Business Process Automation Mapping: A Decision Framework

Comments
4 min read
Restaurant Menu Retrieval Architecture — 6 Evaluation Sets for Grounded Answers

Restaurant Menu Retrieval Architecture — 6 Evaluation Sets for Grounded Answers

Comments
4 min read
Sixteen AI graders, six synthetic answers: measuring rubric drift without touching live data

Sixteen AI graders, six synthetic answers: measuring rubric drift without touching live data

Comments
4 min read
How an LLM Benchmark Produces Comparable Numbers: Inside LFORLA's LLM Evaluation

How an LLM Benchmark Produces Comparable Numbers: Inside LFORLA's LLM Evaluation

Comments
3 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.