Evaluating a Model
Every model launch comes with a table of numbers going up. Almost none of those numbers tell you whether it will work for what you are building — and the eval that would is one you have to write yourself.
Three Kinds of Evaluation
They answer different questions, and confusing them is how teams end up shipping a model that benchmarks beautifully and disappoints in production.
Benchmarks
A fixed set of questions with known answers, scored by exact match or multiple choice. Cheap, repeatable, comparable across models — and the easiest to game, deliberately or by accident.
Preference
Show people two answers and ask which is better. This is the closest thing to ground truth for open-ended work, and the most expensive and slowest signal you can buy.
Task-specific
A set of inputs drawn from your actual traffic, with what a good answer looks like for each. The only evaluation that predicts whether the thing you are building works.
What Benchmarks Actually Measure
Each is a proxy for something broader, and worth knowing by what it leaves out.
MMLU
Exam questions across dozens of subjects. Measures recall and test-taking, not reasoning or usefulness.
GSM8K
Word problems needing a few steps of arithmetic. A reasonable probe of chained reasoning, long since saturated by frontier models.
HumanEval
Write a function, run the unit tests. Genuinely objective, but small and narrow — nothing like maintaining a real codebase.
HellaSwag
Pick the plausible ending. Useful for base models, largely uninformative for tuned ones.
Arena Elo
Head-to-head votes on real prompts, ranked like chess. Harder to game, but rewards style and length alongside substance.
Long-context recall
Hide a fact in a long input and ask for it back. Tests whether a large context window is usable or merely advertised.
Hallucination: Naming the Failure
Not one behaviour but several, with different causes and different fixes.
Fabricated specifics
Plausible, precise and wrong. Citations, version numbers, API methods, case law. The model is sampling from a distribution over things that look like valid answers, and a fake reference looks exactly like a real one.
See Restatement (Third) of Torts, section 47(b) — a section that has never existedConfident interpolation
Two facts the model knows, joined by a relationship it does not. Common where the training data is thin: the shape of the answer is learned even when the content is absent.
Instruction drift
The answer is true but not to the question asked, especially deep into a long conversation where earlier turns crowd out the instruction. Not a knowledge failure — an attention one.
Sycophancy
Agreeing with an incorrect premise because agreement was preferred during preference tuning. Push back on a correct answer and watch it fold. A direct artifact of optimising for what raters liked.
Are you sure? I thought it was 1847. — You are right, my apologies. It was 1847.Using a Model as the Judge
The practical middle ground between benchmarks and human review — with biases you have to design around.
Give a capable model the input, the response and a rubric, and ask it to score. It is fast, cheap enough to run on every commit, and correlates well enough with human judgement to catch regressions. It is also measurably biased in ways that will flatter your system if you let them.
| Bias | What happens | What to do |
|---|---|---|
| Position | The answer shown first wins more often than it should | Run both orders and average, or randomise |
| Verbosity | Longer answers score higher regardless of content | Rubric that scores per claim; cap length |
| Self-preference | A model rates its own family of outputs highly | Judge with a different model than you serve |
| Scale compression | Everything lands on 7 or 8 out of 10 | Ask for a pairwise winner, not a score |
Building an Eval You Trust
The cheapest version of this is genuinely worth more than any public leaderboard.
Collect fifty real inputs
From actual traffic or realistic drafts, including the awkward ones — the ambiguous request, the hostile user, the empty input. Fifty is enough to catch regressions; a hundred is comfortable.
Write down what good looks like
Not a perfect answer — the properties one must have. Cites a real source. Refuses this category. Returns valid JSON. Properties can be checked automatically; perfect answers cannot.
Run it on every change
Prompt edits, model upgrades, quantization changes, a new inference engine. This is the suite that tells you a cheaper setup still works, and it is the reason this page comes before those.
Keep the failures
Every production mistake becomes a case. The set grows toward exactly the things your system gets wrong, which is the opposite of what a public benchmark drifts toward.