CyberFreezeDevChoosing a model
Compare hosted and open-weight models on your own data before committing.
Read more about Choosing a model
Comparing a hosted model with an open-weight model on your own data before committing to either.
- Home
- AI glossary
- Model evaluation
What is model evaluation?
Scoring a model on real cases with known answers, every time something changes.
The plain definition
It shows whether one model beats another on your task, whether a prompt helped and whether a change broke something. Public benchmarks cannot.
Read more about The plain definition
Model evaluation is the practice of scoring an AI model, or a whole AI pipeline, against a test set whose correct answers you already know. It tells you whether model A is better than model B for your task, whether a new prompt helped, and whether a change broke something that used to work.
Public benchmarks are a rough guide to general ability. They say little about whether a model reads your vendors' invoices or answers your customers' questions correctly. For that you need your own evaluation set.
How it works
-
01 Collect real cases
Take 50 to a few hundred real inputs, including messy and rare ones.
Read more about Collect real cases
Take 50 to a few hundred real inputs, including the messy and rare ones, and remove personal data you do not need.
-
02 Write the expected result
Correct field values, or a reference answer or checklist for open text.
Read more about Write the expected result
For extraction, the correct field values. For open answers, a reference answer or a checklist of what a good answer must contain.
-
03 Choose a scoring method
Exact match for fields, rules for format, a person or model as judge for text.
Read more about Choose a scoring method
Exact match for fields and labels; rules for format; a person or a strong model acting as judge (LLM-as-judge) for open text.
-
04 Run and record
Rerun the full set on every model, prompt or settings change, and keep results.
Read more about Run and record
Run the full set on every change of model, prompt, quantization or retrieval settings, and keep the results.
Where it matters in business
Safe changes
Catch the prompt tweak that fixes one complaint but breaks twenty answers.
Read more about Safe changes
Catching the case where a prompt tweak fixes one complaint and quietly breaks twenty other answers.
Accountability
A recorded score gives managers and auditors something concrete.
Read more about Accountability
A recorded score gives managers and auditors something concrete, which supports AI governance.
A practical example
An HR assistant was scored on 80 real questions and 15 unanswerable ones. A new embedding model improved retrieval but worsened the "not covered" answers.
Read more about A practical example
A team building a RAG assistant for HR policies writes 80 real staff questions with the correct policy clause for each, plus 15 questions the policies do not answer. Each run scores two things: did retrieval return the right clause, and did the answer match it (or correctly say "not covered")? When they swap embedding models, retrieval improves but the "not covered" cases get worse, a trade-off nobody would have noticed by trying a few questions by hand.
Common mistakes
Only easy cases
Tidy test sets give high scores and no information. Include inputs that cause trouble.
Read more about Only easy cases
A test set of tidy examples gives high scores and no information. Include the inputs that cause trouble in production.
Testing on training data
Keep fine-tuning and prompt examples out of the test set.
Read more about Testing on training data
If examples were used for fine-tuning or as prompt examples, keep them out of the test set.
Unchecked AI judges
Model judges have biases, such as favouring longer answers. Spot-check them against a person.
Read more about Unchecked AI judges
LLM-as-judge is useful but has biases, for example favouring longer answers. Spot-check its scores against a person's.
Questions
How many test cases do we need?
Enough to cover the real variety of inputs, including rare and difficult ones. For a narrow business task, a carefully chosen set of 50 to 200 cases is a practical start; grow it with every production error you find.
Which tools are used for evaluation?
Many teams start with a spreadsheet and a script. Open-source frameworks such as promptfoo, Ragas (for RAG) and OpenAI Evals add structure once the set grows.
Last updated