Lesson 0006 · ~20 minutes
"The model should be good at legal work" is untestable. Anthropic's framing (define your success criteria): make it specific and measurable, along multiple dimensions. For Case AI, the job post literally hands you the dimensions — turn each into a number:
| Dimension (their words) | Measurable form |
|---|---|
| Legal-task accuracy | % of golden-set questions answered correctly per task type |
| Citation quality | % of claims with a citation; % of citations whose chunk actually supports the claim |
| Tone | Rubric score (LLM judge, calibrated) for professional register |
| Jurisdiction awareness | % of jurisdiction-trap questions handled correctly (HK vs NY vs Cayman law differ!) |
A golden set is your curated exam: real (anonymized) questions with known-correct answers and the evidence that supports them — including deliberate edge cases: ambiguous questions, questions whose answer is "not in the documents" (the hallucination trap), jurisdiction traps, and adversarial documents containing prompt injection. It lives versioned in the repo, grows from production (every lawyer correction in the HITL queue is a free labeled example — the loop from Lesson 0005), and mirrors the real task distribution (Anthropic: strong empirical evaluations).
Code-based grading — fast, deterministic, cheap; use it wherever possible: does the cited chunk contain the quoted text (string match)? Does the output parse against the schema? Did retrieval return the labeled relevant chunk in the top k (recall@k)? Did the model correctly say "not found" on the trap questions?
LLM-as-judge — for what code can't score: tone, completeness, reasoning quality. A (different) model grades against a written rubric. Three rules that make judges trustworthy: give the judge a rubric with examples, ask it to reason before scoring (measurably better judgments — Anthropic's guidance), and calibrate against human labels on a sample — a judge that disagrees with your lawyers is itself a failed eval.
A wrong answer can come from retrieval (right chunk never arrived) or generation (chunk arrived, model misused it). So test the layers separately, like unit vs integration tests:
Hallucination testing falls out of this structure: generation evals measure the unsupported-claim rate (claims with no supporting chunk), and the golden set's "answer is not in the documents" traps measure whether the model admits ignorance instead of inventing law.
The framework earns its keep on change: every prompt tweak, model upgrade, or pipeline change runs the eval suite — cheap code-graded evals on every PR, judge-graded and end-to-end suites nightly or pre-release. Scores are tracked over time; a drop below threshold blocks the deploy, exactly like a failing test suite. Model upgrades stop being scary ("run the evals, compare, decide") and prompt changes stop being vibes. Close the loop with production: sample real (consented) traffic for spot-checks, and feed lawyer edits back into the golden set.
Interview sound bite: "I'd treat evals like a test pyramid — lots of fast code-graded checks (citation verification, schema, retrieval recall), a layer of calibrated LLM-judge rubrics for quality, a few end-to-end workflow evals — versioned golden set, run in CI, thresholds that block deploys, and lawyer corrections feeding the set."
Read Anthropic's "Create strong empirical evaluations" (~15 min), then skim "Demystifying evals for AI agents" for the multi-step-workflow angle.