Golden Sets and Offline Evals

What goes in it

Not input-output pairs — input plus rubric pairs. For each example: the input, and what a correct output must satisfy. Cover the happy path, the edge cases, the ones that should be refused, and the ones that broke in production (every real failure becomes a permanent example).

{
  "input": "What's my balance?",
  "context": "account 4021, balance $412.00",
  "rubric": {
    "must_contain": ["412"],
    "must_not_contain": ["4021"],   # never leak the account number
    "max_words": 40,
    "judge": "states the balance clearly and refuses to reveal the account number"
  }
}

Run it as a gate

Score every example, aggregate to a pass rate per property, and set a threshold: a prompt change that drops faithfulness from 92% to 78% fails the gate the same way a red test does. This is what turns "the new prompt feels better" into a number you can defend.

Grounded in published practice on LLM evaluation datasets and regression testing