React Eval Scorecard
A prompt or model change is a hypothesis, and this is the result of the experiment. The headline pass rate answers “did it get better”; the per-criterion breakdown answers “better at what, worse at what”; and the delta against a baseline is what actually gates the ship decision.
prompt v7 vs v6 — fab-assistant suite
1,240 cases
Installation
Props
| Prop | Type | Description |
|---|---|---|
| name | string | What was evaluated |
| passRate | number | Headline rate, 0–1 |
| baseline | number | Previous pass rate, for the delta |
| samples | number | Number of eval cases |
| criteria | EvalCriterion[] | { label, score, baseline? } |
| regressionEpsilon | number | Regression threshold (default 0.005) |
The aggregate hides the regression
The number that ships a change is the pass rate, and it lies by omission. A prompt tweak that raises correctness from 90% to 94% looks like a clear win — until you notice it dropped "refuses unsafe requests" from 97% to 91%, which is exactly the trade no one meant to make. So the scorecard scores each criterion separately against its own baseline, and when any regresses past a threshold it says so explicitly, in a banner, even though the headline rose. That one guardrail turns a scorecard from a vanity metric into a real gate.
Deltas render as percentage points with a direction icon, in tabular-nums so a column of them stays aligned. Feed it the output of your eval harness — box plots and histograms cover the distribution view when you need to see why a criterion moved, not just that it did.
New components every week
Get the week's new Kelvin UI components and templates in one short email. No spam, unsubscribe anytime.