Eval Board
Which model to ship, and what is still broken — as a ranking plus a case list rather than a grid. Chips carry their own model name, so there is no column header to align to and nothing that scrolls sideways at four models.
src/components/ui/eval-board.tsxAbout
From the component's own source, where it was written.
Which model to ship, and what is still broken — as a ranking, not a grid.
This replaced a models-across/cases-down matrix, which was the obvious shape and the wrong one. A matrix needs a column per model, so it starts scrolling sideways at three of them; it needs a sticky first column to stay readable while it does; and every cell is a coloured block whose glyph is too small to carry the meaning, so the colour ends up doing all the work and the screen reads as noise.
The two questions are separated instead, because they are asked at different moments and want different shapes:
**Which model?** A ranked list, one row per model, each with a bar segmented by outcome. Comparing four numbers down a column is something people are good at; comparing them across a scrolling header is not.
**What is broken?** A case list, each case carrying one labelled chip per model. The chip names its model, so there is no column header to align to and nothing to scroll — it wraps at any width, on any number of models.
Scores are computed from the results here rather than accepted as a prop. A rate passed in can disagree with the rows beneath it, and when it does nobody notices, because the number is what gets read.
Install
npx astralyx-ui add eval-board
Usage
import { EvalBoard } from '@/components/ui/eval-board'
<EvalBoard models={models} cases={cases} onSelectCase={openCase} />Composer
Change the props and the source below follows.
Models
- Opus 5
4 pass1 fail
80%
- Sonnet 5
3 pass1 partial1 fail
70%
- Haiku 4.5
1 pass1 partial2 fail1 skipped
38%
Cases
- Refuses an over-cap refundOpus 5: PassSonnet 5: PassHaiku 4.5: Fail
- Cites the retrieved documentOpus 5: PassSonnet 5: PartialHaiku 4.5: Partial
- Handles an ambiguous ticketNo model passed
The ticket names two orders and asks about “the refund”.
Opus 5: FailSonnet 5: FailHaiku 4.5: Fail - Escalates a chargebackOpus 5: PassSonnet 5: PassHaiku 4.5: Pass
- Respects British spellingOpus 5: PassSonnet 5: PassHaiku 4.5: Skipped
<EvalBoard
models={models}
cases={cases}
partialWeight={0.5}
/>Examples
Three models, one broken test
<EvalBoard models={models} cases={cases} />API
Generated from the source, so it cannot fall behind the code. One table per exported component, then hooks, functions and types.
EvalBoard
Extends Omit<ComponentProps<'div'>, 'onSelect'>.
| Prop | Type | Default | Description |
|---|---|---|---|
| models | EvalModel[] | required | EvalCase is { id, name, description?, results } keyed by model id. A missing entry is “not run”, which is distinct from skipped. |
| cases | EvalCase[] | required | EvalCase is { id, name, description?, results } keyed by model id. A missing entry is “not run”, which is distinct from skipped. |
| partialWeight | number | 0.5 | What a partial earns when scoring. |
| onSelectCase | (evalCase: EvalCase) => void | — | Makes each case row a button for drilling in. Omit and the rows are static. |
| flagUniversalFailures | boolean | true | A case no model passed is usually a broken test rather than four broken models — flagged inline as a badge, not as a sentence that doubles the row height. |
| universalFailureLabel | string | 'No model passed' | — |
| leaderboardLabel | string | 'Models' | — |
| casesLabel | string | 'Cases' | — |
| emptyLabel | string | 'No cases yet.' | — |
| formatScore | (rate: number, scored: number, total: number) => ReactNode | — | Formats a model's score. Defaults to a whole percentage. |
Types
EvalModel
| Field | Type | Description |
|---|---|---|
| id | string | — |
| name | string | — |
EvalCase
| Field | Type | Description |
|---|---|---|
| id | string | — |
| name | string | — |
| description? | ReactNode | — |
| results | Record<string, EvalOutcome> | Keyed by model id. A missing entry is "not run", distinct from skipped. |
| Type | Definition | Description |
|---|---|---|
EvalOutcome | 'pass' | 'fail' | 'partial' | 'skip' | Which model to ship, and what is still broken — as a ranking, not a grid. |
Behaviour
| Topic | Detail | Description |
|---|---|---|
| why not a matrix | design note | A column per model starts scrolling sideways at three, needs a sticky first column to stay readable, and turns every cell into a coloured block whose glyph is too small to carry meaning. A ranking plus labelled chips wraps at any width. |
| scores | computed here | From the results, never a prop. Skips and not-run are excluded from the denominator — a model is not penalised for a case that never ran against it. |