Method

The baseline is not a stock Claude Code

The global CLAUDE.md (~/.claude/CLAUDE.md) on the machine that generated these cells routes all website and frontend work to two design skills. Every subagent spawned on this machine inherits that file, regardless of its working directory — running Arm A from a scratch directory keeps project CLAUDE.md files and their skills out of scope, but it does not touch the global one.

Arm A therefore represents Claude Code as configured by a developer who already cares about design, not a fresh install. It is naive about this project specifically — no dataviz research, no craft module, no reference images, no project conventions — but it is not a stock Claude Code baseline, and it is not naive about design in general.

Two things follow. First, the gap this gallery shows is a floor, not a ceiling: a stock install would likely start further back. Second, closing this would require generating baselines through the Claude API with a bare system prompt, which costs money and needs explicit approval. Until that happens, the disclosure stands in for the isolation.

A second contamination source: the contract names a library

cells/_contract.md — handed identically to every arm, Arm A included — says: “Use svg.ownerDocument and createElementNS, or a d3 selection over it.” That sentence names a specific library to the naive baseline generator before it has written a line of code.

This is harmless for Phase 1: none of the nine Phase 1 rows is about library choice. It is not harmless for lib-01-ceiling — preset chart library vs. Observable Plot’s function-mark escape hatch vs. raw D3/SVG, all on the same brief — where library choice is the entire subject of the row. A contract that pre-names d3 to that row’s Arm A puts a thumb on exactly the scale it is meant to measure.

Before that row is generated it needs either a contract variant that describes the DOM access pattern (a real <svg> element in a jsdom document) without naming a library, or an explicit caveat attached to that row’s comparison disclosing that Arm A was told about d3 and its competitors were not. It is disclosed here so it is caught before the row ships, not after.

Run counts

Arm A runs three times and the median result ships, because one generation is not “the default”. Median is a mechanical rule, not a judgment call: all three runs are rendered, the run with the median total SVG element count ships, and a tie is broken by the smaller chart.js. Anyone holding the three runs can re-derive the same choice.

That is the point — a subjective “median-looking” pick would let the prettiest baseline ship, which is exactly what this gallery claims not to do. The discarded runs are not committed, so the selection is reproducible from the runs, not from the repository.

Arms B and C run exactly once. Shipping the prettiest of several directed attempts would manufacture the result.

Permitted post-generation edits

Generated code is immutable. Three edits are permitted, applied uniformly to every arm and hash-recorded before they land:

1. Injecting the shared dataset import.
2. Injecting the shared mount point / container id.
3. Removing a hardcoded page background that would fight the gallery shell.

Anything else is a regeneration, versioned -v2, with the original kept. The integrity gate re-hashes chart.js and re-renders every render.svg on each build, so a fourth edit fails the build rather than shipping quietly.

Refine vs from-scratch

Each row declares one. refine rows run the treated arms on the default arm’s output, so the only difference is the lever. from-scratch rows generate independently, because there the lever is the generation decision — library choice, radial vs cartesian, glyph design.

A Phase 1 / Phase 2 prompt difference

cells/_contract.md states four constraints. Phase 1’s prompt templates transmitted only some of them: no arm that generated the first 24 cells was told “do not call document globally”, “no fonts loaded at render time”, or “renders must be byte-stable across runs”. All 24 shipped cells satisfy all three anyway — verified by re-rendering every cell byte-identically and by checking that none loads a font — but that was luck, not instruction.

The templates in the generation runbook now state all four, so Phase 2’s prompts are not byte-identical to Phase 1’s. The addition is a harness constraint, never a design hint, and it is stated here rather than slipped in because Arm A’s prompt is the experiment’s control. Byte-stability in particular is now enforced: the integrity gate re-renders each cell and byte-compares against the committed render.svg.

6 cells whose premise was never engaged

Every treated cell in this gallery differs from its baseline. 6 of them do not differ in the dimension their row’s title names, which is a weaker result than the row claims. They ship anyway, labelled, because a null result is a finding — “the named skill, invoked as its documentation says to, did not touch this lever” is the most load-bearing thing this gallery can report.

The first three passed the Phase 1 gate untouched: every check it ran asked whether a cell was well-formed and honestly provenanced, and none asked whether it engaged the lever its row exists to demonstrate. The gate now runs a premise probe (check 12) that compares each treated arm against its baseline on exactly that dimension and fails when the two are identical. The declaration is enforced in both directions — a cell whose lever did move is not allowed to carry the disclosure — so it cannot go stale. A probe can only prove a lever moved, never that it moved well. It is a floor under the claim these pages make, not a judgement of the design.

• type-04-numerals.B → Tabular vs proportional figures dataviz probe: numeric-format

  dataviz cut 24 numeric annotations down to 2 without ever reaching for font-variant-numeric, font-feature-settings, or a tabular-figure font; both arms keep the same Helvetica stack. That is a real annotation-density edit, not the tabular-vs-proportional lever this row names, which stays untested. The row was deliberately not relabelled to fit the outcome — picking the hypothesis after seeing the result is the move this project argues against.

• mark-07-overlap.B → Overplotting: opacity, jitter, ordering dataviz probe: mark-geometry

  dataviz recoloured every mark and thickened its stroke while leaving all 16 circles' cx/cy/r/fill-opacity byte-identical to Arm A. The cause is upstream of the skill: the hero8 baseline's eight bubbles are already disjoint (smallest clearance about 7px), so there was no overplotting available to fix. Testing this lever needs a fixture that actually overplots, not a prompt that forces an edit onto a chart that does not.

• mark-07-overlap.C → Overplotting: opacity, jitter, ordering /impeccable craft probe: mark-geometry

  /impeccable craft collapsed five secondary text greys into one and changed no circle attribute at all — a typographic-hierarchy edit. Same cause as Arm B: the hero8 baseline does not overplot, so this row's premise was never available to engage on either of its treated arms.

• axis-03-grid.B → Gridlines: whether, how faint, which direction /impeccable distill probe: mark-geometry

  /impeccable distill deleted all twelve delta labels under the bars and shortened the baseline annotation from World baseline: 73.48 to World baseline, cutting the chart from 46 text elements to 34. It changed no gridline at all: the six horizontal rules keep the same positions, the same 1px weight and the same light grey, and the dashed baseline rule and the axis line are byte-identical too. Whether, how faint, which direction is the one thing this row names, and the one thing distill left alone — it read the chart's clutter as annotation density rather than as grid.

• medium-02-theme.B → Dark / light theming of the same chart /impeccable adapt probe: color

  /impeccable adapt read adapt as responsive sizing rather than as theming: it added a preserveAspectRatio attribute and an inline style that lets the chart scale fluidly with whatever container embeds it, and touched not one colour. All 41 fills and strokes are identical to Arm A in document order, the white ground included, so the shipped cell is the same light chart the baseline was. Dark versus light theming is what this row names; the skill's own adapt remit turned out to be a different kind of adaptation, and the theming lever stays untested.

• num-01-units.B → Units and rounding dataviz probe: numeric-text

  dataviz replaced the entire palette, added 12 path elements and moved mark geometry — a large, real edit — but wrote every number exactly as Arm A did: same rounding, same units, same separators, so both arms probe identically on numeric-text and numeric-format. Units and rounding is the one thing this row names and the one thing the skill left alone.

What the 6 of them have in common

Every one is a named skill invoked with no brief — 3 bare dataviz invocations and 3 bare /impeccable ones. The pattern worth stating, and worth holding loosely at n=6: a named skill invoked with no brief reliably changes something — usually palette, annotation density and layout, the things a chart skill reaches for first — and does not reliably change the specific lever a row was built to isolate.

medium-02-theme adds a second mechanism next to that one: a skill’s remit may simply not be the thing the row’s title assumed it was. Asked to adapt a chart, the skill read adapt as responsive sizing and changed not one of 41 colours. A row title can silently encode a guess about what a skill means. Both are claims about how these skills behave unprompted, which is exactly what the default-versus-directed comparison exists to measure. Neither is evidence that the skills are bad at the lever when asked.

Three fixes that were refused

The probe found these cells; it did not get to delete them. Three ways to make the nulls disappear were available, and naming them is part of the disclosure:

1. Regenerating them under lever-forcing prompts. That measures the skill plus an explicit brief while the published method still reads dataviz or /impeccable craft. The whole question these rows ask is what a named skill does when it is simply run — telling it the answer first and publishing the result under the same label would manufacture the finding.
2. Relabelling type-04-numerals as an annotation-density row. Its Arm B edit is real and would fit that title. But choosing the hypothesis after seeing the result is exactly the move this project exists to argue against, and it would quietly retire “tabular vs proportional figures” — a lever nothing has yet tested — from the 48-row inventory. The row keeps its title and keeps its null.
3. Re-running mark-07-overlap against a fixture that overplots. Measured from the committed render.svg, the baseline does not overplot: its eight bubbles sit on one row at cy=210 and every pair is disjoint — the tightest clearance, Jupiter to Saturn, is about 7px. Arm A had also already sorted its marks by descending radius and set fill-opacity before any skill ran. So neither treated arm declined to fix overplotting; there was no overplotting in front of it. That is a defect in the row, not in the skills — testing “opacity, jitter, ordering” needs a fixture that actually overplots, which hero8 at 640×400 is not. Swapping the fixture now would quietly re-run the row against a different chart after seeing the result, so the row is kept as generated with both nulls published, and the fixture defect is recorded here.

Gaps published as gaps

Three techniques are shown as unanswered rather than filled with a guess: the glow / halo filter recipe (its feGaussianBlur + feMerge wiring failed a three-vote verification, 0–3), texture, and legend-as-art. Each of the 3 gap rows states on the gallery what is missing and what would close it.

Claims that failed verification

Listed so no reader picks them up secondhand:

• the glow / halo filter recipe (its feGaussianBlur + feMerge wiring, 0–3)
• a “three annotations, five maximum” density budget
• D3 line-count figures
• a 175-chart / 100%-flawed / 2.47-flaws-per-chart defect baseline
• “conversational refinement mostly fails”
• “the requester’s vocabulary is the binding constraint”
• “models cannot judge aesthetics”
• “nine of ten practitioners export to Illustrator”
• “the annotation layer is what defines editorial graphics”

Each failed a three-vote adversarial check. Failing is not proof of falsehood — it means this project does not rely on them.

Not a benchmark

No scores. No winner declared per row. Some rows show almost no difference; those ship too.