- 2,013 valid runs
- A run: One agent (Claude alone, Claude with Grace, Grace alone, Claude finishing Grace's result, or a third-party tool) working on one task once, with token and cost readings. A valid run: A run that finished and was measured as planned. Crashed, blocked, cut-off or misconfigured runs are listed with their reason and never count toward any result or total. 2,142 runs were started; 129 of them were invalid. Not included: 107 calibration measurements (single calls without a task), counted separately.
- 273 distinct tasks
- Each task counted once (same catalogue number or same task text) if it has at least one valid run in any series – also in a series that is invalid as a comparison (R73, with 6 valid runs), since its valid runs are in the run count too. Tasks traced back to the raw data of every series; no series was left uncounted. 2 of the 273 appear only in R73.
- 115 subscription tasks
- Counted the same way, over the series set up to read the subscription’s 5-hour or weekly limit (marked “Subscription limit” in the table), plus the series with Grace’s subscription build: R06, R67, R68, R69, R70, R71, R76, R78, R79, R83, R84, R89, R90 and R91. 105 of them have a valid Grace run; the other 10 have only valid Claude-alone runs in these series: Grace’s run crashed at start (2), Grace’s run was not judged (1) or Grace never got to the task because the series was stopped (7). Not counted here: R73, R74, R75, R80, R81 and R82. In these series the test script also logged the 5-hour limit before and after each run, but they were not set up to read it (R75 reports a side finding from it). Counting them too would give 151.
- 205 tasks with API-price cost
- Counted the same way, over the series in which the cost at Anthropic API list prices, computed from the measured tokens, was one of the planned measurements (marked “API cost at list prices” in the table): R01, R02, R03, R06, R10, R11, R14, R16, R19, R22, R23, R24, R25, R26, R27, R28, R29, R30, R31, R32, R33, R34, R35, R36, R39, R40, R44, R45, R46, R48, R52, R53, R55, R56, R57, R58, R59, R62, R64, R65, R66, R67, R68, R69, R70, R71, R74, R76, R78, R79, R80, R81, R82, R83, R84, R85, R87, R88, R89, R90 and R91. 186 of them have a valid Grace run. Grace’s own API build ran only in R80–R82, on 6 tasks. Not counted here: R54, R73 and R75. These series also recorded the cost of each run, but they were not set up to measure it. Counting them too would give 235. All 115 subscription tasks are among them, and 68 tasks are in neither group — the two numbers overlap and do not add up to 273.
- 71 tasks with a released version
- The tasks of the series run with a released version of Grace (the release of 28 September, 0.3.6, 0.3.7 and 0.3.8, including its release candidate): R83, R84, R87, R88, R89, R90 and R91 (271 valid runs, 96 valid pairs of Grace and Claude alone in the same series). Results from different Grace versions are never averaged. Series with released versions of Grace: R83 and R84 (release of 28 Sep 2026), R87 (0.3.6), R88 and R89 (0.3.7), R90 (0.3.8, release candidate with the same behaviour as the release) and R91 (0.3.8); all other Grace series are development builds.
Acceptance criteria, full task texts, project and customer names, file paths and the names of third-party tools are withheld. Raw data per task is published for the fill test of 29 Sep 2026, the two tests of 2 Oct 2026 and the confirmation test of 3 Oct 2026. All series as JSON (104 KB) · Task counts by kind of measurement (JSON, 6 KB)