First month free · then $19.99 a month · No VAT (§ 19 UStG) First month free · then $19.99 a month

See pricing
Measurements

Measured, task by task.

On October 2 – 3, 2026 we measured Grace 0.3.8 against Claude Code + caveman + context-mode on 60 tasks in two tests, on Claude Max 20x. Here are the results, the method and the raw data per task. Grace 0.3.7 on 2 October and our measurement of 29 September 2026 are shown as earlier measurements.

of the 5-hour limit for the same 60 tasks — Grace 0.3.8, read from the limit, two tests added up: 30 tasks at the same time, 30 with the comparison run measured separately ≈ −49 % October 2 – 3, 2026
5-hour limit used, with Grace 0.3.8 vs Claude alone 32 % vs 63 % read in whole percentage points, both tests added up
quality on the 58 tasks both runs finished, with Grace 0.3.8 vs Claude alone 92.5 % vs 91.9 % hidden acceptance criteria met
less time, Grace 0.3.8 vs Claude alone at the same time — faster on 28 of 30 tasks −49 % simultaneous test, 30 of the 60 tasks, October 3, 2026

Claude Max 20x on every account · 3 Claude Code sessions in parallel per account · baseline: Claude Code with the token-saving plugins caveman and context-mode, not bare Claude · 5-hour limit read in whole percentage points.

A gap in the readings: One rule fixed before the start was missed: on the baseline account, the 54 seconds before a 5-hour reset were not covered by a reading, so usage in those seconds is missing from the baseline's 31 %. That can only make the baseline's share look smaller (about 0.2 points at its average rate), never Grace's. Every rule on the result itself (saving, quality, crashes, time) was met. Details

2–3 October 2026: Grace 0.3.8 on 60 tasks, and an earlier version

All compared with Claude Code + caveman + context-mode, all on Claude Max 20x. The headline on this site adds up the two Grace 0.3.8 tests (box below): 30 tasks at the same time, and 30 tasks with the comparison run measured separately. Grace 0.3.7 ran the same 30 tasks at the same time on 2 October — an earlier version, shown for comparison.

Grace 0.3.8 · both tests together · 60 tasks

≈ −49 % of the 5-hour limit · October 2 – 3, 2026

headline on this site
5-hour limit
18 % + 14 % = 32 % with Grace 0.3.8, 32 % + 31 % = 63 % for Claude Code + caveman + context-mode, 60 tasks: 1 − 32 ÷ 63 ≈ −49 %. All four values are read from the limit in whole percentage points and added up; nothing is subtracted. 30 tasks were measured with the comparison run separately on 2 October, 30 other tasks at the same time on two accounts in the night to 3 October.
Rounding
Each sum carries up to ±1 point of rounding. At those edges the result lies between about −47 % and −52 %.
Quality
92.5 % vs 91.9 % (with Grace vs Claude alone) over the 58 tasks both runs finished. Clearly worse with Grace (more than 10 points): G020 (Post scheduler for several accounts – build stage 1, extend stage 2 past midnight: 5 % with Grace, 86 % Claude alone) and E004 (ISO duration P1D silently becomes 0: 86 % with Grace, 100 % Claude alone).
Time
Only from the test at the same time: −49 %, Grace faster on 28 of 30 tasks. The other 30 tasks were measured separately and are not compared on time.
A gap in the readings
One rule fixed before the start was missed: on the baseline account, the 54 seconds before a 5-hour reset were not covered by a reading, so usage in those seconds is missing from the baseline's 31 %. That can only make the baseline's share look smaller (about 0.2 points at its average rate), never Grace's. Every rule on the result itself (saving, quality, crashes, time) was met. Because of it, the rules we fixed before the test count it as formally invalid; we use it anyway and show the gap here.

Grace 0.3.8 · at the same time · 30 other tasks

≈ −55 % of the 5-hour limit · October 3, 2026

5-hour limit
14 % read from the limit with Grace 0.3.8, 31 % for Claude Code + caveman + context-mode — both accounts in the same hours, same tasks, same order, no other usage on either account: 1 − 14 ÷ 31 ≈ −55 %.
Rounding
Readings come in whole percentage points. At the edges of that rounding the result lies between about −52 % and −57 %.
A gap in the readings
One rule fixed before the start was missed: on the baseline account, the 54 seconds before a 5-hour reset were not covered by a reading, so usage in those seconds is missing from the baseline's 31 %. That can only make the baseline's share look smaller (about 0.2 points at its average rate), never Grace's. Every rule on the result itself (saving, quality, crashes, time) was met.
Quality
93 % of the hidden acceptance criteria with Grace 0.3.8, 90 % Claude alone, over the 29 tasks both runs finished. Clearly worse with Grace (more than 10 points): E004 (ISO duration P1D silently becomes 0: 86 % with Grace, 100 % Claude alone). Not compared: X003 — the Claude-alone run stopped with an API error before the end.
Time
3 h 41 min with Grace, 7 h 10 min Claude alone — all 30 tasks, added up: −49 %. Grace was faster on 28 of 30. Measured at the same time, so fair to compare.
Weekly limit
2 % with Grace against 5 % Claude alone — too coarse for a ratio of its own.
Cost at API list prices
$39.76 with Grace 0.3.8, $95.10 Claude alone, all 30 tasks.

Grace 0.3.8 · measured separately

≈ −44 % of the 5-hour limit · October 2, 2026

5-hour limit
18 % with Grace 0.3.8, read from the limit; 32 % for Claude Code + caveman + context-mode on the same 30 tasks: 1 − 18 ÷ 32 ≈ −44 %. The comparison run used the same task set on the same day, measured separately (not at the same time). Grace ran alone on one account whose 5-hour window started at 0 %.
Rounding
Readings come in whole percentage points. At the edges of that rounding the result lies between about −41 % and −46 %.
Quality
92 % of the hidden acceptance criteria with Grace 0.3.8, 94 % Claude alone, over the 29 tasks both runs finished. Clearly worse with Grace (more than 10 points): G020 (Post scheduler for several accounts – build stage 1, extend stage 2 past midnight: 5 % with Grace, 86 % Claude alone).
Time
Not compared. The Grace 0.3.8 run and the Claude-alone run were measured at different hours, so their times are not comparable. The time comparison below comes from the simultaneous test.
Weekly limit
2 % with Grace against 5 % Claude alone — too coarse for a ratio of its own.
Cost at API list prices
$34.41 with Grace 0.3.8, $96.77 Claude alone, all 30 tasks — Claude Code’s own figure per run; both accounts are subscriptions and paid nothing per token.

Grace 0.3.7 · at the same time · earlier version

≈ −31 % of the 5-hour limit · October 2, 2026

5-hour limit
23 % read from the limit with Grace 0.3.7, 32 % for Claude Code + caveman + context-mode — both accounts in the same hours, same tasks, same order. Right before the last reading the Grace account also served about 1.0 points of other usage (counted from its session logs, like on 29 September); without it 22.0 against 32: ≈ −31 %, calculated.
Rounding
Readings come in whole percentage points. At the edges of that rounding the result lies between about −29 % and −34 %.
Quality
95 % vs 94 % (with Grace vs Claude alone) over the 29 tasks both runs finished. Clearly worse with Grace (more than 10 points): E031 (Minecraft 1.21.1 datapack: 82 % with Grace, 100 % Claude alone).
Time
5 h 15 min with Grace, 7 h 22 min Claude alone — all 30 tasks, added up: −29 %. Grace was faster on 28 of 30. Measured at the same time, so fair to compare.
Weekly limit
4 % with Grace against 5 % Claude alone (as read, other usage included) — too coarse for a ratio of its own.
Cost at API list prices
$57.30 with Grace 0.3.7, $96.77 Claude alone, all 30 tasks.

2–3 October, level by level

6 tasks per level. Quality over the tasks both runs finished; Claude’s cost at API list prices over all tasks; time only for the simultaneous test — the Grace 0.3.8 run was measured at different hours, so its times are in the download but not compared here.

Results of 2 October 2026 by level: quality, cost and time per run
Level Quality Cost at API list prices Time (same hours)
0.3.8 0.3.7 alone 0.3.8 0.3.7 alone 0.3.7 alone
Easy 99 % 96 % 99 % $3.71 $4.86 $6.97 27 min 33 min
Medium 94 % 96 % 89 % $4.00 $5.85 $8.62 32 min 41 min
Hard 98 % 98 % 96 % $6.98 $12.90 $18.72 72 min 88 min
Extreme 90 % 91 % 89 % $12.98 $21.63 $41.05 117 min 183 min
Mixed 78 % 94 % 96 % $6.75 $12.06 $21.40 67 min 96 min
All 30 tasks 92 % 95 % 94 % $34.41 $57.30 $96.77 315 min 442 min

Grace figures below Claude alone are marked with a dotted underline. The “alone” quality column is compared with each Grace run on the tasks both finished, so it can differ slightly between the two pairings. G028: the Claude-alone run stopped before the end — not scored.

Test at the same time, October 3, 2026: Grace 0.3.8, 30 other tasks

Results of October 3, 2026 by level: quality, cost and time, Grace 0.3.8 and Claude alone at the same time
Level Quality Cost at API list prices Time (same hours)
0.3.8 alone 0.3.8 alone 0.3.8 alone
Easy 95 % 99 % $1.66 $4.31 10 min 20 min
Medium 98 % 90 % $4.65 $12.60 25 min 59 min
Hard 94 % 96 % $7.20 $17.53 45 min 84 min
Extreme 85 % 88 % $19.11 $49.08 98 min 208 min
Mixed 90 % 77 % $7.14 $11.58 43 min 59 min
All 30 tasks 93 % 90 % $39.76 $95.10 221 min 430 min

X003: the Claude-alone run stopped with an API error before the end — not compared on quality.

Where Grace did worse

More than 10 points below Claude alone with Grace 0.3.7: E031 (Minecraft 1.21.1 datapack: 82 % with Grace, 100 % Claude alone). With Grace 0.3.8: G020 (Post scheduler for several accounts – build stage 1, extend stage 2 past midnight: 5 % with Grace, 86 % Claude alone). With Grace 0.3.8 in the test at the same time (October 3, 2026): E004 (ISO duration P1D silently becomes 0: 86 % with Grace, 100 % Claude alone).

Method, 2–3 October 2026

Tasks

30 tasks, 6 per level (easy, medium, hard, extreme, mixed), from our own task catalogue. None of them had been run with Grace before. Each run started in an empty folder; some tasks give starter code in the task text.

Simultaneous test

Grace 0.3.7 on account A and Claude Code + caveman + context-mode on account B received the same tasks in the same order at the same time, 11:02–13:36 CEST, 3 sessions in parallel per account. Both accounts on Claude Max 20x.

Separate run

Grace 0.3.8 (the release candidate, with the same behaviour as the release) ran the same 30 tasks the same afternoon, 15:58–17:04 CEST, alone on one Claude Max 20x account whose 5-hour window started unused. It is compared with the Claude-alone run of the simultaneous test — same task set, same day, measured separately (not at the same time).

Test at the same time

Grace 0.3.8 on account A and Claude Code + caveman + context-mode on account B received 30 other tasks (6 per level, from our test of Grace 0.3.6 on 1 October; none of them in the tests above) in the same order at the same time, 01:41–04:29 CEST on October 3, 2026, 3 sessions in parallel per account, both 5-hour windows unused at the start.

Rules fixed before the test

Before it started we wrote down when this test would confirm the Grace 0.3.8 result: no gaps in the readings, little other usage, a saving of at least 38 %, quality of at least 90 % and at most 3 points below Claude alone, at most 2 tasks clearly worse, no Grace crash, not slower. All rules on the result were met. One rule fixed before the start was missed: on the baseline account, the 54 seconds before a 5-hour reset were not covered by a reading, so usage in those seconds is missing from the baseline's 31 %. That can only make the baseline's share look smaller (about 0.2 points at its average rate), never Grace's. Every rule on the result itself (saving, quality, crashes, time) was met.

Headline: 60 tasks

The headline adds up both Grace 0.3.8 tests: the separate run on 2 October and the test at the same time on October 3, 2026 — the 5-hour readings of each side summed, then compared. Time comes only from the test at the same time.

Readings

Each account's share of its 5-hour and weekly limit was read every 5 minutes from the account's own usage display, in whole percentage points. All readings are in the downloads.

Quality

Every task has acceptance criteria written with the task and kept secret from all runs. A separate judging session read each result against them, criterion by criterion: met counts 1, partly met ½, not met 0. Only tasks both runs finished are compared. Not scored: G028 — the Claude-alone run stopped before the end, so there is no pair. Not compared in the test at the same time: X003 — the Claude-alone run stopped with an API error; its partial result is in the download.

What the result is

A share of the 5-hour limit for the same 60 tasks. It is neither a cost nor a count of what Claude wrote. Anthropic does not publish how the limit is counted; we report what the usage display showed.

Every task with every run — quality, cost at API list prices, time and the number of tokens Claude wrote — plus every usage reading per account. The figures in this section are calculated from the same files.

Grace 0.3.7 vs Claude alone, same hours (JSON, 23 KB) Grace 0.3.8, measured separately (JSON, 18 KB) Grace 0.3.8 vs Claude alone, same hours, 30 other tasks (JSON, 25 KB)

Acceptance criteria and full task texts are withheld so the tasks stay usable for future tests.

Calculated, not measured

Only Claude Max 20x was measured, and only for the test as a whole. The plan figures on the home page are calculated from the headline run, as a range: the lower bound from its share of the 5-hour limit, the upper bound from the tasks on which Claude’s cost with Grace fell by half or more. Neither bound is itself a measurement.

Lower bound

The same 60 tasks used 32 % of the 5-hour limit with Grace 0.3.8 and 63 % with Claude alone: ≈ 49 % less, two tests added up (one at the same time, one with the comparison run measured separately). The same limit then holds 1 ÷ (1 − 0.49) ≈ 1.961 times the work — the lower end of 1.96–2× on the Pro card. This bound is calculated, not read from the limit.

Upper bound

On 43 of the 58 tasks both runs finished, Claude’s cost with Grace 0.3.8 was at most half of Claude alone’s: E002, E006, E013, E015, E019, E020, E031, G008, G010, G017, G019, G022, G025, G040, M009, M014, M018, M029, M034, M036, M038, M039, S002, S004, S005, S008, S009, S010, S011, S013, S022, S026, X004, X006, X008, X010, X013, X016, X018, X023, X030, X031 and X032. Where the cost halves, the limit holds 1 ÷ (1 − 0.50) = 2 times the work — assuming the limit counts a task roughly in proportion to its cost, which Anthropic does not publish. Calculated from cost, not measured on the limit.

Not counted, although the cost also stayed at or below half, because Grace’s quality there was more than 10 points below Claude alone: G020 and E004 (quality 45 % with Grace, 93 % alone).

Quality on those 43 tasks: 94 % with Grace, 93 % alone.

Cost is Claude Code’s own figure for each run at Anthropic’s API list prices — neither account paid per token. It is in the raw data as claudeCostUsd.

Work per 5-hour window, Pro = 1×, calculated as a range: lower bound from the headline run’s share of the limit, upper bound from cost
Plan without Grace with Grace, lower bound with Grace, upper bound
Pro 1× 1.96× 1 × 1.961 2× 1 × 2
Max 5x 5× 9.8× 5 × 1.961 10× 5 × 2
Max 20x measured on this plan 20× 39× 20 × 1.961 40× 20 × 2

Work per 5-hour window, Pro = 1×. The plan cards show each row as one range from the lower to the upper bound (1.96–2× on Pro). Anthropic sells Max 5x and Max 20x as 5 and 20 times the usage of Pro (anthropic.com/pricing, September 22, 2026). The plans scale the same limit, so we expect the ratio to carry over, but Pro and Max 5x have not been measured separately.

Earlier measurement

29 September 2026, an earlier version of Grace

Two Claude Max 20x accounts ran the same 34 tasks at the same time on September 29, 2026 — one with Grace, one without. This was the release version of that day, not today’s Grace; it is no longer the headline on this site.

of the 5-hour limit for the same task list, read from the limit (two Grace runs crashed at start) — 29 Sep 2026, earlier version−43 %
5-hour limit used, with Grace vs without25 % vs 44 %
weekly limit used, with Grace vs without4 % vs 7 %
quality on the 31 tasks both runs finished91 % vs 94 %

Claude Opus 5.5 in Claude Code on both accounts · 3 sessions in parallel per account · baseline: Claude with the token-saving plugins caveman and context-mode, not bare Claude.

Raw data of 29 September (JSON, 31 KB)

29 September: the result, side by side

The same 34 tasks on both accounts: how much of the 5-hour and the weekly limit each one used, and how many of the hidden acceptance criteria the results met.

Result of the fill test of 29 September 2026, Claude alone vs with Grace
Measure Claude + caveman + context-mode With Grace
5-hour limit used for the same 34 tasks44 %25 %
Weekly limit used7 %4 %
Quality: hidden criteria met (31 tasks both runs finished)94 %91 %

The 5-hour window over time

Both accounts started with an unused 5-hour window and the same task list. Every 5 minutes, the share of the limit each account had used was read. The dashed line at 40 % is the stop rule: once Claude alone reached it, no new tasks were started.

Share of the 5-hour limit used over 3 h 25 min: Claude alone rose to 44 %, with Grace to 25 %.

September 29, 2026, 03:41–07:06 CEST · readings every 5 minutes · both accounts Claude Max 20x

All 42 readings
Time (CEST) Minute With Grace Claude + caveman + context-mode
03:41 0 0 % 0 %
03:46 5.1 0 % 1 %
03:51 10.2 1 % 2 %
03:56 15.3 1 % 3 %
04:02 20.4 2 % 4 %
04:07 25.5 2 % 5 %
04:12 30.6 3 % 7 %
04:17 35.7 3 % 8 %
04:22 40.8 4 % 9 %
04:27 45.9 5 % 10 %
04:32 51 6 % 11 %
04:37 56.1 7 % 13 %
04:42 61.2 7 % 13 %
04:47 66.3 8 % 14 %
04:53 71.4 9 % 16 %
04:58 76.5 9 % 17 %
05:03 81.6 10 % 18 %
05:08 86.7 10 % 18 %
05:13 91.8 11 % 20 %
05:18 96.9 12 % 22 %
05:23 102 12 % 22 %
05:28 107.1 13 % 23 %
05:33 112.2 13 % 25 %
05:38 117.3 14 % 26 %
05:44 122.4 15 % 27 %
05:49 127.5 15 % 28 %
05:54 132.6 15 % 29 %
05:59 137.7 16 % 31 %
06:04 142.8 17 % 32 %
06:09 147.9 17 % 33 %
06:14 153 18 % 34 %
06:19 158.1 19 % 35 %
06:24 163.2 20 % 36 %
06:29 168.2 21 % 37 %
06:35 173.3 22 % 38 %
06:40 178.4 22 % 40 %
06:45 183.5 23 % 40 %
06:50 188.6 23 % 42 %
06:55 193.7 24 % 42 %
07:00 198.8 24 % 44 %
07:05 203.9 25 % 44 %
07:06 204.5 25 % 44 %

29 September: quality and time, level by level

Quality is the share of a task's secret acceptance criteria that a result met. Over the 31 tasks both runs finished, Grace reached 91 % and Claude alone 94 %. Time was about the same: 471 min with Grace and 498 min alone, summed over those tasks.

Quality and time by level, over the tasks both runs finished
Level Scored Quality Time
Grace alone Grace alone
Easy 6 of 6 99 % 98 % 27 min 19 min
Medium 6 of 7 98 % 95 % 67 min 75 min
Hard 7 of 7 90 % 95 % 101 min 105 min
Extreme 5 of 7 76 % 88 % 165 min 192 min
Mixed 7 of 7 88 % 92 % 111 min 106 min
All levels 31 of 34 91 % 94 % 471 min 498 min

Means and sums over the tasks both runs finished. Grace figures below Claude alone are marked with a dotted underline.

Where Grace did worse

Grace's quality was lower on hard, extreme and mixed tasks, and furthest behind on extreme ones: 76 % against 88 % over the 5 tasks both runs finished. The widest single gap is X008 (2D retro game maker with editor, test mode and headless replay): 45 % with Grace, 86 % alone.

Easy tasks took longer with Grace: 27 min against 19 min over the 6 easy tasks. On extreme tasks it went the other way: 165 min against 192 min over 5.

API cost: new builds with Grace 0.3.8

With an API key or account the question is cost, not a limit: the tokens of every run, priced at Anthropic’s API list prices. For new builds these are the same 60 tasks as the 5-hour headline (2–3 October). The runs used subscriptions; in API mode Grace uses the same flow for new projects. Baseline: Claude Code with the token-saving plugins caveman and context-mode, on the same tasks.

New builds with Grace 0.3.8 · October 2 – 3, 2026

60 tasks in two tests: 30 with the Grace run measured separately against the baseline run of the same day, 30 with both accounts at the same time. Claude’s cost calculated from each run’s measured tokens at Anthropic API list prices.

cost at Anthropic API list prices, all 60 tasks≈ −61 %
the same, only the 30 tasks measured at the same time≈ −58 %
quality on the 58 tasks both runs finished, Grace vs Claude alone92.5 % vs 91.9 %

$74.18 with Grace, $191.88 Claude alone over all 60 tasks; $39.76 against $95.10 on the 30 measured at the same time. Clearly worse with Grace (more than 10 points): G020, E004. Per-task costs are in the downloads above.

Changes to existing code · earlier measurement, September 28, 2026

6 tasks · API mode, interim result of September 28, 2026, not the release version and not Grace 0.3.8. There is no newer API measurement for changes to existing code; its baseline was Claude alone with caveman and context-mode on the same tasks.

cost at Anthropic list prices−48 %
quality with Grace95–100 %
faster than Claude alone2.7×

Cost from each run’s token counts at Anthropic list prices.

Method, 29 September

The subscription test of September 29, 2026, point by point.

Setup

Two Claude Max 20x accounts, both running Claude Opus 5.5, 3 sessions in parallel per account. Account A ran Grace (the release version of that day), started by the test script. Account B was Claude alone in Claude Code. Both received the same tasks in the same order at the same time, each in an empty folder, starting at 03:41 CEST.

Baseline

Claude alone was not bare Claude: it ran with the token-saving plugins caveman and context-mode. A saving against that is a saving on top of what careful users already do.

Stop rule

The test ran until account B reached 40 % of its 5-hour limit. Runs already started were allowed to finish. By then both accounts had started the same 34 tasks.

Readings

Each account's share of its 5-hour limit was read every 5 minutes from 03:41 to 07:06 CEST — 42 readings per account, all in the chart above and in the download. The weekly limit was read at the start and the end: account A 69 % → 73 %, account B 70 % → 77 %. The quality judging ran after the last reading, so it is in neither account's numbers.

Settings

In this test, Grace used the project’s settings. The product uses your user settings instead (since plugin 0.3.2).

Quality

Every task has acceptance criteria that were written with the task and kept secret from both runs. A separate Claude Opus 5.5 session judged each result against them, criterion by criterion: met counts 1, partly met ½, not met 0. Quality is the average over a task's criteria. Only tasks both runs finished normally are compared.

What the result is

The share of the 5-hour limit each account used for the same list of 34 tasks (2 Grace runs crashed at start — see Left out): 25 % with Grace, 44 % without. 1 − 25 ÷ 44 ≈ 43 % less. The weekly limit: 4 % against 7 %. Time: about the same. The result is a share of the limit; it is neither a cost nor a count of what Claude wrote.

Every one of the 34 tasks with both runs — quality, Claude's output tokens, requests, cost at API list prices and time — and all 42 limit readings per account are in the raw data. The level-by-level figures on this page are calculated from the same file.

Raw data per task (JSON, 31 KB)

Acceptance criteria and full task texts are withheld so the tasks stay usable for future tests. Error messages are replaced by a short reason.

29 September: left out, and what it means

3 of the 34 tasks are not in the comparison, because one of the two runs did not finish normally. Why, run by run, is below; their numbers are in the raw data.

  • X002 Finance dashboard for small businesses with tenant separation · extreme

    Grace: Grace run crashed at start (database lock with parallel sessions, fixed since) — not scored

    Claude alone: finished and scored, but left out of the comparison: the Grace run of this task crashed at start

  • M021 Speed up the monthly report query · medium

    Grace: Grace run crashed at start (database lock with parallel sessions, fixed since) — not scored

    Claude alone: finished and scored, but left out of the comparison: the Grace run of this task crashed at start

  • X003 Pet-supplies online shop from scratch · extreme

    Grace: finished, but not scored: the Claude-alone run of this task was stopped, so there is no pair to compare

    Claude alone: stopped by Anthropic's content filter — not scored

The two Grace runs that crashed at start (X002 and M021) used almost none of Grace's limit, while Claude alone finished those two tasks inside its 44 %. They were 9.2 % of Claude alone's cost in this test. If the 5-hour limit counts roughly in proportion to cost — Anthropic does not publish how it counts — Claude alone would have used about 39.9 % without them, and the result would be about −37 % instead of −43 %: a lower bound of 1.59× instead of 1.75× on the Pro card. Compared by cost on the 31 tasks both runs finished, Claude's cost with Grace was 45 % lower. Until 2 October 2026 the headline on this site used ≈ −45 %: the value read from the limit with the other usage on the Grace account taken out (below), marked as calculated. The −43 % is the value read from the limit; the ≈ −37 % adjusts for the crashed runs instead.

Three ways to read the same result

−43 % Read from the limit
25 % with Grace against 44 % alone, as the usage readings showed. Measured.
≈ −45 % Other usage on the Grace account taken out headline until 2 October
Right before the last reading, the Grace account also served a chat session and a memory plugin: $1.83 of Claude usage at list prices, counted once per reply in the account's session logs. All Grace runs ($71.40) plus this $1.83 make up the 25 % read, so the other usage is about 0.6 points. Without it: about 24.4 % against 44 %. Calculated, assuming the limit counts roughly in proportion to cost; the crashed runs stay in.
≈ −37 % Crashed Grace runs taken out
Claude alone without the tasks of the two Grace runs that crashed at start: about 39.9 % against 25 % (see above). Calculated from cost.

What the 29 September measurement cannot tell you

One night, two accounts, one model

One test run on September 29, 2026, 03:41–07:06 CEST, on two Claude Max 20x accounts with Claude Opus 5.5 only. A repeat would not give exactly the same numbers.

Whole percentage points

The usage readings come in whole percentage points, so 25 % and 44 % are rounded. Taken as read, the result is −43 %; at the edges of that rounding it lies between about −41 % and −45 %. The weekly readings (4 % against 7 %) are too coarse for a ratio of their own.

Quality is a judgement

The reviewer read each result's code against the criteria; it did not run it. A criterion it judged met can still fail when the code runs, for either account.

Our own catalogue, empty folders

All 34 tasks come from our own task catalogue, not a public benchmark, and each run started in an empty folder. Some tasks — bug fixes, repairs, a port — give their starter code in the task text. Changes inside an existing project are covered by the API measurement above.

The outer Claude Code session is not in it

Account A's runs were started by the test script, not typed into a Claude Code session. A real user also has that session around Grace's work — the task, Grace's progress messages and the short report at the end, each carrying the full Claude Code context. It is not in the 25 %. We estimate the everyday saving at about −36 % to −41 % of the 5-hour limit instead of the −43 % read — estimated, not measured. The estimate starts from the value read, with the other usage on the Grace account still in it; its allowance for plugin texts was worked out with hooks still on, so it is on the high side.

The limit is Anthropic’s

Anthropic does not publish how the 5-hour limit is counted. We report what each account's usage readings showed and nothing beyond that.

All measurement series

2,013 valid runs on 273 distinct tasks — 115 subscription tasks, 205 tasks with API-price cost (6 in Grace’s API mode; the groups overlap) — 13 Sep – 3 Oct 2026; 71 tasks with a released version of Grace.

The result figures on the measurements page come from R90 and R91 (Grace 0.3.8, 2–3 Oct 2026: 5-hour limit and quality over 60 tasks; time only from R91, measured at the same time); R89 (Grace 0.3.7), R84 (fill test), R83 (large test) and R82 (API interim build) are shown there as earlier measurements. The total number of runs does not prove the saving; it shows how much was measured in total – including everything that did not work. Our own measurements. Every series from the first day on is in the download below — the ones where Grace did worse, the dropped development builds and the invalid runs, each with its reason.

What each number counts

2,013 valid runs
A run: One agent (Claude alone, Claude with Grace, Grace alone, Claude finishing Grace's result, or a third-party tool) working on one task once, with token and cost readings. A valid run: A run that finished and was measured as planned. Crashed, blocked, cut-off or misconfigured runs are listed with their reason and never count toward any result or total. 2,142 runs were started; 129 of them were invalid. Not included: 107 calibration measurements (single calls without a task), counted separately.
273 distinct tasks
Each task counted once (same catalogue number or same task text) if it has at least one valid run in any series – also in a series that is invalid as a comparison (R73, with 6 valid runs), since its valid runs are in the run count too. Tasks traced back to the raw data of every series; no series was left uncounted. 2 of the 273 appear only in R73.
115 subscription tasks
Counted the same way, over the series set up to read the subscription’s 5-hour or weekly limit (marked “Subscription limit” in the table), plus the series with Grace’s subscription build: R06, R67, R68, R69, R70, R71, R76, R78, R79, R83, R84, R89, R90 and R91. 105 of them have a valid Grace run; the other 10 have only valid Claude-alone runs in these series: Grace’s run crashed at start (2), Grace’s run was not judged (1) or Grace never got to the task because the series was stopped (7). Not counted here: R73, R74, R75, R80, R81 and R82. In these series the test script also logged the 5-hour limit before and after each run, but they were not set up to read it (R75 reports a side finding from it). Counting them too would give 151.
205 tasks with API-price cost
Counted the same way, over the series in which the cost at Anthropic API list prices, computed from the measured tokens, was one of the planned measurements (marked “API cost at list prices” in the table): R01, R02, R03, R06, R10, R11, R14, R16, R19, R22, R23, R24, R25, R26, R27, R28, R29, R30, R31, R32, R33, R34, R35, R36, R39, R40, R44, R45, R46, R48, R52, R53, R55, R56, R57, R58, R59, R62, R64, R65, R66, R67, R68, R69, R70, R71, R74, R76, R78, R79, R80, R81, R82, R83, R84, R85, R87, R88, R89, R90 and R91. 186 of them have a valid Grace run. Grace’s own API build ran only in R80–R82, on 6 tasks. Not counted here: R54, R73 and R75. These series also recorded the cost of each run, but they were not set up to measure it. Counting them too would give 235. All 115 subscription tasks are among them, and 68 tasks are in neither group — the two numbers overlap and do not add up to 273.
71 tasks with a released version
The tasks of the series run with a released version of Grace (the release of 28 September, 0.3.6, 0.3.7 and 0.3.8, including its release candidate): R83, R84, R87, R88, R89, R90 and R91 (271 valid runs, 96 valid pairs of Grace and Claude alone in the same series). Results from different Grace versions are never averaged. Series with released versions of Grace: R83 and R84 (release of 28 Sep 2026), R87 (0.3.6), R88 and R89 (0.3.7), R90 (0.3.8, release candidate with the same behaviour as the release) and R91 (0.3.8); all other Grace series are development builds.

Not in these numbers

  • Grace's predecessor as a standalone agent (14–17 Sep 2026): about 210 runs on 18 website tasks. Not in the totals: a different product, not compared with Claude Code and not checked run by run for validity.
  • Judge and analysis calls (several thousand single verdicts). They score runs but are not runs themselves.
  • Manual runs without surviving raw data (including R20 and a mini probe for R72).
  • Development before 13 Sep 2026 (different product stage).

Acceptance criteria, full task texts, project and customer names, file paths and the names of third-party tools are withheld. Raw data per task is published for the fill test of 29 Sep 2026, the two tests of 2 Oct 2026 and the confirmation test of 3 Oct 2026. All series as JSON (104 KB) Task counts by kind of measurement (JSON, 6 KB)

What would make a result invalid

  • Tasks or acceptance criteria changed after seeing a result
  • Runs left out without saying which and why
  • A different baseline than the one named on this page
  • A saving reported without the quality of the same tasks

Something here does not add up? Write to taha.aslan08@gmail.com.

Try it on your own work.

First month free, then $19.99 a month. Cancel anytime. No VAT (§ 19 UStG).

Try 30 days free Just create an account · no payment details · ends on its own

Continue on your computer

Grace runs on your Mac or Windows PC. Copy the link and send it to yourself — by email, a note or a message — then open it there.