Measurements · token savers

Token savers for Claude Code, measured

Plenty of tools promise to make your Claude plan go further. This page lists what has actually been measured: our own test, and measurements published by others — each with who measured, how, and a link.

As of 3 October 2026 Diese Seite auf Deutsch

Our measurement: Grace vs Claude Code with caveman and context-mode

Grace is our own product, so this is our own measurement. The method, every task and the raw data are public — check them rather than trust us.

Test result

≈ −49 % of the 5-hour limit for the same 60 tasks.

Measured against Claude Code with two token-saving plugins already installed: caveman and context-mode.

Read from the limit in whole percentage points, two measurements added up: 32 % with Grace 0.3.8, 63 % with Claude Code + caveman + context-mode · 60 tasks, Claude Max 20x on both sides · October 2 – 3, 2026: 30 tasks at the same time on two accounts, 30 with the comparison run measured separately on the same day · on the comparison account 54 s of readings are missing at a 5-hour reset, which can only make the saving look smaller

5-hour limit used for the same 60 tasks

lower is better

Quality: hidden criteria met

higher is better · the 58 tasks both runs finished

Claude alone vs with Grace, as a table
Measure	Claude + caveman + context-mode	With Grace
5-hour limit used for the same 60 tasks	63 %	32 %
Quality: hidden criteria met	91.9 %	92.5 %

Claude Max 20x on both sides · Grace 0.3.8 · October 2 – 3, 2026 · baseline: Claude Code + caveman + context-mode · two measurements: one at the same time, one with the comparison run measured separately

≈ 96–100 %more work in the same limit
Calculated from the limit value above: ≈ 96 % more. On 43 of the 58 tasks both runs finished, at least twice as much: 2.0× to 4.4×, calculated from Claude’s cost per task (quality on those: 94 % with Grace, 93 % Claude alone). 2 more such tasks are not counted because of their quality (45 % with Grace, 93 % Claude alone).
49 %less time
Measured at the same time with Grace 0.3.8 (October 3, 2026): 3 h 41 min with Grace, 7 h 10 min Claude alone — the time of all 30 tasks, added up. Grace was faster on 28 of them; quality: 93 % with Grace, 90 % Claude alone. Only the 30 tasks measured at the same time are compared on time; the other 30 were measured separately.

How it was measured

Grace 0.3.8 against Claude Code with the token-saving plugins caveman and context-mode — not bare Claude — on 60 tasks in two tests, October 2 – 3, 2026, on Claude Max 20x accounts: 30 tasks on two accounts in the same order at the same time, and 30 tasks whose comparison run used the same tasks on the same day but was measured separately.
Each account’s share of its 5-hour limit was read every 5 minutes, in whole percentage points; the headline adds up both tests.
On the comparison account, 54 s of readings are missing at a 5-hour reset. That can only make the saving look smaller.
Quality: every task has acceptance criteria kept secret from the runs, judged after the run. Only tasks both runs finished are compared.
Earlier tests — Grace 0.3.7 on 2 October and an earlier version on 29 September 2026 — are on the measurements page.

Raw data per task (JSON, 25 KB) Full method and raw data

What this test does not show

How much caveman or context-mode save on their own — they were part of the baseline and not measured separately.
Pro and Max 5x — only Max 20x was measured; the plan figures on our site are calculated with Anthropic’s plan multiples.
Your own work — tasks differ. The fair check is /usage on your own tasks.
Time for the 30 tasks whose comparison run was measured separately — only the 30 tasks measured at the same time are compared on time.

Try free for 30 days

Just create an account · no payment details · ends on its own

Measurements by others

These are not our measurements. Each was published by the named author; we quote the result word for word and link the source. Tools change — a result belongs to the version and date shown.

Measured by others

caveman

Measured by
JetBrains AI blog (Denis Shiryaev) · 6 July 2026

Setup
SkillsBench, 86 tasks · Claude Code 2.1.200 · claude-sonnet-5 at low reasoning effort · 3 runs · output tokens

Result (quote)

Advertised saving: 65%. Measured saving: 8.5%. Quality: “no detectable degradation”. caveman’s README has since been updated and cites this measurement itself.

Source
Speaking to AI Agents like Cavemen Saves 65% of Tokens. We Test.
Measured by others

caveman vs “Be brief.”

Measured by
Max Taylor · 29 April 2026

Setup
24 prompts as single calls with claude -p — not agent tasks

Result (quote)

The two-word prompt matched it on tokens and quality.

Source
I benchmarked caveman against two words
Measured by others

RTK

Measured by
JetBrains AI blog (Denis Shiryaev) · 20 July 2026

Setup
SkillsBench · Claude Code 2.1.201 · claude-sonnet-5 at low and high effort · cost per task

Result (quote)

+7.6% more expensive at low reasoning effort (p=0.004), ±0% at high effort “Task quality: unchanged in both arms, at both effort levels.”

Source
rtk Claude Code Token Savings: A Skill Trial Benchmark
Measured by others

RTK

Measured by
Quesma (Bartosz Kotrys, Jacek Migdal) · 11 September 2026

Setup
Terminal-Bench 2.1 · Claude Code with Fable 5.0 and OpenCode with DeepSeek V4 Pro · 1,740 attempts

Result (quote)

With RTK, costs fell by 5% for Fable and rose by 5% for DeepSeek.

Source
RTK reports huge token savings, but our cost benchmarks disagree

context-mode

We found no independent measurement of context-mode on Claude Code as of 2 October 2026. Its own figure for tool output kept out of the context: “315 KB becomes 5.4 KB. 98% reduction.” (context-mode on GitHub) In our test it was part of the baseline.

Measure it on your own work

Type /usage in Claude Code and note your 5-hour bar.
Work as usual for a while — the same kind of tasks with and without the tool.
Compare the bars. Anthropic’s own ways to use less are in our guide to the usage limit.

Guide: Claude usage limit reached — what to do