Ten days of measuring agent runs: what the token log shows that the dashboard doesn't

Discussions
1

I am not an engineer. I set out to answer a narrow question: how much of what an agent bills you is actual work?

Ten days later I have eleven billed requests, two plans, two models I didn’t choose, and a dashboard that moved 10x while my usage moved 1.7x. Here is everything, including the parts where I was wrong.

The setup

One task, held constant the whole way: a Python script that pulls BTC, ETH and SOL prices from Binance’s public ticker endpoint, prints a table, and retries on network errors. Same prompt, word for word, every time. Fresh empty folder and a new chat for each run.

Windows 11. Python was not installed when I started, which turns out to matter.

My success criterion, fixed on day one and never adjusted: a run counts as successful only if the agent reached working code without me stepping in. If I corrected it, hinted, or fixed anything, it’s a failure — even if the thing worked in the end. I checked every script by hand in a terminal afterwards, because “the agent said it’s done” is not evidence.

Part 1: the free plan

Seven requests over three days, all on composer-2.5-fast. Total: $1.01.

Date (UTC) Fresh input Cache read Output Total Cost
Aug 17 14:20:10 16,733 139,904 1,797 158,434 $0.15
Aug 17 14:20:33 31,732 122,688 1,984 156,404 $0.19
Aug 17 14:38:14 7,200 58,368 1,477 67,045 $0.07
Aug 17 14:44:47 1,911 92,864 653 95,428 $0.06
Aug 17 14:51:59 25,114 110,496 3,362 138,972 $0.18
Aug 18 07:03:58 52,536 57,280 2,327 112,143 $0.22
Aug 19 06:52:14 34,463 36,640 1,429 72,532 $0.14

Three of those requests were the identical task. The other four were setting up my machine: checking for Python, installing it through winget, dealing with the Microsoft Store alias that hijacks the python command on Windows.

Setup cost roughly two and a half times more than the work.

The three identical runs

  • Run 1: $0.18, two self-corrections, worked
  • Run 2: $0.22, one self-correction, worked
  • Run 3: $0.14, zero corrections, failed

Run 3 shipped a script that raised 400 Bad Request on every execution. The agent had built the Binance batch query with spaces after the commas in the JSON array — the API rejects that. It never ran the script to check.

So: cost per run across the three is $0.18. Cost per successful run is $0.27. Fifty percent higher, and the naive metric would have called the broken run the most efficient of the three.

That gap is the entire reason I started doing this.

The retry that could never work

Run 3’s script retried the failing request three times with a two-second pause, exactly as the prompt asked. But a 400 is a client error. The same malformed request never becomes valid. It burned three attempts and six seconds on something structurally impossible.

Run 2, from the same prompt the day before, had the agent narrow its retry logic to ConnectionError and Timeout on its own. Run 3 dropped that refinement. Same instruction, worse engineering, no way to predict which you get.

The same bug, three days running

Runs 1, 2 and 3 all made the identical Binance formatting mistake. The first two times the agent caught it and fixed it. The third time it didn’t.

There is no memory between sessions. You pay for the same bug every time it occurs.

Part 2: what the token counts actually say

This is the part I did not expect.

77.2% of every token I paid for that week was cache read — the agent re-reading context it had already seen. Less than a quarter was new work.

Which means total token count barely predicts cost:

Total tokens Cost
Largest run 158,434 $0.15
Most expensive run 112,143 $0.22

41% more tokens, 47% cheaper.

Correlation of cost with total tokens: 0.60. With fresh input plus output: 0.90.

Cost per million fresh tokens ranged from $3.90 to $23.40 across those seven requests. Same model, same machine, same week.

I also ran a small deliberate test. One run in an empty chat, then a second task appended to that same conversation:

Fresh input Cache read Cache share
Empty chat 28,449 44,800 60.1%
Continuing the conversation 20,101 62,080 74.1%

Fresh input down 29%, cache read up 39%. The high-water mark in my log is a request with 645 fresh tokens against 66,304 cache reads — 98.7%.

Part 3: the free plan dies at 51%

I had been tracking Included Usage percentage as if it were the limit. On day three the free plan stopped working with the counter reading 51%, not 100%.

There is a separate cap on agent requests, and that is what ran out. The percentage I’d published for two days was never the binding constraint.

To be precise about my own error: the percentage was not wrong as a measure of cost. Run 2 read 11% and cost $0.22; run 3 read 7% and cost $0.14. The ratios match to three decimal places. It tracked spend linearly. It just wasn’t the thing that stops you working.

Part 4: what the log stops showing on Pro

I upgraded to Pro. In the same CSV export, same column:

  • Free plan, per request: $0.15, $0.19, $0.07, $0.06, $0.18, $0.22, $0.14
  • Pro plan, per request: Included

No figure. Both pools — Cursor Models and Other Models — report the same word. I ran a third-party model (gpt-5.6-sol-medium) specifically to test whether the dollar-denominated pool would show dollars. It shows Included too.

A model I did not select

Every run since upgrading is logged as cursor-grok-4.6-medium. My composer said Composer 2.5 Fast.

I don’t know when or why it switched. The practical consequence is that my free-plan series and my Pro series aren’t cleanly comparable, and I only found out because the raw export has a Model column that the UI does not surface.

One way to recover a number

The Other Models pool is documented as including at least $20 of usage. After two GPT-5.6 runs the dashboard read 1% of that pool.

1% of $20 is $0.20, so roughly $0.10 per run. With rounding, 1% could be anywhere from 0.5% to 1.49%, so the honest range is $0.05–$0.15 per run.

That’s indirect, but it’s the only path to a dollar figure I’ve found on Pro.

Part 5: the number I got wrong in public

On August 22 the dashboard showed 315K tokens consumed and 0.1% used. I extrapolated a monthly quota of roughly 315M tokens and published that.

On August 26 I checked again:

Snapshot Tokens used Dashboard
Aug 22 314,959 0.1%
Aug 26 528,975 1%

Usage rose 68%. The meter rose 900%.

Same pool, same model, same account. I recalculated weighting by published token rates rather than raw counts — dollar-weighted usage rose 1.56x. That doesn’t explain it either.

Reading the same two figures the other way round gives incompatible answers. 0.1% implies a quota somewhere between 211M and 630M tokens. 1% implies 36M to 106M. The ranges don’t overlap.

Cursor announced increased included limits effective August 24, between my two snapshots. If the quota went up, the percentage should have gone down per token, not up tenfold.

So my 315M estimate does not hold, and I’d ask anyone who saw it to disregard it. I can’t currently tell whether the quota changed, whether the percentage measures something other than tokens, or whether one-significant-figure rounding is hiding the answer.

What I’m left with

Things I can state from my own data:

  • One week of light agent use on the free plan: $1.01
  • Cost per successful run was 50% higher than cost per run, because one in three runs produced nothing
  • 77% of tokens billed were re-reads of existing context
  • Total token count is a poor predictor of cost; fresh tokens are a good one
  • The free plan stops at roughly half the displayed quota
  • The paid plan reports Included instead of a price
  • The model in the interface is not necessarily the model in the bill

Things I cannot state:

  • What a run costs in dollars on Pro, directly
  • What the Pro quota actually is
  • Why the meter moved 10x when my usage moved 1.7x

To be clear, this isn’t a billing complaint — my usage is trivial and nothing was overcharged. I’m trying to work out what the meter measures.

If someone with a longer Pro history can post their own two snapshots — tokens consumed and percentage displayed, a few days apart — that would settle the last question quickly. I have eleven requests. That is not a sample, it’s an anecdote with a spreadsheet attached.

Raw CSV exports and the full run log are available if useful.

4

Hey, thanks for such a detailed write-up. It’s rare to see data put together this cleanly, and it’s especially helpful that you called out your own mistakes along the way. I’ll reply point by point where I can be specific.

“Included” instead of a total on Pro. This is expected behavior, not missing data. On Pro, requests within the included limit are marked as “Included” instead of showing a dollar amount per request, so both pools (Cursor Models and Other Models) show the same word, even for a third-party gpt-5.6-sol. Dollar amounts in that column only show up for usage above the included limit. We’re working on better visibility into included usage on Pro.

Model in the UI vs export (Composer 2.5 Fast vs cursor-grok-4.6-medium). This is the most interesting part, and it shouldn’t happen for a directly selected model. If you can grab a Request ID from one of the Pro runs (chat menu top right > Copy Request ID), I can pass it to the team so they can check attribution for that specific request. Note: if Privacy Mode is enabled, the Request ID might be unavailable.

Separately on the “Fast” label. The compact picker view currently shows the effort or speed level (like “Fast”) instead of the model name. That’s just UI display and we’re improving it, it doesn’t switch the actual model. The expanded picker shows what’s actually selected.

The percent jump (usage 1.7x, 10x scale) and the switch from $ to tokens. This is being actively discussed here, and users are sharing their screenshots too: Usage Page $$ to Token Amount? WHAT?. Your case would fit perfectly there. Also, an update to included limits landed around Aug 24, between your two screenshots, which can shift the percentages too. So the mismatch between 0.1% and 1% is partly about changes in what the scale counts and how it’s displayed, not just token growth.

Your note about 77% cache read and that fresh input predicts cost better than total tokens is basically correct. Cache read is billed at a much lower rate, so total tokens and cost don’t correlate strongly.

Let me know the Request ID. That’s the only part that’s really worth checking on a specific request.

5

Continue a chat and cache read goes up 39%, fresh input down 29%. Your own table. And cost follows fresh input at 0.90, total tokens at 0.60. So a long thread reads cheap per turn while the base under every turn keeps growing.

Runs 1, 2 and 3 all hit the same Binance bug. That belongs in a rules file. Written down it bills at cache read rate. Rediscovered it bills output tokens, and run 3 shows it does not always get rediscovered.

Cost per successful run $0.27 against $0.18 per run. Nobody else publishes that split.

7

Update, with a bigger correction than I expected.

I’ve been reading Included Usage from the summary card, which rounds to a whole percent. The detailed table (Billing & Invoices → Included Usage) reads to one decimal place, and mine has now shown 0.1% three times in a row:

Aug 22 — 314,959 tokens
Aug 31 — 529,000 tokens
Aug 31 — 658,500 tokens

Tokens roughly doubled. The needle never moved.

Three readings that all have to be true at once constrain the answer. A 0.1% reading means the real figure is somewhere between 0.05% and 0.149%. Intersecting all three:

442M–630M tokens per month, for $20.

That also corrects my earlier post in this thread: I estimated ~315M and told people to disregard it as too uncertain. I was wrong in the other direction — the quota looks larger than I thought, not smaller.

To see the needle move to 0.2% I need roughly 5–10 more runs of this size. Still looking for anyone with a longer Pro history who can post their own detailed-table readings a few days apart.