Tracking token spend and identifying waste
Cache hit ratios, tokens per shipped artifact, the uncommitted-output day diagnostic.
Why token tracking matters more than the bill
Most operators look at the monthly API bill and either feel comfortable (under their internal threshold) or panic (over it). The bill is the wrong signal. Two operators with the same bill can be in completely different states — one shipping efficiently, one bleeding. The bill doesn’t tell you which.
The useful signals are ratios, not absolutes. Cache hit ratio tells you whether your sessions are warm or cold. Tokens per shipped artifact tells you whether the spend is producing output. Uncommitted-output day count tells you whether briefs are drifting. None of these show up on the bill.
Run the three diagnostics weekly. They take ten minutes total. They’ll catch waste 2-4 weeks before the bill does.
The three diagnostic metrics
Each catches a different waste pattern. Skip any of them and you’ll miss the corresponding pattern entirely.
| Metric | Healthy range | What it catches |
|---|---|---|
| Cache hit ratio | >60% | Cold-cache thrash — sessions too fragmented to reuse context |
| Tokens per shipped artifact | Trending flat or down vs baseline | Per-unit cost drift — agents getting less efficient over time |
| Uncommitted-output days | <1 per week | Brief drift — sessions that consume tokens but ship nothing |
Cache hit ratio is the cheapest to fix. Brief drift is the hardest. Token-per-artifact drift is somewhere in between. Run them in cost-of-fix order so you bank the easy wins first.
The rest of Expert · Lesson 03 is for subscribers.
Tracking token spend and identifying waste
- Every Expert-tier lesson — diagnostic prompts, transcripts, prompt kits, full homework
- Every research paper — methodology, figures, tables, reproducibility appendices
- New Expert lessons + papers as they ship (quarterly cadence)
- Foundations + Operating lessons stay free; bundles on GitHub stay free; this tier is the deep stuff
Free while the early catalog ships. Paid tier comes later — subscribe now and you’re grandfathered in.