Token Efficiency: Same Quality, Fewer Tokens

Prompts grow the way codebases do. Someone adds an instruction to fix a bug, someone else adds three more examples, the context builder starts including the full customer record "just in case," and the support thread goes in with every quoted reply and email signature intact. Six months later, a feature that started at 2,000 tokens per request sends 14,000, and nobody can say which of them matter.

Every token costs money, takes processing time, and draws from your rate limits. But cutting tokens blindly is how you ship a cheaper feature that's quietly worse. In this lesson you'll find where a feature's tokens actually go, cut them in order of payoff (output first, then uncached input, then everything else), prune conversation history without breaking caching, and prove with evals that quality held.

Find where the tokens go#

Start with a breakdown, not a hunch. Split the prompt into its sections and count each one with the token-counting endpoint, which uses the model's real tokenizer (other vendors' tokenizers give different numbers for the same text):

import Anthropic from "@anthropic-ai/sdk";
 
const anthropic = new Anthropic();
 
/** Count tokens per prompt section, so you know what to trim first. */
export async function tokenBreakdown(model: string, sections: Record<string, string>) {
  const rows = await Promise.all(
    Object.entries(sections).map(async ([name, text]) => {
      const { input_tokens } = await anthropic.messages.countTokens({
        model,
        messages: [{ role: "user", content: text }],
      });
      return { name, tokens: input_tokens };
    }),
  );
  const total = rows.reduce((sum, r) => sum + r.tokens, 0);
  console.table(rows.map((r) => ({ ...r, share: `${Math.round((r.tokens / total) * 100)}%` })));
}
 
// Example:
// await tokenBreakdown("claude-sonnet-5-5", {
//   instructions: SYSTEM_PROMPT,
//   examples: FEW_SHOT_EXAMPLES,
//   customer: JSON.stringify(customer, null, 2),
//   thread: rawThreadText,
// });

Each count includes a few tokens of message overhead, so treat the numbers as close estimates. Run it on a dozen real requests, not one, because the variable sections (threads, retrieved documents, history) dominate on some requests and barely register on others.

Then look at output. Pull output_tokens from your logs for the same feature. Output on current models includes thinking, so a feature can produce 200 visible tokens and still bill for 1,500.

Spend your effort where the money is#

Not all tokens cost the same, and that changes what's worth trimming. At the time of writing, output is priced at five times input on current Claude models, and cache reads cost a tenth of normal input or less. So, roughly in order of payoff:

The counterintuitive consequence: shaving 1,000 tokens off a well-cached system prompt saves very little money. Cutting 1,000 tokens of quoted email from every uncached thread saves at least ten times as much, and trimming 300 tokens of output from every response saves more again. Fix the expensive rows first.