Output Token Efficiency

Simple Definition

Output token efficiency is how much useful work a model completes relative to the number of output tokens it consumes. It’s a measure of results per token, not just the sticker price per token.

Why Fewer Tokens Isn’t Always Better, and Cheaper Isn’t Always Cheaper

A model with a low price per token can still be the more expensive choice in practice if its answers need more retries, more follow-up questions, or more human editing to become usable. A pricier model that solves the task correctly in one clean pass, using more tokens to do it, can end up cheaper overall. Output token efficiency is about judging cost per completed, usable result, not cost per token in isolation.

Example

Two models are asked to fix the same bug. Model A is cheaper per token but produces a fix that fails tests twice before working, tripling the effective token spend once retries are counted. Model B costs more per token but fixes it correctly on the first attempt. Measured by price per token, A looks cheaper. Measured by output token efficiency, B was the better deal.

What Affects It

Retries: failed attempts that have to be redone add hidden token cost
Verbosity: unnecessarily long answers spend tokens without adding value
Accuracy on the first pass: fewer corrections needed means fewer wasted tokens overall
Task fit: a model well-suited to a task type tends to need less back-and-forth