Over the last few years, we watched and helped companies implement AI to solve business problems. Customer service, documents, internal search, emails, sales, reporting, agents. And then something started bothering us.
It was how much could be spent on AI that was not necessarily producing a better outcome. A lot of the time, companies were not paying more because the task actually required more intelligence. They were paying more because they were afraid that the less expensive model would not be enough.
You end up with a frontier model classifying an email that a much smaller model, sometimes even an open-weight model, could classify in the same exact way for a fraction of the price.
You send an entire document into context when only three paragraphs matter, and now the frontier model has to process everything. You invoke an LLM to perform something deterministic that normal code could execute perfectly. You use an expensive reasoning model for an extraction task. You build an agent that goes through several calls, retries and reasoning phases for something that probably required one.
That distinction is important. If a task truly requires the strongest model available, then by all means, use it. The problem starts when we use the strongest model by default.
Benchmarks do not answer the exact question we were trying to answer
Now, obviously, you could say benchmarks are there for exactly that. And they are important. They let us compare models on reasoning, coding, retrieval, mathematics, knowledge, instruction following and a lot of other dimensions.
But benchmarks usually classify a model for a certain test. They do not tell me what model is economically optimal for one very specific business task inside one very specific system.
If I am classifying an email between six known categories, I do not necessarily care that Model A is much better than Model B at advanced mathematics, long-form reasoning or some general benchmark. I care about whether Model B can classify my emails correctly.
And if it can do it at almost the same quality for one tenth of the price, then that difference matters a lot more to me than its position on a general leaderboard.
There is also another thing. Most benchmark conversations are quality-first conversations. Which model scores the highest? Which one is closest to the expected answer? Which one wins?
In production, that is only half of the question.
We did not want to optimize by intuition
When you work with AI systems for long enough, you naturally develop intuition. You look at a task and think, “This looks simple, we probably do not need the frontier model.” Or, “This one is more complicated, we should probably use the stronger model.”
And experience is useful. You need experience to understand the task, design the architecture, build the right evaluation dataset and decide what should actually be measured.
But we did not want to build infrastructure based on how we felt about a model.
We wanted explicit tests. Rigorous tests. We wanted to be able to prove and back the model choice we made.
That became one of the fundamental philosophies behind TokenTrim.
The human should decide the tolerance, not the winning model
For a lot of AI-native tasks, especially generative ones, there is not one perfect answer. There can be several excellent emails, several excellent summaries and several excellent ways to explain something.
So for those tasks, we do not think the most useful question is, “Did the model reproduce exactly what a human wrote?”
We think the useful question is, “How close is this model to the best quality that is currently available?”
That best quality becomes the reference frontier.
Then the human makes the decision that actually matters. Not which model should win. The human decides how much degradation is acceptable for the task.
For a simple internal email workflow, maybe being within 10% of frontier quality is fine. For an important customer communication workflow, maybe the tolerance is 5%. For a legal document workflow, maybe it is 3%, 1%, or effectively zero.
The business decides the tolerance. The benchmark measures the models. The framework selects the cheapest model that stays inside that acceptable range.
Truth where truth exists. Frontier quality where it does not.
If an email objectively belongs to Category A, Category A is the ground truth. If we are evaluating an email response or a summary where several answers can be good, the frontier reference becomes more useful.
Introducing the Least Cost Model Framework
The idea behind the Least Cost Model Framework, or LCMF, is intentionally simple. We do not want a complicated equation that nobody can interpret.
For a given task, we test a set of candidate models. We establish the quality of the frontier reference for that task. Then the organization defines the maximum acceptable quality degradation.
M*T = arg min C(M,T)
subject to Q(M,T) ≥ QF(T) - δTLCMF selection rule
In plain English, test the models. Establish the quality frontier. Decide how far below that frontier the use case can safely operate. Remove every model that falls outside that tolerance. Then choose the cheapest model that remains.
That is it.
The tighter we want to be, the more expensive it usually gets
This is where model selection becomes an economic problem instead of just a technical one.
| Model | Relative quality | Cost per 1K tasks |
|---|---|---|
| Frontier Model | 100% | $18.00 |
| Model B | 99% | $7.00 |
| Model C | 96% | $1.80 |
| Model D | 90% | $0.60 |
If the accepted degradation is 5%, Model C qualifies. The company goes from $18 to $1.80 while staying inside the quality range it decided was acceptable.
The tighter the quality requirement, the more expensive the solution usually becomes. The broader the tolerance, the more candidates become available and the lower the cost can go.
That means the question is no longer simply, “Which model is better?” It becomes, “How much are we willing to pay for the last few percentage points of quality?”
For email classification, maybe a company can accept a 10% difference from the frontier. For legal document retrieval, maybe the acceptable difference is 3%. There is no universal answer because there is no universal price for quality.
Why model selection and routing come first
There are a lot of ways to optimize AI infrastructure. Context engineering matters. RAG architecture matters. Caching matters. Prompt optimization matters. Retries matter. Agent orchestration matters.
TokenTrim wants to optimize all of them.
But model selection sits at the center because the model itself can create one of the largest discrete differences in cost.
If one model costs 10x more than another, reducing the context by 20% is useful, but it does not solve the biggest problem if the cheaper model was already capable of doing the task.
This is why routing is central to TokenTrim. The objective is not to find one cheap model and use it everywhere. It is to understand every task well enough that each one can be routed to the least expensive model capable of satisfying its quality requirement.
Simple tasks go to simple models. Complex tasks go to more capable models. Ambiguous tasks can escalate. And when a task does not need an LLM at all, it should not receive one.
The bigger idea is AI waste
LCMF is one layer of a much bigger problem.
AI waste can come from the wrong model, too much context, poor retrieval, duplicated calls, bad retries, unnecessary reasoning loops, agents doing work that deterministic software could do, users sending much more context than necessary, and infrastructure that nobody is really measuring.
That is where TokenTrim is going.
We want to build frameworks, tooling and techniques to optimize every layer of AI engineering. Model selection. Routing. Context. RAG. Caching. Agent orchestration. Observability. Human usage. Traditional software when AI is not the right tool.
The objective is not to make companies use less AI for the sake of using less AI.
The objective is to help them use AI in the most responsible and economically rational way possible, and to get as much value as possible from the compute they are already paying for.
This is ultimately why we built TokenTrim.
Not because frontier models are too expensive. Not because companies should stop using the best models. And not because every task should be pushed toward the smallest model possible.
We built it because companies should know why they are paying for a particular level of intelligence.
If the strongest model materially improves an important business outcome, use it. If another model costs one tenth as much and stays inside the acceptable quality threshold, use that instead.
But make the decision with evidence.