Use the Cheapest Model That Can Reliably Do the Job
Model selection is one of the largest cost levers, but I would not reduce it to 'small model cheap, large model expensive'. The real goal is to route each task to the lowest-cost option that still meets the required quality, latency, and reliability.
- Task classification
- Model routing
- Escalation paths
- Quality gates
- Provider diversification
The situation
It is convenient to use one strong model for everything. It keeps the code simple and avoids routing logic. It also means I pay premium reasoning cost for tasks that may only require extraction, classification, formatting, or a short deterministic transformation.
The naive solution
Replace the expensive model with a cheaper one globally.
The hidden constraint
Different parts of a workflow have different difficulty. A cheap model may be perfectly adequate for classification but poor at a complex code repair. A premium model may be unnecessary for summarising structured data but valuable when a failed build needs multi-step reasoning.
The decision
Route by task, not by product. I want an explicit policy that says which class of work starts on which model, what quality signal triggers escalation, and when the system is allowed to use an expensive fallback.
The implementation
I would classify calls into categories such as extraction, classification, drafting, coding, planning, repair, vision, and high-stakes review. Each category gets a default model and one or more fallbacks. Escalation can be triggered by low confidence, failed validation, tool errors, or an explicit user choice. The routing decision itself should be logged so I can later compare cost and success rate by route.
The tradeoff
Routing adds product and evaluation complexity. It is only worth doing when I can measure whether the cheaper route maintains acceptable quality. Otherwise I am just moving errors around.
The evidence I would look for
I would compare cost per successful task, not cost per call. If a cheaper default plus occasional escalation beats a premium model on every turn without reducing task success, the routing policy is working.
The reusable lesson
The right model is a workload decision. Premium intelligence should be spent where it changes the outcome, not where it is simply available.
The consultant's questions
- Which calls genuinely require the most capable model?
- Which calls are classification, extraction, or formatting disguised as 'AI'?
- What evidence triggers escalation to a stronger model?
- Are routing decisions logged so they can be evaluated rather than guessed?