The Model Families and When Each Is Right

The LLM space in 2026 has roughly four shapes of model, and the FDE's job on any new task is to pick the smallest one that still does the job. Reaching for the biggest model on reflex is how POCs end up unshippable at the customer's actual traffic. This lesson is about the four shapes, when each is right, and the one discipline that keeps you from defaulting upward.

The four families

The specific names and versions move constantly. The shapes they occupy do not.

The last family is the one FDEs most reliably misuse. Reasoning models are not "the frontier plus more time," they are a different tool. On a task where the answer is already close to the first token — a short summary, a lookup, a polite rewording — the extra thinking is pure latency and pure bill.

The one rule

This rule is boring and it is the whole discipline of the role. Every other piece of routing logic, caching strategy, and cost conversation is downstream of it. The reason it is hard is that frontier models produce output that reads better by default — longer, more fluent, more confident — even on tasks where the smaller model got the same answer right. Reading the diff is not the eval.

The thing that makes this rule executable is the eval. The habit that makes the eval executable is writing it before you have picked a model, so the measurement is doing the choosing.

Routing by cost and latency

Once you accept the rule, most production systems end up with a router. The simple version is two models:

1. Step 1

   Request arrives

   The small or mid-tier model handles it by default. Cheap, fast, good enough on the bulk of the traffic.

2. Step 2

   A judge or a confidence signal reads the answer

   Either an LLM-as-judge rubric, or a cheap score from the model itself (logprobs, refusal, schema-validation failure). This is the escalation signal.

3. Step 3

   Low-confidence answers escalate

   The same request, retried on the frontier model. Users see the better answer on the hard subset; the bill is dominated by the easy majority.

4. Step 4

   Everything is logged with model and outcome

   So the eval can measure both tiers on the same slice and tell you whether the split is right.

The number that matters is the escalation rate. A system escalating 2% of requests has routing working. A system escalating 60% is a system that should have skipped the small model and started on the mid-tier. A system escalating 0% is a system without a good signal, and the small model is probably passing through answers that would have failed a real eval.

The customer-facing version

The reason this lesson is in the FDE track and not just an engineering chapter is that the customer is in the room for this decision, and they usually came in with a wrong one. The common versions:

"We need the best model." The best model on their task, not the best model on a benchmark. The honest reply is to run an offline eval on a slice of their own data and show them the gap, or the lack of it.

"Can we use GPT-4-level for everything?" The customer is reaching for a quality floor. The reply is a router, not a flat choice — the floor is the frontier model behind the escalation, and most requests never need it.

"Will reasoning mode make it smarter?" Sometimes. On a planning task, almost always. On a short extraction, almost never. The reply is to try both and read the eval, not to argue.

What to take away

There are roughly four shapes of model — frontier, mid-tier, small, and reasoning — and they are tools for different jobs. The one rule is to use the cheapest model that passes the eval, and escalate only when the eval says to. Most production systems end up as a two-tier router with a confidence signal, and the escalation rate is the number that tells you whether the split is right. The customer is always in the room for this choice, and the strongest move is to run the eval on their own data in front of them.

Next: sampling, temperature and the honest limits of reproducibility.