LLM vs decision model: Does your AI need to talk or decide?

When a customer sends a message, the software may not need to write an answer first. It may need to decide whether the issue is about billing or technology, whether it should be escalated to a person, or which system should handle it.

That is the difference between a large language model (LLM) and a decision model. An LLM is built to understand and generate language. A decision model is built to return a bounded judgment that software can use. Code and rules still handle policies that are already clear and deterministic.

These are not necessarily competing technologies. A business may use all three in the same workflow.

TLDR; LLMs, decision models and code

LLM

Best suited to open-ended language, reasoning and content generation. It can return a reply, summary, explanation, plan or piece of code.

Decision model

Best suited to repeated judgments with a defined answer space. It can return a label, choice, score, probability or route.

Code or rules

Best suited to explicit and deterministic policies. It returns an action based on fixed conditions.

The key question is not which system is more intelligent. It is what the software needs the system to return.

What an LLM does

Large language models are trained to work with language. They can read a customer complaint, identify the problem, summarise it and draft a response. They can translate documents, answer questions, suggest plans and generate code.

Their strength is flexibility. An LLM can work with a problem that has not been neatly defined in advance, and it can explain its response in a way that is useful to a person.

LLMs can also classify information. A developer can ask one whether a support ticket is about billing or a technical problem and request the answer in JSON. But the model is still generating that response as a sequence of tokens. The application must validate the output and account for the possibility that it is incomplete, incorrectly formatted or simply wrong.

Structured output improves the interface between an LLM and software, but it does not guarantee that the model selected the right category. JSON mode can enforce a format; it cannot ensure that the underlying decision is correct.

What a decision model does

A decision model is designed to answer a bounded question that software can consume directly. The possible result is defined in advance, even if the input is difficult to interpret.

A developer might ask it to:

• choose one category from a list
• decide whether a condition is met
• score something against a defined scale
• select a route or flag a case for review

The input can still be messy or written in natural language. The difference here is that the output is a typed result, a label, choice, score or probability, rather than a paragraph explaining the reasoning.

The term can cover different systems. A rules engine follows explicit if-then logic. A fraud model may calculate a risk score from structured data. A newer neural decision model can interpret unstructured text while returning a constrained choice. These systems are built differently, and a decision model is not necessarily deterministic.

TypeSafe AI calls its new class of models “System One models”. Its first public model, Jev, is one example of the approach, but the broader idea is more important than the individual model. A System One model can understand natural-language input while returning typed decisions and probabilities instead of free-form text. TypeSafe explains the approach in its documentation.

A constrained output can make a system easier to integrate. Code does not have to search a paragraph for a label, and developers can automate high-confidence results while sending uncertain cases for review.

That does not make the model infallible. It can still select the wrong option. A probability is not an explanation, permission or guarantee of accuracy. The possible answers, evaluation data, thresholds and escalation process still need to be designed and tested.

One workflow, three roles

Consider a customer-support request about a failed payment.

The LLM reads the customer’s message, summarises the problem and drafts a helpful reply.

The decision model determines whether the issue is related to billing, identifies whether the customer is asking for a refund and estimates whether human attention is needed.

Code or rules check the account, purchase date and refund policy before taking action.

If the decision model is uncertain, the case can go to a support employee. If the policy is completely fixed, the application may not need a model for that part at all.

This separation prevents a business from using an expensive, flexible tool for every small decision. It also gives each part of the system a clearer responsibility: the LLM handles language, the decision model supplies signals and code enforces permissions and hard rules.

The same pattern can apply to AI agents. An LLM may interpret a user’s request, plan a task and call a tool. A decision model could classify the request, help select the right model or flag a proposed action for review. Authorisation should still be enforced in code or through an appropriate human process; a model’s confidence score is not a substitute for permission checks.

The approach could also support fraud reviews, ecommerce returns and SaaS request routing. In high-stakes areas such as lending, hiring, insurance and compliance, businesses would still need human oversight, fairness testing and a process for reviewing outcomes.

Which one should businesses use?

Need a message, summary, explanation, plan or piece of code?

Use an LLM.

Need a choice from a defined set, a score or a route?

Consider a decision model.

Need to enforce a clear and deterministic policy?

Use code or a rules engine.

Need all three?

Combine them in one workflow.

A decision model may be especially useful when the same judgment needs to be made repeatedly, at low latency and high volume. Its value is not simply that it produces a shorter answer. Its output is designed from the start to be used by software.

The practical question is therefore not, “Should the business replace its LLM?” It is, “Which parts of this workflow require language, and which parts require a decision?”

The likely answer will often be both. One system can understand what a person wants and communicate with them. Another model or simple code can determine what should happen next. As AI moves deeper into business processes, some of the most useful intelligence may not be the kind that talks like a chatbot. It may be the kind that quietly returns a clear, usable signal when software needs to act.