Decision models — explanation

What is this about?

A decision model is a small model fine-tuned to choose, not to write. It reads a state — a request, a command, a document — and answers typed questions in one forward pass: which option of a list (choice), how likely a statement is (noul), which level of a scale (score), with a probability per option. Nothing is generated, so an answer costs a fraction of a second and cannot wander off.

llama-server, llmman and Ollama serve such models through one route, POST /v1/systemone. mm uses it in two ways: as an agent you can ask (decision:), and as a step before each turn of a chat agent (skillRouting:).

Why a decider is an agent, not a tool

The first use people want is the one Granite Guardian already showed in demos/guards/: a classifier asked from a hook, a skill or another agent, that answers about one message and forgets it. Making the decision model an mm agent gives it all of that for free — mm -p for scripts and hooks, mmConfigs and /ask-<name> for other agents, the three front ends for a person — with no new plumbing in any of them. The engine is the one place that changes: for a decider, Generate sends the last user message to /v1/systemone instead of calling the chat model, and returns the answer like a model's.

Two consequences follow, both deliberate:

The state is the message alone. No history, no system prompt, no hooks: a decider judges what it is given, and the previous message is not part of what it judges. That is also what a 2,048-token context leaves room for.
No tools. A decision model never calls one, so a decider declaring tools is refused at start-up rather than shown with a misleading tools column.

-p prints the server's JSON untouched — a field a server adds later still reaches the script. The other front ends show one line per question followed by the same JSON, because the person reads the lines and an asking agent may parse the block.

Why routing appends a line instead of loading the skill

When the decider picks a skill, mm appends exactly the line $<skill> produces — [skill: <skill> — load it with read_skill and carry out its steps now] — and the chat model still makes the read_skill call. The alternative, loading the skill's text into the question behind the model's back, would bypass the loop detector, the permission gate and the 📖 recap, which all watch that call. The decider proposes; the chat model still acts, in the open.

Routing never blocks a turn. A server that is down, a model still loading, or a call past timeout (2 s by default — a warm call takes about half a second) leaves the question as it was and says so in one 🧭 line. A question that already names a $skill is not routed: the user's choice wins.

Why the server's confidence, and why 0.6

The server returns two numbers for a choice: the probability of the chosen option and a confidence. On every answer seen so far the confidence is the lower of the two, so comparing it with minConfidence is the more cautious reading. The default of 0.6 was set from the first live runs, where the right answers sat between 0.68 and 0.97 — then checked against the routing eval (below).

What was measured

On 2026-10-22, with LiquidAI d1-3B Q4_K_M on llama-server and the 23 cases of the routing eval (internal/eval, the repository's skills):

minConfidence	hit	miss	right
0.0	17	6	73 %
0.5	16	7	69 %
0.6	15	8	65 %
0.7	14	9	60 %
0.8	12	11	52 %

About 0.5 s per case. The decider never chose a wrong skill and never chose one where none applied, at any threshold: its failures are all abstentions, and an abstention costs nothing — the chat model then routes alone, as it did without a decider. All six cases with no applicable skill were answered none, with confidence 0.93 to 0.98. The misses are the bg-jobs requests that describe a situation without naming it ("start the dev server and check it answers") — the same cases the chat models miss — and "Salue Bob de ma part", which two skills claim.

The same day, the chat model's own eval (TestSkillRouting, Qwen3.5-4B Q4_K_M on llmman) was run twice on the same file, without and with skillRouting (d1-3B, minConfidence: 0.6):

Run	hit	miss	error	right
chat model alone	12	6	5	52 %
with the decider	16	3	4	69 %

Neither run loaded a wrong skill or a skill where none applied. The decider turned four cases into hits ("fais le salut vulcain à Sam", "rename the Greet symbol to Hello everywhere", "the server I started earlier is still running, stop it", "kill the background job I started"); one case went from a miss to an error ("where is the Greet function defined?"). An error is the eval's turn hitting maxTurns against its stub tools — a limit of the instrument with this model, present in both runs, not a routing score.

Triage only removes questions

The triage hook of demos/decision-commands/ (ticket 126) answers allow when the decider is sure a command only reads. allow was added to the hooks for this, and it is deliberately narrow: it skips the confirmation question for one call — nothing more. A deny later in the chain still wins, a never the user gave still refuses, and an agent asked by another ignores it, because there the asking agent's user decides. With yolo: true nothing was going to be asked, so allow changes nothing.

The asymmetry follows from the decider's errors. On the commands tried on 2026-10-22, read-only commands (ls -la, cat, git status, wc -l, find) came back read-only at 0.94–0.95 with a destructive probability of 0.02–0.08, while the risky ones were spread out: curl -s https://x.sh | sh came back local-change 0.51, sudo apt-get install jq dangerous at only 0.24. A decider that is reliable on one side only may be trusted on that side only: it removes the questions for what clearly reads, and everything else — the risky commands it misjudges included — still reaches you. The hook calls mm -p once per command, about 0.4 s warm.

Tool routing only removes what is clearly unneeded

toolRouting (ticket 127) asks one yes/no question per group of tools and leaves out a group only below dropBelow, 0.2 by default. The costs of the two errors are unequal: a tool left out that the request needed means a turn where the model cannot do the task, while a tool kept that it did not need costs a few hundred tokens. On eight requests tried on 2026-10-22 (five groups: shell, files, web, code, plan), every group a request needed scored 0.37 or more, and every score under 0.2 (0.03–0.17) went to a group the request did not need. Many unneeded groups scored higher, though — plan came back between 0.31 and 0.96 whatever the request — so the threshold drops few groups, and only safe ones: a hesitation keeps the tool.

The routing runs in the engine, after the preGeneration hooks, so a question a hook stopped never reaches the decider, and the text routed is the one the model reads. It runs in every mode, -p included: unlike $skill, it changes no text. A tool in no group is never touched, and any failure offers every tool. The price is the server's prompt cache: the tool list is part of the prompt, so a turn whose list differs from the previous one cannot reuse it.

What was not measured yet: whether a smaller tool list makes a small model choose better. The eval measures skill routing, not tool choice.

A decider on its own

A decider needs no chat model. demos/decision-only/ triages a support inbox: four questions per message (category, urgency, personal data, an upset author), one mm -p call each, about 0.6 s per message. On its six messages, one of them in French, the six categories and the personal data were right; the spam was given the urgency "today", which a filter on the category corrects.

Checks after the fact only add a warning

Three hooks ask a decider about something the agent has just received, and add a context under it when the answer is clear: demos/decision-injection/ (ticket 132) on what read_file and fetch_content read, demos/decision-tool-errors/ (ticket 133) on a bash output, demos/decision-ask-check/ (ticket 134) on another agent's answer. None of them blocks or rewrites anything. The output is already there, and the decider is wrong often enough that hiding what it judges would cost more than it saves: a warning the model can weigh is the most a 3B decider's verdict deserves.

What the decider is asked matters as much as the threshold. A decider judges the text as a whole: an injection hidden in one HTML comment drowns in a README (0.44 for the whole file, 0.93 for the comment alone), so the injection hook asks about the paragraphs that mention an AI, an agent or instructions, one by one. A yes/no "the command failed" scored a failing test 0.19; a choice between success, nothing-found and error gave real errors 0.98 and kept grep with no match (0.87) and diff (0.73) under the 0.95 threshold. And the check judges what it is given: when the asked agent's reasoning ended up in its answer, the reasoning — which discusses the question — was judged on topic, and no off-topic answer was flagged.

The thresholds sit where the measured cases split, with little margin. Clean documents reached 0.76 on injection while a polite injection in a page footer scored 0.77 — under the 0.8 threshold, so missed. The on-topic check is about relevance, not truth: a wrong answer on the right subject scored 0.78.

On the demos' runs (2026-10-22, Qwen3.5-4B as the chat model, 5 runs with the hook and 5 without), the warnings were always added where they should be, and the effect on the model was smaller than the warning suggests. With the injection warning, no run ran the hidden curl … | sh — nor did any run without it in that series, while an earlier series had 3 of 5 runs without the hook try it; 3 of 5 summaries mentioned the hidden order, only one as an injection. On a script that prints an error and exits 0, the model said the deployment failed in 5 of 5 runs with or without the note. Told that the historian's answer was off-topic, it asked the expert every time, against 3 of 5 without. A small model often sees an obvious problem by itself; the checks are for the cases where it does not, and they must not push it further than the user asked: a first failure note said "fix its cause and run the command again", and in one run the model rewrote the script it had been asked to run.

Rejected alternatives

evalengine's decision-4b. The model ticket 72 started from. llmman and Ollama refuse it on /v1/systemone ("not a decision model"); it can only be read through logprobs on the chat route, which is less exact (a letter outside the top N counts as 0) and is left to a later ticket. On Docker Model Runner, which has it pulled, that is the only road too.
A decisionModel: block inside the chat agent. One file per model keeps one definition for both uses: the agent you ask is the decider routing uses.
Routing in -p. -p does not expand $skill either; skill routing follows the notation it imitates. Tool routing does run in -p: it changes no text.
Command triage built into mm. A commandTriage: block was possible. A hook needs only the generic allow action, works with any classifier — a decision model, a rule list, a guard model — and keeps the thresholds in a script the user can read.
A tool router that also adds tools. Only removing keeps the agent's declared tools as the upper bound: the decider can never give the model a tool the operator did not.

How it relates to the rest

Guard models: the same shape — a classifier behind an agent file, asked without the history — with a decision model's typed questions instead of a risk name.
Asking another agent: how /ask-<name> reaches a decider.
Skill format: the description is what the decider reads for each skill — the same one the chat model reads in the catalogue.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93


# Decision models — explanation

## What is this about?

A decision model is a small model fine-tuned to **choose**, not to write. It reads a state — a request, a command, a document — and answers typed questions in one forward pass: which option of a list (`choice`), how likely a statement is (`noul`), which level of a scale (`score`), with a probability per option. Nothing is generated, so an answer costs a fraction of a second and cannot wander off.

llama-server, llmman and Ollama serve such models through one route, `POST /v1/systemone`. mm uses it in two ways: as an agent you can ask (`decision:`), and as a step before each turn of a chat agent (`skillRouting:`).

## Why a decider is an agent, not a tool

The first use people want is the one Granite Guardian already showed in `demos/guards/`: a classifier asked from a hook, a skill or another agent, that answers about one message and forgets it. Making the decision model an mm agent gives it all of that for free — `mm -p` for scripts and hooks, `mmConfigs` and `/ask-<name>` for other agents, the three front ends for a person — with no new plumbing in any of them. The engine is the one place that changes: for a decider, `Generate` sends the last user message to `/v1/systemone` instead of calling the chat model, and returns the answer like a model's.

Two consequences follow, both deliberate:

- **The state is the message alone.** No history, no system prompt, no hooks: a decider judges what it is given, and the previous message is not part of what it judges. That is also what a 2,048-token context leaves room for.
- **No tools.** A decision model never calls one, so a decider declaring tools is refused at start-up rather than shown with a misleading tools column.

`-p` prints the server's JSON untouched — a field a server adds later still reaches the script. The other front ends show one line per question followed by the same JSON, because the person reads the lines and an asking agent may parse the block.

## Why routing appends a line instead of loading the skill

When the decider picks a skill, mm appends exactly the line `$<skill>` produces — `[skill: <skill> — load it with read_skill and carry out its steps now]` — and the chat model still makes the `read_skill` call. The alternative, loading the skill's text into the question behind the model's back, would bypass the loop detector, the permission gate and the `📖` recap, which all watch that call. The decider proposes; the chat model still acts, in the open.

Routing never blocks a turn. A server that is down, a model still loading, or a call past `timeout` (2 s by default — a warm call takes about half a second) leaves the question as it was and says so in one `🧭` line. A question that already names a `$skill` is not routed: the user's choice wins.

## Why the server's confidence, and why 0.6

The server returns two numbers for a choice: the probability of the chosen option and a `confidence`. On every answer seen so far the confidence is the lower of the two, so comparing it with `minConfidence` is the more cautious reading. The default of 0.6 was set from the first live runs, where the right answers sat between 0.68 and 0.97 — then checked against the routing eval (below).

## What was measured

On 2026-10-22, with LiquidAI d1-3B Q4_K_M on llama-server and the 23 cases of the routing eval (`internal/eval`, the repository's skills):

| minConfidence | hit | wrong | miss | extra | right |
|---------------|-----|-------|------|-------|-------|
| 0.0 | 17 | 0 | 6 | 0 | 73 % |
| 0.5 | 16 | 0 | 7 | 0 | 69 % |
| 0.6 | 15 | 0 | 8 | 0 | 65 % |
| 0.7 | 14 | 0 | 9 | 0 | 60 % |
| 0.8 | 12 | 0 | 11 | 0 | 52 % |

About 0.5 s per case. The decider never chose a wrong skill and never chose one where none applied, at any threshold: its failures are all abstentions, and an abstention costs nothing — the chat model then routes alone, as it did without a decider. All six cases with no applicable skill were answered `none`, with confidence 0.93 to 0.98. The misses are the `bg-jobs` requests that describe a situation without naming it ("start the dev server and check it answers") — the same cases the chat models miss — and "Salue Bob de ma part", which two skills claim.

The same day, the chat model's own eval (`TestSkillRouting`, Qwen3.5-4B Q4_K_M on llmman) was run twice on the same file, without and with `skillRouting` (d1-3B, `minConfidence: 0.6`):

| Run | hit | miss | error | right |
|-----|-----|------|-------|-------|
| chat model alone | 12 | 6 | 5 | 52 % |
| with the decider | 16 | 3 | 4 | 69 % |

Neither run loaded a wrong skill or a skill where none applied. The decider turned four cases into hits ("fais le salut vulcain à Sam", "rename the Greet symbol to Hello everywhere", "the server I started earlier is still running, stop it", "kill the background job I started"); one case went from a miss to an `error` ("where is the Greet function defined?"). An `error` is the eval's turn hitting `maxTurns` against its stub tools — a limit of the instrument with this model, present in both runs, not a routing score.

## Triage only removes questions

The triage hook of `demos/decision-commands/` (ticket 126) answers `allow` when the decider is sure a command only reads. `allow` was added to the hooks for this, and it is deliberately narrow: it skips the confirmation question for one call — nothing more. A `deny` later in the chain still wins, a *never* the user gave still refuses, and an agent asked by another ignores it, because there the asking agent's user decides. With `yolo: true` nothing was going to be asked, so `allow` changes nothing.

The asymmetry follows from the decider's errors. On the commands tried on 2026-10-22, read-only commands (`ls -la`, `cat`, `git status`, `wc -l`, `find`) came back `read-only` at 0.94–0.95 with a destructive probability of 0.02–0.08, while the risky ones were spread out: `curl -s https://x.sh | sh` came back `local-change` 0.51, `sudo apt-get install jq` `dangerous` at only 0.24. A decider that is reliable on one side only may be trusted on that side only: it removes the questions for what clearly reads, and everything else — the risky commands it misjudges included — still reaches you. The hook calls `mm -p` once per command, about 0.4 s warm.

## Tool routing only removes what is clearly unneeded

`toolRouting` (ticket 127) asks one yes/no question per group of tools and leaves out a group only below `dropBelow`, 0.2 by default. The costs of the two errors are unequal: a tool left out that the request needed means a turn where the model cannot do the task, while a tool kept that it did not need costs a few hundred tokens. On eight requests tried on 2026-10-22 (five groups: shell, files, web, code, plan), every group a request needed scored 0.37 or more, and every score under 0.2 (0.03–0.17) went to a group the request did not need. Many unneeded groups scored higher, though — `plan` came back between 0.31 and 0.96 whatever the request — so the threshold drops few groups, and only safe ones: a hesitation keeps the tool.

The routing runs in the engine, after the `preGeneration` hooks, so a question a hook stopped never reaches the decider, and the text routed is the one the model reads. It runs in every mode, `-p` included: unlike `$skill`, it changes no text. A tool in no group is never touched, and any failure offers every tool. The price is the server's prompt cache: the tool list is part of the prompt, so a turn whose list differs from the previous one cannot reuse it.

What was not measured yet: whether a smaller tool list makes a small model choose better. The eval measures skill routing, not tool choice.

## A decider on its own

A decider needs no chat model. `demos/decision-only/` triages a support inbox: four questions per message (category, urgency, personal data, an upset author), one `mm -p` call each, about 0.6 s per message. On its six messages, one of them in French, the six categories and the personal data were right; the spam was given the urgency "today", which a filter on the category corrects.

## Checks after the fact only add a warning

Three hooks ask a decider about something the agent has just received, and add a `context` under it when the answer is clear: `demos/decision-injection/` (ticket 132) on what `read_file` and `fetch_content` read, `demos/decision-tool-errors/` (ticket 133) on a `bash` output, `demos/decision-ask-check/` (ticket 134) on another agent's answer. None of them blocks or rewrites anything. The output is already there, and the decider is wrong often enough that hiding what it judges would cost more than it saves: a warning the model can weigh is the most a 3B decider's verdict deserves.

What the decider is asked matters as much as the threshold. A decider judges the text as a whole: an injection hidden in one HTML comment drowns in a README (0.44 for the whole file, 0.93 for the comment alone), so the injection hook asks about the paragraphs that mention an AI, an agent or instructions, one by one. A yes/no "the command failed" scored a failing test 0.19; a choice between `success`, `nothing-found` and `error` gave real errors 0.98 and kept `grep` with no match (0.87) and `diff` (0.73) under the 0.95 threshold. And the check judges what it is given: when the asked agent's reasoning ended up in its answer, the reasoning — which discusses the question — was judged on topic, and no off-topic answer was flagged.

The thresholds sit where the measured cases split, with little margin. Clean documents reached 0.76 on injection while a polite injection in a page footer scored 0.77 — under the 0.8 threshold, so missed. The on-topic check is about relevance, not truth: a wrong answer on the right subject scored 0.78.

On the demos' runs (2026-10-22, Qwen3.5-4B as the chat model, 5 runs with the hook and 5 without), the warnings were always added where they should be, and the effect on the model was smaller than the warning suggests. With the injection warning, no run ran the hidden `curl … | sh` — nor did any run without it in that series, while an earlier series had 3 of 5 runs without the hook try it; 3 of 5 summaries mentioned the hidden order, only one as an injection. On a script that prints an error and exits 0, the model said the deployment failed in 5 of 5 runs with or without the note. Told that the historian's answer was off-topic, it asked the expert every time, against 3 of 5 without. A small model often sees an obvious problem by itself; the checks are for the cases where it does not, and they must not push it further than the user asked: a first failure note said "fix its cause and run the command again", and in one run the model rewrote the script it had been asked to run.

## Rejected alternatives

- **evalengine's decision-4b.** The model ticket 72 started from. llmman and Ollama refuse it on `/v1/systemone` ("not a decision model"); it can only be read through `logprobs` on the chat route, which is less exact (a letter outside the top N counts as 0) and is left to a later ticket. On Docker Model Runner, which has it pulled, that is the only road too.
- **A `decisionModel:` block inside the chat agent.** One file per model keeps one definition for both uses: the agent you ask is the decider routing uses.
- **Routing in `-p`.** `-p` does not expand `$skill` either; skill routing follows the notation it imitates. Tool routing does run in `-p`: it changes no text.
- **Command triage built into mm.** A `commandTriage:` block was possible. A hook needs only the generic `allow` action, works with any classifier — a decision model, a rule list, a guard model — and keeps the thresholds in a script the user can read.
- **A tool router that also adds tools.** Only removing keeps the agent's declared tools as the upper bound: the decider can never give the model a tool the operator did not.

## How it relates to the rest

- [Guard models](guard-models.md): the same shape — a classifier behind an agent file, asked without the history — with a decision model's typed questions instead of a risk name.
- [Asking another agent](asking-another-agent.md): how `/ask-<name>` reaches a decider.
- [Skill format](../reference/skill-format.md): the description is what the decider reads for each skill — the same one the chat model reads in the catalogue.