rickub
bots-garden/mini-mepublic Fork
⑂ 250d8d0
Branches
feature/agents-file feature/ask-agent-hooks feature/ask-cli feature/declared-tools feature/display-called-hooks feature/fetch-build-in feature/genkit-v1.13.1 feature/github-mcp-in-sbx feature/go-1.27.3 feature/hooks feature/improve-init feature/interactive-prompt feature/last-question feature/llmman-provider feature/log-errors feature/lsp-builtin feature/mcp-gateway-session feature/mcp-support feature/memory-search feature/mini-mes feature/mm-init feature/open-terminal feature/prompt-flag feature/response-request feature/resume-acp feature/sbx-template-kit feature/search-in-doc feature/selectors feature/skill-discovery feature/structured-output feature/system-one feature/talk-to-another-mm feature/tasks-builtin feature/try-again feature/unload-model feature/wasm-custom-tools feature/yolo feature/zed-skill-selector fix/generation-timeout fix/plan-mode fix/too-long-completion fix/tool-order fix/word-wrap improve-mem-usage main thinking
Tags
v0.0.0 v0.0.1 v0.0.2 v0.0.3 v0.0.4 v0.1.0 v0.1.1 v0.1.2 v0.1.3 v0.1.4 v0.1.5 v0.1.6 v0.1.7 v0.1.8 v0.1.9 v0.2.0 v0.2.1 v0.2.2 v0.2.3 v0.2.4 v0.2.5 v0.2.6 v0.3.0 v0.4.0 v0.5.0 v0.6.0 v0.7.0 v0.8.0 v0.9.0 v1.0.0 View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/bots-garden/mini-me.git
git clone ssh://git@rickub.com/bots-garden/mini-me.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Download ZIP

Merge feature/llmman-provider into main

rickub committed 2026-10-01T06:28:18Z Browse files
250d8d0 parents: 1d966fe 4692a6a
added .tickets/issues/0058-add-more-tools-to-the-sbx-template-redis-cli-to.yaml +13 -0
new file mode 100644
@@ -0,0 +1,13 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 58
3+title: Add more tools to the sbx template (redis, cli to parse source code ...)
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T04:39:22.452Z
9+updatedAt: 2026-10-01T04:39:22.452Z
10+labels: []
11+body: ""
12+tasks: []
13+comments: []
new file mode 100644
@@ -0,0 +1,13 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 58
3+title: Add more tools to the sbx template (redis, cli to parse source code ...)
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T04:39:22.452Z
9+updatedAt: 2026-10-01T04:39:22.452Z
10+labels: []
11+body: ""
12+tasks: []
13+comments: []
added .tickets/issues/0059-how-to-start-in-yolo-mode.yaml +106 -0
new file mode 100644
@@ -0,0 +1,106 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 59
3+title: How to start in yolo mode?
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T04:39:53.878Z
9+updatedAt: 2026-10-01T06:30:00.000Z
10+labels:
11+ - kind::feature
12+ - priority::low
13+related:
14+ - 57
15+body: |
16+ Question: is there a way to start mm in "yolo" mode (tools run without any
17+ confirmation)?
18+
19+ Answer as of 2026-10-01 (main 1d966fe): there is no --yolo flag and no
20+ dedicated key, but it depends on the front end:
21+ - Line mode (`mm agent.yaml`) and one-shot (`mm -p "…"`): never asks.
22+ bash / write_file / edit_file run at once (no sink, no gate).
23+ - Window mode (`mm -tui`): asks only with `permissions: ask`. Default is
24+ `auto` = yolo. With `auto`, MCP tools marked `ask` do not ask either
25+ (internal/tui/sink.go Allow returns true).
26+ - Editor (`mm -acp`): ALWAYS asks through the editor's dialog
27+ (internal/acp/sink.go Allow -> RequestPermission). No setting skips it;
28+ only "allow always", remembered per tool for the session.
29+ - MCP tools: `mcp.permissions` defaults to `ask` (also per server); set
30+ `auto` to skip.
31+ - `permissions` has no AGENT_* environment override: YAML only.
32+
33+ Yolo in the window mode today:
34+ permissions: auto
35+ mcp:
36+ permissions: auto
37+
38+ Proposed feature (later): a real switch, `--yolo`, that forces `auto`
39+ everywhere, ACP included (mm would approve without opening the editor's
40+ dialog). Side benefit: no pending permission dialog, so ticket 57's
41+ "watchdog kills the turn while a permission is pending" cannot happen.
42+
43+ Open questions:
44+ - Name: `--yolo` only, or also `AGENT_PERMISSIONS=auto` / a config key
45+ (`permissions: yolo`?) so the sandbox kits can set it via env.
46+ - ACP: should yolo be allowed there at all? The editor's dialog is the
47+ main safety an editor adds; maybe require the flag explicitly (never
48+ from config).
49+ - Should yolo override a per-server MCP `permissions: ask`, or respect it?
50+ - Make it visible: banner/status line "yolo" so nobody runs it by mistake.
51+tasks:
52+ - id: 1
53+ title: "Decide the open questions below (flag name, ACP scope, MCP scope, banner)"
54+ state: open
55+ priority: none
56+ author:
57+ name: k33g
58+ email: ph.charriere@gmail.com
59+ createdAt: 2026-10-01T06:30:00.000Z
60+ updatedAt: 2026-10-01T06:30:00.000Z
61+ - id: 2
62+ title: "Failing tests first: with yolo on, window mode (permissions: ask) and ACP Allow() return true without asking; MCP ask-tools run"
63+ state: open
64+ priority: none
65+ author:
66+ name: k33g
67+ email: ph.charriere@gmail.com
68+ createdAt: 2026-10-01T06:30:00.000Z
69+ updatedAt: 2026-10-01T06:30:00.000Z
70+ - id: 3
71+ title: "Flag --yolo (main.go, alongside -p/-tui/-acp) + optional env/config override; forces Permissions=auto and mcp.permissions=auto, per-server MCP permissions included"
72+ state: open
73+ priority: none
74+ author:
75+ name: k33g
76+ email: ph.charriere@gmail.com
77+ createdAt: 2026-10-01T06:30:00.000Z
78+ updatedAt: 2026-10-01T06:30:00.000Z
79+ - id: 4
80+ title: "ACP sink Allow(): short-circuit when yolo is on (no RequestPermission sent to the editor)"
81+ state: open
82+ priority: none
83+ author:
84+ name: k33g
85+ email: ph.charriere@gmail.com
86+ createdAt: 2026-10-01T06:30:00.000Z
87+ updatedAt: 2026-10-01T06:30:00.000Z
88+ - id: 5
89+ title: "Banner/status line shows that yolo is on (terminal, window, ACP agent info)"
90+ state: open
91+ priority: none
92+ author:
93+ name: k33g
94+ email: ph.charriere@gmail.com
95+ createdAt: 2026-10-01T06:30:00.000Z
96+ updatedAt: 2026-10-01T06:30:00.000Z
97+ - id: 6
98+ title: "Docs EN+FR: reference/cli.md, reference/configuration.md, how-to (window mode, editor); answer the question in the how-to"
99+ state: open
100+ priority: none
101+ author:
102+ name: k33g
103+ email: ph.charriere@gmail.com
104+ createdAt: 2026-10-01T06:30:00.000Z
105+ updatedAt: 2026-10-01T06:30:00.000Z
106+comments: []
new file mode 100644
@@ -0,0 +1,106 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 59
3+title: How to start in yolo mode?
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T04:39:53.878Z
9+updatedAt: 2026-10-01T06:30:00.000Z
10+labels:
11+ - kind::feature
12+ - priority::low
13+related:
14+ - 57
15+body: |
16+ Question: is there a way to start mm in "yolo" mode (tools run without any
17+ confirmation)?
18+
19+ Answer as of 2026-10-01 (main 1d966fe): there is no --yolo flag and no
20+ dedicated key, but it depends on the front end:
21+ - Line mode (`mm agent.yaml`) and one-shot (`mm -p "…"`): never asks.
22+ bash / write_file / edit_file run at once (no sink, no gate).
23+ - Window mode (`mm -tui`): asks only with `permissions: ask`. Default is
24+ `auto` = yolo. With `auto`, MCP tools marked `ask` do not ask either
25+ (internal/tui/sink.go Allow returns true).
26+ - Editor (`mm -acp`): ALWAYS asks through the editor's dialog
27+ (internal/acp/sink.go Allow -> RequestPermission). No setting skips it;
28+ only "allow always", remembered per tool for the session.
29+ - MCP tools: `mcp.permissions` defaults to `ask` (also per server); set
30+ `auto` to skip.
31+ - `permissions` has no AGENT_* environment override: YAML only.
32+
33+ Yolo in the window mode today:
34+ permissions: auto
35+ mcp:
36+ permissions: auto
37+
38+ Proposed feature (later): a real switch, `--yolo`, that forces `auto`
39+ everywhere, ACP included (mm would approve without opening the editor's
40+ dialog). Side benefit: no pending permission dialog, so ticket 57's
41+ "watchdog kills the turn while a permission is pending" cannot happen.
42+
43+ Open questions:
44+ - Name: `--yolo` only, or also `AGENT_PERMISSIONS=auto` / a config key
45+ (`permissions: yolo`?) so the sandbox kits can set it via env.
46+ - ACP: should yolo be allowed there at all? The editor's dialog is the
47+ main safety an editor adds; maybe require the flag explicitly (never
48+ from config).
49+ - Should yolo override a per-server MCP `permissions: ask`, or respect it?
50+ - Make it visible: banner/status line "yolo" so nobody runs it by mistake.
51+tasks:
52+ - id: 1
53+ title: "Decide the open questions below (flag name, ACP scope, MCP scope, banner)"
54+ state: open
55+ priority: none
56+ author:
57+ name: k33g
58+ email: ph.charriere@gmail.com
59+ createdAt: 2026-10-01T06:30:00.000Z
60+ updatedAt: 2026-10-01T06:30:00.000Z
61+ - id: 2
62+ title: "Failing tests first: with yolo on, window mode (permissions: ask) and ACP Allow() return true without asking; MCP ask-tools run"
63+ state: open
64+ priority: none
65+ author:
66+ name: k33g
67+ email: ph.charriere@gmail.com
68+ createdAt: 2026-10-01T06:30:00.000Z
69+ updatedAt: 2026-10-01T06:30:00.000Z
70+ - id: 3
71+ title: "Flag --yolo (main.go, alongside -p/-tui/-acp) + optional env/config override; forces Permissions=auto and mcp.permissions=auto, per-server MCP permissions included"
72+ state: open
73+ priority: none
74+ author:
75+ name: k33g
76+ email: ph.charriere@gmail.com
77+ createdAt: 2026-10-01T06:30:00.000Z
78+ updatedAt: 2026-10-01T06:30:00.000Z
79+ - id: 4
80+ title: "ACP sink Allow(): short-circuit when yolo is on (no RequestPermission sent to the editor)"
81+ state: open
82+ priority: none
83+ author:
84+ name: k33g
85+ email: ph.charriere@gmail.com
86+ createdAt: 2026-10-01T06:30:00.000Z
87+ updatedAt: 2026-10-01T06:30:00.000Z
88+ - id: 5
89+ title: "Banner/status line shows that yolo is on (terminal, window, ACP agent info)"
90+ state: open
91+ priority: none
92+ author:
93+ name: k33g
94+ email: ph.charriere@gmail.com
95+ createdAt: 2026-10-01T06:30:00.000Z
96+ updatedAt: 2026-10-01T06:30:00.000Z
97+ - id: 6
98+ title: "Docs EN+FR: reference/cli.md, reference/configuration.md, how-to (window mode, editor); answer the question in the how-to"
99+ state: open
100+ priority: none
101+ author:
102+ name: k33g
103+ email: ph.charriere@gmail.com
104+ createdAt: 2026-10-01T06:30:00.000Z
105+ updatedAt: 2026-10-01T06:30:00.000Z
106+comments: []
added .tickets/issues/0060-add-llmman-provider.yaml +121 -0
new file mode 100644
@@ -0,0 +1,121 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 60
3+title: Add llmman provider
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T04:48:02.829Z
9+updatedAt: 2026-10-01T07:00:00.000Z
10+labels:
11+ - kind::feature
12+ - priority::medium
13+related:
14+ - 61
15+ - 62
16+body: |
17+ Add llmman (https://github.com/llmmanorg/llmman, `llmman serve`) as a
18+ provider, OpenAI API only (/v1/chat/completions), and a demo config in
19+ demo-llmman/ using hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M.
20+tasks:
21+ - id: 1
22+ title: "Registry entry llmman (URL, fallback, key, progress, thinking off)"
23+ state: closed
24+ priority: none
25+ author:
26+ name: k33g
27+ email: ph.charriere@gmail.com
28+ createdAt: 2026-10-01T07:00:00.000Z
29+ updatedAt: 2026-10-01T07:00:00.000Z
30+ - id: 2
31+ title: "Probe: /v1/models warning + served window from /api/ps (not the stub /props)"
32+ state: closed
33+ priority: none
34+ author:
35+ name: k33g
36+ email: ph.charriere@gmail.com
37+ createdAt: 2026-10-01T07:00:00.000Z
38+ updatedAt: 2026-10-01T07:00:00.000Z
39+ - id: 3
40+ title: "Tests: Resolve rows + TestProbeLlmman"
41+ state: closed
42+ priority: none
43+ author:
44+ name: k33g
45+ email: ph.charriere@gmail.com
46+ createdAt: 2026-10-01T07:00:00.000Z
47+ updatedAt: 2026-10-01T07:00:00.000Z
48+ - id: 4
49+ title: "demo-llmman/agent.llmman.yaml with hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M"
50+ state: closed
51+ priority: none
52+ author:
53+ name: k33g
54+ email: ph.charriere@gmail.com
55+ createdAt: 2026-10-01T07:00:00.000Z
56+ updatedAt: 2026-10-01T07:00:00.000Z
57+ - id: 5
58+ title: "Docs EN+FR (how-to, reference, explanation, indexes)"
59+ state: closed
60+ priority: none
61+ author:
62+ name: k33g
63+ email: ph.charriere@gmail.com
64+ createdAt: 2026-10-01T07:00:00.000Z
65+ updatedAt: 2026-10-01T07:00:00.000Z
66+ - id: 6
67+ title: "Live check against llmman serve (tool call, ctx, compaction)"
68+ state: closed
69+ priority: none
70+ author:
71+ name: k33g
72+ email: ph.charriere@gmail.com
73+ createdAt: 2026-10-01T07:00:00.000Z
74+ updatedAt: 2026-10-01T07:00:00.000Z
75+ - id: 7
76+ title: "Quality gate (qlty) where available"
77+ state: open
78+ priority: none
79+ author:
80+ name: k33g
81+ email: ph.charriere@gmail.com
82+ createdAt: 2026-10-01T07:00:00.000Z
83+ updatedAt: 2026-10-01T07:00:00.000Z
84+ - id: 8
85+ title: "Review, commit, merge"
86+ state: open
87+ priority: none
88+ author:
89+ name: k33g
90+ email: ph.charriere@gmail.com
91+ createdAt: 2026-10-01T07:00:00.000Z
92+ updatedAt: 2026-10-01T07:00:00.000Z
93+comments:
94+ - id: 1
95+ author:
96+ name: claude
97+ email: noreply@anthropic.com
98+ createdAt: 2026-10-01T07:00:00.000Z
99+ body: |
100+ **Implemented** on branch `feature/llmman-provider` (uncommitted, 2026-10-01).
101+
102+ - Registry entry `llmman` (internal/engine/openai_compat.go): base
103+ `http://127.0.0.1:17434/v1`, fallback `http://host.docker.internal:17434/v1`,
104+ key `LLMMAN_API_KEY` (optional), `progress: true`,
105+ `thinkingOff: thinkingOffTemplate`.
106+ - Probe `probeLlmman` (internal/engine/llmman.go): /v1/models (a missing
107+ model is a warning: llmman pulls it on the first request) then the served
108+ window from /api/ps (shared reading in apips.go, also used by Ollama #62).
109+ - Measured on llmman 0.1.498: `/props` is a stub (n_ctx 4096, empty
110+ chat_template, loaded or not) -> NOT used. prompt_progress,
111+ reasoning_content, usage and `chat_template_kwargs.enable_thinking:false`
112+ pass through to its llama-server. /api/show's trained context rejected
113+ (LLMMAN_CONTEXT_LENGTH can serve less).
114+ - Verified live: one-shot prompts (bash tool call, read_skill, reasoning
115+ shown), REPL banner `ctx: 262144 (/api/ps)`, a real compaction
116+ (`compressed 8 messages -> 1 summary + 2 kept`).
117+ - Tests: Resolve rows, TestProbeLlmman; `go test ./...` green.
118+ - Docs EN+FR: how-to/run-with-llmman.md, reference (configuration, cli,
119+ mcp-configuration EN), explanation/providers.md, README indexes.
120+ - Not done: automatic download with progress -> ticket 61. Quality gate (qlty)
121+ not run: absent from the sandbox.
new file mode 100644
@@ -0,0 +1,121 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 60
3+title: Add llmman provider
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T04:48:02.829Z
9+updatedAt: 2026-10-01T07:00:00.000Z
10+labels:
11+ - kind::feature
12+ - priority::medium
13+related:
14+ - 61
15+ - 62
16+body: |
17+ Add llmman (https://github.com/llmmanorg/llmman, `llmman serve`) as a
18+ provider, OpenAI API only (/v1/chat/completions), and a demo config in
19+ demo-llmman/ using hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M.
20+tasks:
21+ - id: 1
22+ title: "Registry entry llmman (URL, fallback, key, progress, thinking off)"
23+ state: closed
24+ priority: none
25+ author:
26+ name: k33g
27+ email: ph.charriere@gmail.com
28+ createdAt: 2026-10-01T07:00:00.000Z
29+ updatedAt: 2026-10-01T07:00:00.000Z
30+ - id: 2
31+ title: "Probe: /v1/models warning + served window from /api/ps (not the stub /props)"
32+ state: closed
33+ priority: none
34+ author:
35+ name: k33g
36+ email: ph.charriere@gmail.com
37+ createdAt: 2026-10-01T07:00:00.000Z
38+ updatedAt: 2026-10-01T07:00:00.000Z
39+ - id: 3
40+ title: "Tests: Resolve rows + TestProbeLlmman"
41+ state: closed
42+ priority: none
43+ author:
44+ name: k33g
45+ email: ph.charriere@gmail.com
46+ createdAt: 2026-10-01T07:00:00.000Z
47+ updatedAt: 2026-10-01T07:00:00.000Z
48+ - id: 4
49+ title: "demo-llmman/agent.llmman.yaml with hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M"
50+ state: closed
51+ priority: none
52+ author:
53+ name: k33g
54+ email: ph.charriere@gmail.com
55+ createdAt: 2026-10-01T07:00:00.000Z
56+ updatedAt: 2026-10-01T07:00:00.000Z
57+ - id: 5
58+ title: "Docs EN+FR (how-to, reference, explanation, indexes)"
59+ state: closed
60+ priority: none
61+ author:
62+ name: k33g
63+ email: ph.charriere@gmail.com
64+ createdAt: 2026-10-01T07:00:00.000Z
65+ updatedAt: 2026-10-01T07:00:00.000Z
66+ - id: 6
67+ title: "Live check against llmman serve (tool call, ctx, compaction)"
68+ state: closed
69+ priority: none
70+ author:
71+ name: k33g
72+ email: ph.charriere@gmail.com
73+ createdAt: 2026-10-01T07:00:00.000Z
74+ updatedAt: 2026-10-01T07:00:00.000Z
75+ - id: 7
76+ title: "Quality gate (qlty) where available"
77+ state: open
78+ priority: none
79+ author:
80+ name: k33g
81+ email: ph.charriere@gmail.com
82+ createdAt: 2026-10-01T07:00:00.000Z
83+ updatedAt: 2026-10-01T07:00:00.000Z
84+ - id: 8
85+ title: "Review, commit, merge"
86+ state: open
87+ priority: none
88+ author:
89+ name: k33g
90+ email: ph.charriere@gmail.com
91+ createdAt: 2026-10-01T07:00:00.000Z
92+ updatedAt: 2026-10-01T07:00:00.000Z
93+comments:
94+ - id: 1
95+ author:
96+ name: claude
97+ email: noreply@anthropic.com
98+ createdAt: 2026-10-01T07:00:00.000Z
99+ body: |
100+ **Implemented** on branch `feature/llmman-provider` (uncommitted, 2026-10-01).
101+
102+ - Registry entry `llmman` (internal/engine/openai_compat.go): base
103+ `http://127.0.0.1:17434/v1`, fallback `http://host.docker.internal:17434/v1`,
104+ key `LLMMAN_API_KEY` (optional), `progress: true`,
105+ `thinkingOff: thinkingOffTemplate`.
106+ - Probe `probeLlmman` (internal/engine/llmman.go): /v1/models (a missing
107+ model is a warning: llmman pulls it on the first request) then the served
108+ window from /api/ps (shared reading in apips.go, also used by Ollama #62).
109+ - Measured on llmman 0.1.498: `/props` is a stub (n_ctx 4096, empty
110+ chat_template, loaded or not) -> NOT used. prompt_progress,
111+ reasoning_content, usage and `chat_template_kwargs.enable_thinking:false`
112+ pass through to its llama-server. /api/show's trained context rejected
113+ (LLMMAN_CONTEXT_LENGTH can serve less).
114+ - Verified live: one-shot prompts (bash tool call, read_skill, reasoning
115+ shown), REPL banner `ctx: 262144 (/api/ps)`, a real compaction
116+ (`compressed 8 messages -> 1 summary + 2 kept`).
117+ - Tests: Resolve rows, TestProbeLlmman; `go test ./...` green.
118+ - Docs EN+FR: how-to/run-with-llmman.md, reference (configuration, cli,
119+ mcp-configuration EN), explanation/providers.md, README indexes.
120+ - Not done: automatic download with progress -> ticket 61. Quality gate (qlty)
121+ not run: absent from the sandbox.
added .tickets/issues/0061-llmman-provider-download-the-model-automatically.yaml +13 -0
new file mode 100644
@@ -0,0 +1,13 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 61
3+title: "llmman provider: download the model automatically if does not exist"
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T05:48:44.919Z
9+updatedAt: 2026-10-01T05:48:44.919Z
10+labels: []
11+body: ""
12+tasks: []
13+comments: []
new file mode 100644
@@ -0,0 +1,13 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 61
3+title: "llmman provider: download the model automatically if does not exist"
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T05:48:44.919Z
9+updatedAt: 2026-10-01T05:48:44.919Z
10+labels: []
11+body: ""
12+tasks: []
13+comments: []
added .tickets/issues/0062-add-ollama-provider.yaml +146 -0
new file mode 100644
@@ -0,0 +1,146 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 62
3+title: Add Ollama provider
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T06:10:07.295Z
9+updatedAt: 2026-10-01T07:00:00.000Z
10+labels:
11+ - kind::feature
12+ - priority::medium
13+related:
14+ - 60
15+body: |
16+ Add Ollama as a provider, OpenAI API only (/v1/chat/completions), and a demo
17+ config in demo-ollama/ using qwen3.5:2b. Ollama, llmman, DMR and llama.cpp
18+ look alike, but each keeps its own registry entry so that a difference found
19+ later is one field to change.
20+tasks:
21+ - id: 1
22+ title: "Measure Ollama's /v1 (reasoning field, progress, thinking switch, window, 404)"
23+ state: closed
24+ priority: none
25+ author:
26+ name: k33g
27+ email: ph.charriere@gmail.com
28+ createdAt: 2026-10-01T07:00:00.000Z
29+ updatedAt: 2026-10-01T07:00:00.000Z
30+ - id: 2
31+ title: "Per-provider thinkingOff field; reasoning_effort none for Ollama"
32+ state: closed
33+ priority: none
34+ author:
35+ name: k33g
36+ email: ph.charriere@gmail.com
37+ createdAt: 2026-10-01T07:00:00.000Z
38+ updatedAt: 2026-10-01T07:00:00.000Z
39+ - id: 3
40+ title: "Registry entry ollama + probeOllama (shared /api/ps reading with llmman)"
41+ state: closed
42+ priority: none
43+ author:
44+ name: k33g
45+ email: ph.charriere@gmail.com
46+ createdAt: 2026-10-01T07:00:00.000Z
47+ updatedAt: 2026-10-01T07:00:00.000Z
48+ - id: 4
49+ title: "Tests: Resolve rows, TestProbeOllama, thinking-off middleware"
50+ state: closed
51+ priority: none
52+ author:
53+ name: k33g
54+ email: ph.charriere@gmail.com
55+ createdAt: 2026-10-01T07:00:00.000Z
56+ updatedAt: 2026-10-01T07:00:00.000Z
57+ - id: 5
58+ title: "demo-ollama/agent.ollama.yaml with qwen3.5:2b"
59+ state: closed
60+ priority: none
61+ author:
62+ name: k33g
63+ email: ph.charriere@gmail.com
64+ createdAt: 2026-10-01T07:00:00.000Z
65+ updatedAt: 2026-10-01T07:00:00.000Z
66+ - id: 6
67+ title: "Docs EN+FR (how-to, reference, explanation, indexes)"
68+ state: closed
69+ priority: none
70+ author:
71+ name: k33g
72+ email: ph.charriere@gmail.com
73+ createdAt: 2026-10-01T07:00:00.000Z
74+ updatedAt: 2026-10-01T07:00:00.000Z
75+ - id: 7
76+ title: "Live check against Ollama (tool call, ctx, compaction)"
77+ state: closed
78+ priority: none
79+ author:
80+ name: k33g
81+ email: ph.charriere@gmail.com
82+ createdAt: 2026-10-01T07:00:00.000Z
83+ updatedAt: 2026-10-01T07:00:00.000Z
84+ - id: 8
85+ title: "Check reasoning_effort none on a non-thinking model"
86+ state: open
87+ priority: none
88+ author:
89+ name: k33g
90+ email: ph.charriere@gmail.com
91+ createdAt: 2026-10-01T07:00:00.000Z
92+ updatedAt: 2026-10-01T07:00:00.000Z
93+ - id: 9
94+ title: "Quality gate (qlty) where available"
95+ state: open
96+ priority: none
97+ author:
98+ name: k33g
99+ email: ph.charriere@gmail.com
100+ createdAt: 2026-10-01T07:00:00.000Z
101+ updatedAt: 2026-10-01T07:00:00.000Z
102+ - id: 10
103+ title: "Review, commit, merge"
104+ state: open
105+ priority: none
106+ author:
107+ name: k33g
108+ email: ph.charriere@gmail.com
109+ createdAt: 2026-10-01T07:00:00.000Z
110+ updatedAt: 2026-10-01T07:00:00.000Z
111+comments:
112+ - id: 1
113+ author:
114+ name: claude
115+ email: noreply@anthropic.com
116+ createdAt: 2026-10-01T07:00:00.000Z
117+ body: |
118+ **Implemented** on branch `feature/llmman-provider` (same branch as #60,
119+ uncommitted, 2026-10-01).
120+
121+ Measured on Ollama 0.35.0 (qwen3.5:2b), three real differences:
122+ 1. Reasoning streams as `delta.reasoning` (not `reasoning_content`): already
123+ read by reasoning.go.
124+ 2. No `prompt_progress` (`return_progress` ignored) -> `progress: false`; the
125+ firstTokenTimeout has to cover the whole prompt and the model load.
126+ 3. `chat_template_kwargs.enable_thinking:false` is IGNORED;
127+ `reasoning_effort: "none"` is obeyed. Proved on a real compaction: with
128+ chat_template_kwargs the summary reasoned until the 300-token limit and
129+ came back empty; with reasoning_effort it succeeded.
130+ -> new per-provider field `thinkingOff` (usage.go: thinkingOffTemplate for
131+ dmr/llamacpp/llmman, thinkingOffEffort for ollama), replacing the old
132+ "progress implies chat_template_kwargs" coupling.
133+
134+ Also: /api/ps reports the served window like llmman (`context_length: 32768`),
135+ shared helper apips.go; Ollama's OpenAI API does NOT pull a missing model
136+ (404 `model '…' not found`) -> probe warning says `ollama pull`.
137+
138+ - Registry entry `ollama`: base `http://localhost:11434/v1`, fallback
139+ `http://host.docker.internal:11434/v1`, key `OLLAMA_API_KEY` (optional,
140+ for ollama.com).
141+ - Tests: Resolve rows, TestProbeOllama, TestStreamOptionsWithoutThinkingOllama;
142+ the unknown-provider tests now use `vllm`. `go test ./...` green.
143+ - Live: banner `ctx: 32768 (/api/ps)`, one-shot prompt with a bash tool call.
144+ - Docs EN+FR: how-to/run-with-ollama.md, reference, explanation, READMEs.
145+ - Not done: quality gate (qlty absent from the sandbox). Unverified:
146+ `reasoning_effort: "none"` on a NON-thinking model (none installed).
new file mode 100644
@@ -0,0 +1,146 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 62
3+title: Add Ollama provider
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-01T06:10:07.295Z
9+updatedAt: 2026-10-01T07:00:00.000Z
10+labels:
11+ - kind::feature
12+ - priority::medium
13+related:
14+ - 60
15+body: |
16+ Add Ollama as a provider, OpenAI API only (/v1/chat/completions), and a demo
17+ config in demo-ollama/ using qwen3.5:2b. Ollama, llmman, DMR and llama.cpp
18+ look alike, but each keeps its own registry entry so that a difference found
19+ later is one field to change.
20+tasks:
21+ - id: 1
22+ title: "Measure Ollama's /v1 (reasoning field, progress, thinking switch, window, 404)"
23+ state: closed
24+ priority: none
25+ author:
26+ name: k33g
27+ email: ph.charriere@gmail.com
28+ createdAt: 2026-10-01T07:00:00.000Z
29+ updatedAt: 2026-10-01T07:00:00.000Z
30+ - id: 2
31+ title: "Per-provider thinkingOff field; reasoning_effort none for Ollama"
32+ state: closed
33+ priority: none
34+ author:
35+ name: k33g
36+ email: ph.charriere@gmail.com
37+ createdAt: 2026-10-01T07:00:00.000Z
38+ updatedAt: 2026-10-01T07:00:00.000Z
39+ - id: 3
40+ title: "Registry entry ollama + probeOllama (shared /api/ps reading with llmman)"
41+ state: closed
42+ priority: none
43+ author:
44+ name: k33g
45+ email: ph.charriere@gmail.com
46+ createdAt: 2026-10-01T07:00:00.000Z
47+ updatedAt: 2026-10-01T07:00:00.000Z
48+ - id: 4
49+ title: "Tests: Resolve rows, TestProbeOllama, thinking-off middleware"
50+ state: closed
51+ priority: none
52+ author:
53+ name: k33g
54+ email: ph.charriere@gmail.com
55+ createdAt: 2026-10-01T07:00:00.000Z
56+ updatedAt: 2026-10-01T07:00:00.000Z
57+ - id: 5
58+ title: "demo-ollama/agent.ollama.yaml with qwen3.5:2b"
59+ state: closed
60+ priority: none
61+ author:
62+ name: k33g
63+ email: ph.charriere@gmail.com
64+ createdAt: 2026-10-01T07:00:00.000Z
65+ updatedAt: 2026-10-01T07:00:00.000Z
66+ - id: 6
67+ title: "Docs EN+FR (how-to, reference, explanation, indexes)"
68+ state: closed
69+ priority: none
70+ author:
71+ name: k33g
72+ email: ph.charriere@gmail.com
73+ createdAt: 2026-10-01T07:00:00.000Z
74+ updatedAt: 2026-10-01T07:00:00.000Z
75+ - id: 7
76+ title: "Live check against Ollama (tool call, ctx, compaction)"
77+ state: closed
78+ priority: none
79+ author:
80+ name: k33g
81+ email: ph.charriere@gmail.com
82+ createdAt: 2026-10-01T07:00:00.000Z
83+ updatedAt: 2026-10-01T07:00:00.000Z
84+ - id: 8
85+ title: "Check reasoning_effort none on a non-thinking model"
86+ state: open
87+ priority: none
88+ author:
89+ name: k33g
90+ email: ph.charriere@gmail.com
91+ createdAt: 2026-10-01T07:00:00.000Z
92+ updatedAt: 2026-10-01T07:00:00.000Z
93+ - id: 9
94+ title: "Quality gate (qlty) where available"
95+ state: open
96+ priority: none
97+ author:
98+ name: k33g
99+ email: ph.charriere@gmail.com
100+ createdAt: 2026-10-01T07:00:00.000Z
101+ updatedAt: 2026-10-01T07:00:00.000Z
102+ - id: 10
103+ title: "Review, commit, merge"
104+ state: open
105+ priority: none
106+ author:
107+ name: k33g
108+ email: ph.charriere@gmail.com
109+ createdAt: 2026-10-01T07:00:00.000Z
110+ updatedAt: 2026-10-01T07:00:00.000Z
111+comments:
112+ - id: 1
113+ author:
114+ name: claude
115+ email: noreply@anthropic.com
116+ createdAt: 2026-10-01T07:00:00.000Z
117+ body: |
118+ **Implemented** on branch `feature/llmman-provider` (same branch as #60,
119+ uncommitted, 2026-10-01).
120+
121+ Measured on Ollama 0.35.0 (qwen3.5:2b), three real differences:
122+ 1. Reasoning streams as `delta.reasoning` (not `reasoning_content`): already
123+ read by reasoning.go.
124+ 2. No `prompt_progress` (`return_progress` ignored) -> `progress: false`; the
125+ firstTokenTimeout has to cover the whole prompt and the model load.
126+ 3. `chat_template_kwargs.enable_thinking:false` is IGNORED;
127+ `reasoning_effort: "none"` is obeyed. Proved on a real compaction: with
128+ chat_template_kwargs the summary reasoned until the 300-token limit and
129+ came back empty; with reasoning_effort it succeeded.
130+ -> new per-provider field `thinkingOff` (usage.go: thinkingOffTemplate for
131+ dmr/llamacpp/llmman, thinkingOffEffort for ollama), replacing the old
132+ "progress implies chat_template_kwargs" coupling.
133+
134+ Also: /api/ps reports the served window like llmman (`context_length: 32768`),
135+ shared helper apips.go; Ollama's OpenAI API does NOT pull a missing model
136+ (404 `model '…' not found`) -> probe warning says `ollama pull`.
137+
138+ - Registry entry `ollama`: base `http://localhost:11434/v1`, fallback
139+ `http://host.docker.internal:11434/v1`, key `OLLAMA_API_KEY` (optional,
140+ for ollama.com).
141+ - Tests: Resolve rows, TestProbeOllama, TestStreamOptionsWithoutThinkingOllama;
142+ the unknown-provider tests now use `vllm`. `go test ./...` green.
143+ - Live: banner `ctx: 32768 (/api/ps)`, one-shot prompt with a bash tool call.
144+ - Docs EN+FR: how-to/run-with-ollama.md, reference, explanation, READMEs.
145+ - Not done: quality gate (qlty absent from the sandbox). Unverified:
146+ `reasoning_effort: "none"` on a NON-thinking model (none installed).
added demo-llmman/agent.llmman.yaml +199 -0
new file mode 100644
@@ -0,0 +1,199 @@
1+# Agent settings — mini-me served by llmman (https://github.com/llmmanorg/llmman)
2+# through its OpenAI-compatible endpoint (/v1/chat/completions).
3+#
4+# llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M # optional: the first question pulls it otherwise
5+# llmman serve # 127.0.0.1:17434
6+#
7+# llmman loads the model on the first request (in its own llama-server) and
8+# unloads it after five idle minutes.
9+provider: llmman
10+
11+# The model id exactly as `llmman list` / GET /v1/models spells it.
12+model: hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
13+
14+# `llmman serve`'s default bind; the OpenAI routes sit under /v1.
15+baseUrl: http://127.0.0.1:17434/v1
16+
17+# From a Docker sandbox, the daemon runs on the host.
18+fallback: http://host.docker.internal:17434/v1
19+
20+# Only when the daemon was started with LLMMAN_API_KEYS (mandatory when it
21+# listens beyond loopback); unset otherwise. The key never goes in this file.
22+# apiKeyEnv: LLMMAN_API_KEY
23+
24+# The `bash` tool.
25+# `false` removes it: the model can still read and edit files (editTools) and load skills,
26+# but never runs a command. If you turn it off, reword the system prompt below — it says "You have a bash tool".
27+bashTool: true
28+
29+# The built-in file tools: read_file, write_file, edit_file.
30+editTools: true
31+
32+# The built-in plan tool for managing multi-step plans.
33+# Set to false to disable plan creation and tracking via the `plan` tool.
34+planTool: true
35+
36+# Language servers for the `lsp` tool (definition, references, hover, symbols, diagnostics, rename).
37+# The file extension picks the server. A server missing from the PATH is skipped with a warning.
38+# No server listed = no `lsp` tool. Examples (uncomment what you have installed):
39+# lsp:
40+# servers:
41+# - name: gopls
42+# command: gopls
43+# extensions: [".go"]
44+# - name: typescript
45+# command: typescript-language-server
46+# args: ["--stdio"]
47+# extensions: [".ts", ".tsx", ".js", ".jsx"]
48+# - name: python
49+# command: pyright-langserver
50+# args: ["--stdio"]
51+# extensions: [".py"]
52+# - name: rust
53+# command: rust-analyzer
54+# extensions: [".rs"]
55+
56+# How many tokens the server actually serves.
57+# 0 = ask llmman: /api/ps tells it once the model is loaded, so the window is
58+# learned before the second question. Set it here if you want it from the start
59+# — and to the same value as LLMMAN_CONTEXT_LENGTH if you set that variable.
60+contextWindow: 0
61+
62+# Max number of characters a tool returns to the model.
63+# Context safeguard: beyond that, the output is truncated (beginning + end kept).
64+maxOutput: 16000
65+
66+# Max number of model <-> tools round trips for a single question.
67+# Raise it for tasks that need long chains of commands.
68+maxTurns: 40
69+
70+# Where the `read_skill` tool looks for its markdown procedures.
71+# The path is relative to the CURRENT DIRECTORY: run the agent from demo-llmman/.
72+skillsDir: ../skills
73+
74+# Persistent sessions, per project.
75+# When enabled, every conversation is saved as one JSON file under `dir`.
76+# `/sessions` lists them, `/resume` continues one; in an editor, "Reload Agent" replays the thread.
77+sessions:
78+ enabled: true
79+ dir: .mm/sessions
80+
81+# How many lines of a command's output are echoed to the terminal.
82+# 0 disables it.
83+previewLines: 20
84+
85+displayCommands: true
86+# Show what a thinking model reasons about before it answers (dim italic in the
87+# terminal, its own style in the window, a thought chunk in an editor).
88+# The reasoning is always READ — it is what keeps the watchdog alive while the
89+# model thinks — this only says whether to show it.
90+showThinking: true
91+
92+
93+# The system prompt: what the agent is, and what it is allowed to do.
94+# This is the knob to play with — it is the shortest path to changing behaviour.
95+system: |
96+ Your name is Bob.
97+ You are a coding agent working in a terminal.
98+ You have a "bash" tool to run shell commands.
99+ Use it to explore files, run tests, inspect the repository, etc.
100+ Chain several commands if needed, then answer clearly in English.
101+
102+ A request often mixes things you answer from yourself ("say hello") with
103+ things only a command can answer ("list the files"). Handle every part, in
104+ the order asked, and run a command for each part that needs one.
105+ Never state the contents of a file, the output of a command, or the state of
106+ the repository unless a command in THIS answer returned it. What you did not
107+ read, you do not know: run the command instead of recalling it.
108+
109+ SKILLS
110+ You have a second tool, `read_skill`. Its description lists the procedures
111+ available for this project — one per kind of task.
112+
113+ Any request to DO something to a Go project is a skill, not a shell command
114+ you invent. Match the request against that list, call `read_skill` FIRST,
115+ before any bash command, and then follow what it says step by step.
116+
117+ FILE EDITING
118+ You have three tools for files: `read_file`, `edit_file` and `write_file`.
119+ They are how a file gets read and changed here: each change is exact,
120+ checked before it is written, and comes back as a diff with line numbers.
121+ bash is for running things — building, testing, listing, searching.
122+
123+ - Read before you write: call `read_file` on the file (numbered=true when
124+ you need line numbers). You cannot target text you have not seen; never
125+ rely on what you think you remember about a file.
126+ - To change an existing file, call `edit_file` with one or more {old, new}
127+ pairs. `old` is copied from the file character for character — same
128+ spaces, same indentation, same line breaks — and appears exactly once:
129+ add the surrounding lines until it is unique. Several pairs are applied
130+ together, against the original file. An empty `new` deletes the text.
131+ - Call `write_file` only to create a file, or to rewrite one entirely and
132+ on purpose. On an existing file it replaces everything, including what
133+ you did not intend to touch.
134+ - Read the diff the tool returns: it says exactly what changed and on which
135+ line. If `edit_file` refuses — text not found, ambiguous, overlapping
136+ edits — read the file again and fix `old`. Do not fall back to
137+ `write_file` to force the change through.
138+ - After editing code, run the narrowest check with bash: the formatter, the
139+ compiler, or the test covering that file.
140+
141+ RULES
142+ - Keep everything the file already does, unless the user asked to remove it.
143+ - Touch only the files the request is about. Do not add tests, files or
144+ features that were not asked for.
145+ - Never run a git command unless the user says git, commit or push.
146+ - Never move, rename or delete a file unless the user asked for it.
147+ - Then answer in English, in a few lines.
148+ - If you don't know how to use a <cli>, run `<cli> --help` (or `<cli> help`)
149+ to understand the options, then run the command.
150+
151+ BACKGROUND JOBS
152+ Never let a command block the answer. Anything that serves, watches or runs
153+ long goes to the background, with BOTH streams redirected and its pid kept:
154+
155+ nohup <command> > /tmp/<job>.log 2>&1 & echo $! > /tmp/<job>.pid
156+
157+ Redirecting only stdout still blocks until the process exits. Read the
158+ `bg-jobs` skill before you wait on, inspect or stop such a job — each has a
159+ rule you cannot guess. Stop every job you started before you finish, and say
160+ which ones you left running.
161+
162+# Generation settings (OpenAI API keys). Low temperature for a coding agent:
163+# we want precise and reproducible answers, not creativity.
164+sampling:
165+ temperature: 0.0
166+ parallel_tool_calls: false
167+ top_p: 0.9
168+ max_tokens: 4096
169+
170+# Silence allowed between two tokens once the answer has started.
171+watchdogTimeout: 30s
172+
173+# Silence allowed while the model reads the prompt, before the first token:
174+# on a laptop a 25k-token prompt can take minutes. llama-server reports its
175+# progress meanwhile, through llmman ("Reading the prompt 12.0k/25.3k tokens"), and each report
176+# restarts this clock, so it only has to cover one batch of the prompt.
177+firstTokenTimeout: 3m
178+
179+# Context compression.
180+# ON here: the agent learns the window from /api/ps, so `threshold` has a number to apply to.
181+context:
182+ enabled: true
183+
184+ # Compress when the history reaches this share of the window.
185+ threshold: 75
186+
187+ # Fallback on the message count, for when the window is unknown.
188+ # One command costs 2 messages (call + response), so 80 is roughly 30 commands of history.
189+ maxMessages: 80
190+
191+ # How many recent questions (with their commands and answers) are kept as-is instead of being summarised.
192+ # The model works on the last few turns, so summarising them would make it redo what it just did.
193+ keepLastTurns: 3
194+
195+ # max_tokens of the summary request. Reasoning is turned off for it on this provider
196+ # (a thinking model would otherwise spend the budget thinking and write nothing).
197+ summaryMaxTokens: 1200
198+
199+ showStats: true
new file mode 100644
@@ -0,0 +1,199 @@
1+# Agent settings — mini-me served by llmman (https://github.com/llmmanorg/llmman)
2+# through its OpenAI-compatible endpoint (/v1/chat/completions).
3+#
4+# llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M # optional: the first question pulls it otherwise
5+# llmman serve # 127.0.0.1:17434
6+#
7+# llmman loads the model on the first request (in its own llama-server) and
8+# unloads it after five idle minutes.
9+provider: llmman
10+
11+# The model id exactly as `llmman list` / GET /v1/models spells it.
12+model: hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
13+
14+# `llmman serve`'s default bind; the OpenAI routes sit under /v1.
15+baseUrl: http://127.0.0.1:17434/v1
16+
17+# From a Docker sandbox, the daemon runs on the host.
18+fallback: http://host.docker.internal:17434/v1
19+
20+# Only when the daemon was started with LLMMAN_API_KEYS (mandatory when it
21+# listens beyond loopback); unset otherwise. The key never goes in this file.
22+# apiKeyEnv: LLMMAN_API_KEY
23+
24+# The `bash` tool.
25+# `false` removes it: the model can still read and edit files (editTools) and load skills,
26+# but never runs a command. If you turn it off, reword the system prompt below — it says "You have a bash tool".
27+bashTool: true
28+
29+# The built-in file tools: read_file, write_file, edit_file.
30+editTools: true
31+
32+# The built-in plan tool for managing multi-step plans.
33+# Set to false to disable plan creation and tracking via the `plan` tool.
34+planTool: true
35+
36+# Language servers for the `lsp` tool (definition, references, hover, symbols, diagnostics, rename).
37+# The file extension picks the server. A server missing from the PATH is skipped with a warning.
38+# No server listed = no `lsp` tool. Examples (uncomment what you have installed):
39+# lsp:
40+# servers:
41+# - name: gopls
42+# command: gopls
43+# extensions: [".go"]
44+# - name: typescript
45+# command: typescript-language-server
46+# args: ["--stdio"]
47+# extensions: [".ts", ".tsx", ".js", ".jsx"]
48+# - name: python
49+# command: pyright-langserver
50+# args: ["--stdio"]
51+# extensions: [".py"]
52+# - name: rust
53+# command: rust-analyzer
54+# extensions: [".rs"]
55+
56+# How many tokens the server actually serves.
57+# 0 = ask llmman: /api/ps tells it once the model is loaded, so the window is
58+# learned before the second question. Set it here if you want it from the start
59+# — and to the same value as LLMMAN_CONTEXT_LENGTH if you set that variable.
60+contextWindow: 0
61+
62+# Max number of characters a tool returns to the model.
63+# Context safeguard: beyond that, the output is truncated (beginning + end kept).
64+maxOutput: 16000
65+
66+# Max number of model <-> tools round trips for a single question.
67+# Raise it for tasks that need long chains of commands.
68+maxTurns: 40
69+
70+# Where the `read_skill` tool looks for its markdown procedures.
71+# The path is relative to the CURRENT DIRECTORY: run the agent from demo-llmman/.
72+skillsDir: ../skills
73+
74+# Persistent sessions, per project.
75+# When enabled, every conversation is saved as one JSON file under `dir`.
76+# `/sessions` lists them, `/resume` continues one; in an editor, "Reload Agent" replays the thread.
77+sessions:
78+ enabled: true
79+ dir: .mm/sessions
80+
81+# How many lines of a command's output are echoed to the terminal.
82+# 0 disables it.
83+previewLines: 20
84+
85+displayCommands: true
86+# Show what a thinking model reasons about before it answers (dim italic in the
87+# terminal, its own style in the window, a thought chunk in an editor).
88+# The reasoning is always READ — it is what keeps the watchdog alive while the
89+# model thinks — this only says whether to show it.
90+showThinking: true
91+
92+
93+# The system prompt: what the agent is, and what it is allowed to do.
94+# This is the knob to play with — it is the shortest path to changing behaviour.
95+system: |
96+ Your name is Bob.
97+ You are a coding agent working in a terminal.
98+ You have a "bash" tool to run shell commands.
99+ Use it to explore files, run tests, inspect the repository, etc.
100+ Chain several commands if needed, then answer clearly in English.
101+
102+ A request often mixes things you answer from yourself ("say hello") with
103+ things only a command can answer ("list the files"). Handle every part, in
104+ the order asked, and run a command for each part that needs one.
105+ Never state the contents of a file, the output of a command, or the state of
106+ the repository unless a command in THIS answer returned it. What you did not
107+ read, you do not know: run the command instead of recalling it.
108+
109+ SKILLS
110+ You have a second tool, `read_skill`. Its description lists the procedures
111+ available for this project — one per kind of task.
112+
113+ Any request to DO something to a Go project is a skill, not a shell command
114+ you invent. Match the request against that list, call `read_skill` FIRST,
115+ before any bash command, and then follow what it says step by step.
116+
117+ FILE EDITING
118+ You have three tools for files: `read_file`, `edit_file` and `write_file`.
119+ They are how a file gets read and changed here: each change is exact,
120+ checked before it is written, and comes back as a diff with line numbers.
121+ bash is for running things — building, testing, listing, searching.
122+
123+ - Read before you write: call `read_file` on the file (numbered=true when
124+ you need line numbers). You cannot target text you have not seen; never
125+ rely on what you think you remember about a file.
126+ - To change an existing file, call `edit_file` with one or more {old, new}
127+ pairs. `old` is copied from the file character for character — same
128+ spaces, same indentation, same line breaks — and appears exactly once:
129+ add the surrounding lines until it is unique. Several pairs are applied
130+ together, against the original file. An empty `new` deletes the text.
131+ - Call `write_file` only to create a file, or to rewrite one entirely and
132+ on purpose. On an existing file it replaces everything, including what
133+ you did not intend to touch.
134+ - Read the diff the tool returns: it says exactly what changed and on which
135+ line. If `edit_file` refuses — text not found, ambiguous, overlapping
136+ edits — read the file again and fix `old`. Do not fall back to
137+ `write_file` to force the change through.
138+ - After editing code, run the narrowest check with bash: the formatter, the
139+ compiler, or the test covering that file.
140+
141+ RULES
142+ - Keep everything the file already does, unless the user asked to remove it.
143+ - Touch only the files the request is about. Do not add tests, files or
144+ features that were not asked for.
145+ - Never run a git command unless the user says git, commit or push.
146+ - Never move, rename or delete a file unless the user asked for it.
147+ - Then answer in English, in a few lines.
148+ - If you don't know how to use a <cli>, run `<cli> --help` (or `<cli> help`)
149+ to understand the options, then run the command.
150+
151+ BACKGROUND JOBS
152+ Never let a command block the answer. Anything that serves, watches or runs
153+ long goes to the background, with BOTH streams redirected and its pid kept:
154+
155+ nohup <command> > /tmp/<job>.log 2>&1 & echo $! > /tmp/<job>.pid
156+
157+ Redirecting only stdout still blocks until the process exits. Read the
158+ `bg-jobs` skill before you wait on, inspect or stop such a job — each has a
159+ rule you cannot guess. Stop every job you started before you finish, and say
160+ which ones you left running.
161+
162+# Generation settings (OpenAI API keys). Low temperature for a coding agent:
163+# we want precise and reproducible answers, not creativity.
164+sampling:
165+ temperature: 0.0
166+ parallel_tool_calls: false
167+ top_p: 0.9
168+ max_tokens: 4096
169+
170+# Silence allowed between two tokens once the answer has started.
171+watchdogTimeout: 30s
172+
173+# Silence allowed while the model reads the prompt, before the first token:
174+# on a laptop a 25k-token prompt can take minutes. llama-server reports its
175+# progress meanwhile, through llmman ("Reading the prompt 12.0k/25.3k tokens"), and each report
176+# restarts this clock, so it only has to cover one batch of the prompt.
177+firstTokenTimeout: 3m
178+
179+# Context compression.
180+# ON here: the agent learns the window from /api/ps, so `threshold` has a number to apply to.
181+context:
182+ enabled: true
183+
184+ # Compress when the history reaches this share of the window.
185+ threshold: 75
186+
187+ # Fallback on the message count, for when the window is unknown.
188+ # One command costs 2 messages (call + response), so 80 is roughly 30 commands of history.
189+ maxMessages: 80
190+
191+ # How many recent questions (with their commands and answers) are kept as-is instead of being summarised.
192+ # The model works on the last few turns, so summarising them would make it redo what it just did.
193+ keepLastTurns: 3
194+
195+ # max_tokens of the summary request. Reasoning is turned off for it on this provider
196+ # (a thinking model would otherwise spend the budget thinking and write nothing).
197+ summaryMaxTokens: 1200
198+
199+ showStats: true
added demo-ollama/agent.ollama.yaml +202 -0
new file mode 100644
@@ -0,0 +1,202 @@
1+# Agent settings — mini-me served by Ollama (https://ollama.com) through its
2+# OpenAI-compatible endpoint (/v1/chat/completions).
3+#
4+# ollama pull qwen3.5:2b
5+# ollama serve # localhost:11434 (the desktop app starts it for you)
6+#
7+# Ollama loads the model on the first request and unloads it after five idle
8+# minutes. Unlike llmman, its OpenAI API does NOT pull a missing model: the
9+# question fails with "model not found" — pull it first.
10+provider: ollama
11+
12+# The model name exactly as `ollama list` (or GET /v1/models) spells it.
13+model: qwen3.5:2b
14+
15+# `ollama serve`'s default; the OpenAI routes sit under /v1.
16+baseUrl: http://localhost:11434/v1
17+
18+# From a Docker sandbox, Ollama runs on the host.
19+fallback: http://host.docker.internal:11434/v1
20+
21+# A local Ollama checks no key. Only for ollama.com's API (export the key,
22+# never put it here):
23+# apiKeyEnv: OLLAMA_API_KEY
24+
25+# The `bash` tool.
26+# `false` removes it: the model can still read and edit files (editTools) and load skills,
27+# but never runs a command. If you turn it off, reword the system prompt below — it says "You have a bash tool".
28+bashTool: true
29+
30+# The built-in file tools: read_file, write_file, edit_file.
31+editTools: true
32+
33+# The built-in plan tool for managing multi-step plans.
34+# Set to false to disable plan creation and tracking via the `plan` tool.
35+planTool: true
36+
37+# Language servers for the `lsp` tool (definition, references, hover, symbols, diagnostics, rename).
38+# The file extension picks the server. A server missing from the PATH is skipped with a warning.
39+# No server listed = no `lsp` tool. Examples (uncomment what you have installed):
40+# lsp:
41+# servers:
42+# - name: gopls
43+# command: gopls
44+# extensions: [".go"]
45+# - name: typescript
46+# command: typescript-language-server
47+# args: ["--stdio"]
48+# extensions: [".ts", ".tsx", ".js", ".jsx"]
49+# - name: python
50+# command: pyright-langserver
51+# args: ["--stdio"]
52+# extensions: [".py"]
53+# - name: rust
54+# command: rust-analyzer
55+# extensions: [".rs"]
56+
57+# How many tokens the server actually serves.
58+# 0 = ask Ollama: /api/ps tells it once the model is loaded, so the window is
59+# learned before the second question. Set it here if you want it from the start
60+# — and to the same value as OLLAMA_CONTEXT_LENGTH if you set that variable
61+# (the OpenAI API cannot change num_ctx per request).
62+contextWindow: 0
63+
64+# Max number of characters a tool returns to the model.
65+# Context safeguard: beyond that, the output is truncated (beginning + end kept).
66+maxOutput: 16000
67+
68+# Max number of model <-> tools round trips for a single question.
69+# Raise it for tasks that need long chains of commands.
70+maxTurns: 40
71+
72+# Where the `read_skill` tool looks for its markdown procedures.
73+# The path is relative to the CURRENT DIRECTORY: run the agent from demo-ollama/.
74+skillsDir: ../skills
75+
76+# Persistent sessions, per project.
77+# When enabled, every conversation is saved as one JSON file under `dir`.
78+# `/sessions` lists them, `/resume` continues one; in an editor, "Reload Agent" replays the thread.
79+sessions:
80+ enabled: true
81+ dir: .mm/sessions
82+
83+# How many lines of a command's output are echoed to the terminal.
84+# 0 disables it.
85+previewLines: 20
86+
87+displayCommands: true
88+# Show what a thinking model reasons about before it answers (dim italic in the
89+# terminal, its own style in the window, a thought chunk in an editor).
90+# The reasoning is always READ — it is what keeps the watchdog alive while the
91+# model thinks — this only says whether to show it.
92+showThinking: true
93+
94+
95+# The system prompt: what the agent is, and what it is allowed to do.
96+# This is the knob to play with — it is the shortest path to changing behaviour.
97+system: |
98+ Your name is Bob.
99+ You are a coding agent working in a terminal.
100+ You have a "bash" tool to run shell commands.
101+ Use it to explore files, run tests, inspect the repository, etc.
102+ Chain several commands if needed, then answer clearly in English.
103+
104+ A request often mixes things you answer from yourself ("say hello") with
105+ things only a command can answer ("list the files"). Handle every part, in
106+ the order asked, and run a command for each part that needs one.
107+ Never state the contents of a file, the output of a command, or the state of
108+ the repository unless a command in THIS answer returned it. What you did not
109+ read, you do not know: run the command instead of recalling it.
110+
111+ SKILLS
112+ You have a second tool, `read_skill`. Its description lists the procedures
113+ available for this project — one per kind of task.
114+
115+ Any request to DO something to a Go project is a skill, not a shell command
116+ you invent. Match the request against that list, call `read_skill` FIRST,
117+ before any bash command, and then follow what it says step by step.
118+
119+ FILE EDITING
120+ You have three tools for files: `read_file`, `edit_file` and `write_file`.
121+ They are how a file gets read and changed here: each change is exact,
122+ checked before it is written, and comes back as a diff with line numbers.
123+ bash is for running things — building, testing, listing, searching.
124+
125+ - Read before you write: call `read_file` on the file (numbered=true when
126+ you need line numbers). You cannot target text you have not seen; never
127+ rely on what you think you remember about a file.
128+ - To change an existing file, call `edit_file` with one or more {old, new}
129+ pairs. `old` is copied from the file character for character — same
130+ spaces, same indentation, same line breaks — and appears exactly once:
131+ add the surrounding lines until it is unique. Several pairs are applied
132+ together, against the original file. An empty `new` deletes the text.
133+ - Call `write_file` only to create a file, or to rewrite one entirely and
134+ on purpose. On an existing file it replaces everything, including what
135+ you did not intend to touch.
136+ - Read the diff the tool returns: it says exactly what changed and on which
137+ line. If `edit_file` refuses — text not found, ambiguous, overlapping
138+ edits — read the file again and fix `old`. Do not fall back to
139+ `write_file` to force the change through.
140+ - After editing code, run the narrowest check with bash: the formatter, the
141+ compiler, or the test covering that file.
142+
143+ RULES
144+ - Keep everything the file already does, unless the user asked to remove it.
145+ - Touch only the files the request is about. Do not add tests, files or
146+ features that were not asked for.
147+ - Never run a git command unless the user says git, commit or push.
148+ - Never move, rename or delete a file unless the user asked for it.
149+ - Then answer in English, in a few lines.
150+ - If you don't know how to use a <cli>, run `<cli> --help` (or `<cli> help`)
151+ to understand the options, then run the command.
152+
153+ BACKGROUND JOBS
154+ Never let a command block the answer. Anything that serves, watches or runs
155+ long goes to the background, with BOTH streams redirected and its pid kept:
156+
157+ nohup <command> > /tmp/<job>.log 2>&1 & echo $! > /tmp/<job>.pid
158+
159+ Redirecting only stdout still blocks until the process exits. Read the
160+ `bg-jobs` skill before you wait on, inspect or stop such a job — each has a
161+ rule you cannot guess. Stop every job you started before you finish, and say
162+ which ones you left running.
163+
164+# Generation settings (OpenAI API keys). Low temperature for a coding agent:
165+# we want precise and reproducible answers, not creativity.
166+sampling:
167+ temperature: 0.0
168+ parallel_tool_calls: false
169+ top_p: 0.9
170+ max_tokens: 4096
171+
172+# Silence allowed between two tokens once the answer has started.
173+watchdogTimeout: 30s
174+
175+# Silence allowed while the model reads the prompt, before the first token.
176+# Ollama sends no progress while it reads (no "Reading the prompt" line), so
177+# this one wait has to cover the whole prompt — and the model's loading on the
178+# first question.
179+firstTokenTimeout: 3m
180+
181+# Context compression.
182+# ON here: the agent learns the window from /api/ps, so `threshold` has a number to apply to.
183+context:
184+ enabled: true
185+
186+ # Compress when the history reaches this share of the window.
187+ threshold: 75
188+
189+ # Fallback on the message count, for when the window is unknown.
190+ # One command costs 2 messages (call + response), so 80 is roughly 30 commands of history.
191+ maxMessages: 80
192+
193+ # How many recent questions (with their commands and answers) are kept as-is instead of being summarised.
194+ # The model works on the last few turns, so summarising them would make it redo what it just did.
195+ keepLastTurns: 3
196+
197+ # max_tokens of the summary request. Reasoning is turned off for it on this
198+ # provider (reasoning_effort: "none"), or a thinking model like Qwen3.5 would
199+ # spend the budget thinking and write nothing.
200+ summaryMaxTokens: 1200
201+
202+ showStats: true
new file mode 100644
@@ -0,0 +1,202 @@
1+# Agent settings — mini-me served by Ollama (https://ollama.com) through its
2+# OpenAI-compatible endpoint (/v1/chat/completions).
3+#
4+# ollama pull qwen3.5:2b
5+# ollama serve # localhost:11434 (the desktop app starts it for you)
6+#
7+# Ollama loads the model on the first request and unloads it after five idle
8+# minutes. Unlike llmman, its OpenAI API does NOT pull a missing model: the
9+# question fails with "model not found" — pull it first.
10+provider: ollama
11+
12+# The model name exactly as `ollama list` (or GET /v1/models) spells it.
13+model: qwen3.5:2b
14+
15+# `ollama serve`'s default; the OpenAI routes sit under /v1.
16+baseUrl: http://localhost:11434/v1
17+
18+# From a Docker sandbox, Ollama runs on the host.
19+fallback: http://host.docker.internal:11434/v1
20+
21+# A local Ollama checks no key. Only for ollama.com's API (export the key,
22+# never put it here):
23+# apiKeyEnv: OLLAMA_API_KEY
24+
25+# The `bash` tool.
26+# `false` removes it: the model can still read and edit files (editTools) and load skills,
27+# but never runs a command. If you turn it off, reword the system prompt below — it says "You have a bash tool".
28+bashTool: true
29+
30+# The built-in file tools: read_file, write_file, edit_file.
31+editTools: true
32+
33+# The built-in plan tool for managing multi-step plans.
34+# Set to false to disable plan creation and tracking via the `plan` tool.
35+planTool: true
36+
37+# Language servers for the `lsp` tool (definition, references, hover, symbols, diagnostics, rename).
38+# The file extension picks the server. A server missing from the PATH is skipped with a warning.
39+# No server listed = no `lsp` tool. Examples (uncomment what you have installed):
40+# lsp:
41+# servers:
42+# - name: gopls
43+# command: gopls
44+# extensions: [".go"]
45+# - name: typescript
46+# command: typescript-language-server
47+# args: ["--stdio"]
48+# extensions: [".ts", ".tsx", ".js", ".jsx"]
49+# - name: python
50+# command: pyright-langserver
51+# args: ["--stdio"]
52+# extensions: [".py"]
53+# - name: rust
54+# command: rust-analyzer
55+# extensions: [".rs"]
56+
57+# How many tokens the server actually serves.
58+# 0 = ask Ollama: /api/ps tells it once the model is loaded, so the window is
59+# learned before the second question. Set it here if you want it from the start
60+# — and to the same value as OLLAMA_CONTEXT_LENGTH if you set that variable
61+# (the OpenAI API cannot change num_ctx per request).
62+contextWindow: 0
63+
64+# Max number of characters a tool returns to the model.
65+# Context safeguard: beyond that, the output is truncated (beginning + end kept).
66+maxOutput: 16000
67+
68+# Max number of model <-> tools round trips for a single question.
69+# Raise it for tasks that need long chains of commands.
70+maxTurns: 40
71+
72+# Where the `read_skill` tool looks for its markdown procedures.
73+# The path is relative to the CURRENT DIRECTORY: run the agent from demo-ollama/.
74+skillsDir: ../skills
75+
76+# Persistent sessions, per project.
77+# When enabled, every conversation is saved as one JSON file under `dir`.
78+# `/sessions` lists them, `/resume` continues one; in an editor, "Reload Agent" replays the thread.
79+sessions:
80+ enabled: true
81+ dir: .mm/sessions
82+
83+# How many lines of a command's output are echoed to the terminal.
84+# 0 disables it.
85+previewLines: 20
86+
87+displayCommands: true
88+# Show what a thinking model reasons about before it answers (dim italic in the
89+# terminal, its own style in the window, a thought chunk in an editor).
90+# The reasoning is always READ — it is what keeps the watchdog alive while the
91+# model thinks — this only says whether to show it.
92+showThinking: true
93+
94+
95+# The system prompt: what the agent is, and what it is allowed to do.
96+# This is the knob to play with — it is the shortest path to changing behaviour.
97+system: |
98+ Your name is Bob.
99+ You are a coding agent working in a terminal.
100+ You have a "bash" tool to run shell commands.
101+ Use it to explore files, run tests, inspect the repository, etc.
102+ Chain several commands if needed, then answer clearly in English.
103+
104+ A request often mixes things you answer from yourself ("say hello") with
105+ things only a command can answer ("list the files"). Handle every part, in
106+ the order asked, and run a command for each part that needs one.
107+ Never state the contents of a file, the output of a command, or the state of
108+ the repository unless a command in THIS answer returned it. What you did not
109+ read, you do not know: run the command instead of recalling it.
110+
111+ SKILLS
112+ You have a second tool, `read_skill`. Its description lists the procedures
113+ available for this project — one per kind of task.
114+
115+ Any request to DO something to a Go project is a skill, not a shell command
116+ you invent. Match the request against that list, call `read_skill` FIRST,
117+ before any bash command, and then follow what it says step by step.
118+
119+ FILE EDITING
120+ You have three tools for files: `read_file`, `edit_file` and `write_file`.
121+ They are how a file gets read and changed here: each change is exact,
122+ checked before it is written, and comes back as a diff with line numbers.
123+ bash is for running things — building, testing, listing, searching.
124+
125+ - Read before you write: call `read_file` on the file (numbered=true when
126+ you need line numbers). You cannot target text you have not seen; never
127+ rely on what you think you remember about a file.
128+ - To change an existing file, call `edit_file` with one or more {old, new}
129+ pairs. `old` is copied from the file character for character — same
130+ spaces, same indentation, same line breaks — and appears exactly once:
131+ add the surrounding lines until it is unique. Several pairs are applied
132+ together, against the original file. An empty `new` deletes the text.
133+ - Call `write_file` only to create a file, or to rewrite one entirely and
134+ on purpose. On an existing file it replaces everything, including what
135+ you did not intend to touch.
136+ - Read the diff the tool returns: it says exactly what changed and on which
137+ line. If `edit_file` refuses — text not found, ambiguous, overlapping
138+ edits — read the file again and fix `old`. Do not fall back to
139+ `write_file` to force the change through.
140+ - After editing code, run the narrowest check with bash: the formatter, the
141+ compiler, or the test covering that file.
142+
143+ RULES
144+ - Keep everything the file already does, unless the user asked to remove it.
145+ - Touch only the files the request is about. Do not add tests, files or
146+ features that were not asked for.
147+ - Never run a git command unless the user says git, commit or push.
148+ - Never move, rename or delete a file unless the user asked for it.
149+ - Then answer in English, in a few lines.
150+ - If you don't know how to use a <cli>, run `<cli> --help` (or `<cli> help`)
151+ to understand the options, then run the command.
152+
153+ BACKGROUND JOBS
154+ Never let a command block the answer. Anything that serves, watches or runs
155+ long goes to the background, with BOTH streams redirected and its pid kept:
156+
157+ nohup <command> > /tmp/<job>.log 2>&1 & echo $! > /tmp/<job>.pid
158+
159+ Redirecting only stdout still blocks until the process exits. Read the
160+ `bg-jobs` skill before you wait on, inspect or stop such a job — each has a
161+ rule you cannot guess. Stop every job you started before you finish, and say
162+ which ones you left running.
163+
164+# Generation settings (OpenAI API keys). Low temperature for a coding agent:
165+# we want precise and reproducible answers, not creativity.
166+sampling:
167+ temperature: 0.0
168+ parallel_tool_calls: false
169+ top_p: 0.9
170+ max_tokens: 4096
171+
172+# Silence allowed between two tokens once the answer has started.
173+watchdogTimeout: 30s
174+
175+# Silence allowed while the model reads the prompt, before the first token.
176+# Ollama sends no progress while it reads (no "Reading the prompt" line), so
177+# this one wait has to cover the whole prompt — and the model's loading on the
178+# first question.
179+firstTokenTimeout: 3m
180+
181+# Context compression.
182+# ON here: the agent learns the window from /api/ps, so `threshold` has a number to apply to.
183+context:
184+ enabled: true
185+
186+ # Compress when the history reaches this share of the window.
187+ threshold: 75
188+
189+ # Fallback on the message count, for when the window is unknown.
190+ # One command costs 2 messages (call + response), so 80 is roughly 30 commands of history.
191+ maxMessages: 80
192+
193+ # How many recent questions (with their commands and answers) are kept as-is instead of being summarised.
194+ # The model works on the last few turns, so summarising them would make it redo what it just did.
195+ keepLastTurns: 3
196+
197+ # max_tokens of the summary request. Reasoning is turned off for it on this
198+ # provider (reasoning_effort: "none"), or a thinking model like Qwen3.5 would
199+ # spend the budget thinking and write nothing.
200+ summaryMaxTokens: 1200
201+
202+ showStats: true
modified docs/en/README.md +3 -1
@@ -1,6 +1,6 @@
11 # mini-me — documentation (English)
22
3-`mm` is a coding agent that runs in your terminal — line by line, or in a single window with `mm -tui` — or inside an editor through the Agent Client Protocol. It talks to a local model server (Docker Model Runner or llama.cpp's `llama-server`) and works with a handful of built-in tools: `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — plus, when turned on, `lsp` and `fetch_content`.
3+`mm` is a coding agent that runs in your terminal — line by line, or in a single window with `mm -tui` — or inside an editor through the Agent Client Protocol. It talks to a local model server (Docker Model Runner, llama.cpp's `llama-server`, llmman or Ollama) and works with a handful of built-in tools: `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — plus, when turned on, `lsp` and `fetch_content`.
44
55 This documentation is organised along the four [Diátaxis](https://diataxis.fr) quadrants. Pick the one that matches what you need right now.
66
@@ -11,6 +11,8 @@ This documentation is organised along the four [Diátaxis](https://diataxis.fr)
1111 ## How-to guides — get a task done
1212
1313 - [Run mini-me on llama.cpp instead of Docker Model Runner](how-to/run-with-llama-cpp.md)
14+- [Run mini-me on llmman](how-to/run-with-llmman.md) — `provider: llmman`, the demo config in `demo-llmman/`
15+- [Run mini-me on Ollama](how-to/run-with-ollama.md) — `provider: ollama`, the demo config in `demo-ollama/`
1416 - [Use mini-me from an editor (ACP)](how-to/use-from-an-editor.md)
1517 - [Run mini-me in a Docker Sandbox](how-to/run-in-a-docker-sandbox.md) — the template and the two kits under `docker-sbx/`, window mode and editor
1618 - [Run mini-me with the v3 sandbox kits](how-to/run-with-sandbox-kits-v3.md) — the same two kits for the kit specification v3, under `docker-sbx-v3/`
@@ -1,6 +1,6 @@
1 # mini-me — documentation (English)1 # mini-me — documentation (English)
2 2
3-`mm` is a coding agent that runs in your terminal — line by line, or in a single window with `mm -tui` — or inside an editor through the Agent Client Protocol. It talks to a local model server (Docker Model Runner or llama.cpp's `llama-server`) and works with a handful of built-in tools: `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — plus, when turned on, `lsp` and `fetch_content`.3+`mm` is a coding agent that runs in your terminal — line by line, or in a single window with `mm -tui` — or inside an editor through the Agent Client Protocol. It talks to a local model server (Docker Model Runner, llama.cpp's `llama-server`, llmman or Ollama) and works with a handful of built-in tools: `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — plus, when turned on, `lsp` and `fetch_content`.
4 4
5 This documentation is organised along the four [Diátaxis](https://diataxis.fr) quadrants. Pick the one that matches what you need right now.5 This documentation is organised along the four [Diátaxis](https://diataxis.fr) quadrants. Pick the one that matches what you need right now.
6 6
@@ -11,6 +11,8 @@ This documentation is organised along the four [Diátaxis](https://diataxis.fr)
11 ## How-to guides — get a task done11 ## How-to guides — get a task done
12 12
13 - [Run mini-me on llama.cpp instead of Docker Model Runner](how-to/run-with-llama-cpp.md)13 - [Run mini-me on llama.cpp instead of Docker Model Runner](how-to/run-with-llama-cpp.md)
14+- [Run mini-me on llmman](how-to/run-with-llmman.md) — `provider: llmman`, the demo config in `demo-llmman/`
15+- [Run mini-me on Ollama](how-to/run-with-ollama.md) — `provider: ollama`, the demo config in `demo-ollama/`
14 - [Use mini-me from an editor (ACP)](how-to/use-from-an-editor.md)16 - [Use mini-me from an editor (ACP)](how-to/use-from-an-editor.md)
15 - [Run mini-me in a Docker Sandbox](how-to/run-in-a-docker-sandbox.md) — the template and the two kits under `docker-sbx/`, window mode and editor17 - [Run mini-me in a Docker Sandbox](how-to/run-in-a-docker-sandbox.md) — the template and the two kits under `docker-sbx/`, window mode and editor
16 - [Run mini-me with the v3 sandbox kits](how-to/run-with-sandbox-kits-v3.md) — the same two kits for the kit specification v3, under `docker-sbx-v3/`18 - [Run mini-me with the v3 sandbox kits](how-to/run-with-sandbox-kits-v3.md) — the same two kits for the kit specification v3, under `docker-sbx-v3/`
modified docs/en/explanation/providers.md +6 -2
@@ -2,7 +2,7 @@
22
33 ## What is this about?
44
5-`mm` talks to a local model server through the OpenAI chat-completions API. Two servers are supported out of the box, Docker Model Runner and llama.cpp's `llama-server`, under the names `dmr` and `llamacpp`. The engine package used to be tied to Docker Model Runner; the provider abstraction is what generalised it.
5+`mm` talks to a local model server through the OpenAI chat-completions API. Four servers are supported out of the box: Docker Model Runner, llama.cpp's `llama-server`, llmman and Ollama, under the names `dmr`, `llamacpp`, `llmman` and `ollama`. The engine package used to be tied to Docker Model Runner; the provider abstraction is what generalised it.
66
77 ## Why it is designed this way
88
@@ -16,6 +16,10 @@
1616
1717 **The probe is advice, never a failure.** On a demo machine the server is often started after the agent. The probe checks reachability, warns when the configured model is not in DMR's list, and on llama.cpp reads the served context size from `/props`, one level above `/v1`, because that endpoint reports what the server serves, not what the model was trained with. When the window is still unknown and compression is on, the REPL probes again before the first question.
1818
19+**llmman is a third registry entry, with its own probe.** [llmman](https://github.com/llmmanorg/llmman) (`llmman serve`, port 17434) is a daemon in front of one `llama-server` per loaded model, and speaks several APIs; `mm` uses only its OpenAI one. Everything llama.cpp-specific passes through it — prompt progress, reasoning, `enable_thinking: false` for summaries — so it gets the same treatment as `llamacpp`. Its `/props`, though, is a stub (`n_ctx` 4096, no chat template, measured on llmman 0.1.498), so the probe reads the served window from `/api/ps` instead, for the configured model, once it is loaded. The trained context from `/api/show` is not used: `LLMMAN_CONTEXT_LENGTH` can serve less, and a window too large means compression that never triggers before the server refuses the prompt. A model missing from `/v1/models` is only a warning: llmman pulls it on the first request.
20+
21+**Ollama is a fourth entry, and shows why each server keeps its own.** Ollama's `/v1` looks like the others, but measured on Ollama 0.35.0 it differs in three ways. Reasoning streams as `delta.reasoning`, which the tap already reads. No prompt progress comes back, so `return_progress` is not sent. And `chat_template_kwargs` is ignored: the summary of a compaction reasoned until its token limit and came back empty; `reasoning_effort: "none"` is what Ollama obeys, and with it the same compaction succeeded. That is why how to turn reasoning off is a per-provider field, not something implied by "is llama.cpp". Like llmman, Ollama reports the served window on `/api/ps` once the model is loaded (the two share that reading); unlike llmman, it does not pull a missing model, so the probe's warning says `ollama pull`.
22+
1923 **Errors are rewritten in the server's own words.** A stack of wrapped messages is noise; "tool calls need llama-server started with --jinja" is the whole diagnosis. The provider recognises connection refusals, the missing `--jinja` flag, 401/403, 404 with a pull or alias hint, 429 and 5xx. Unknown errors come back unchanged: a raw message beats a wrong hint.
2024
2125 **A stream the server breaks is retried, and its words are kept.** llama.cpp reports a failure in the middle of a streamed answer as an SSE `error:` event (the 200 status is already sent). openai-go only reads `data:` lines, so the message used to vanish behind `unexpected end of JSON input`. The tap that reads the stream for reasoning also picks up those events, and the model call that broke is retried once — that call only: the tools that ran earlier in the turn are not replayed, and text the broken attempt already streamed is not taken back. If it breaks again, the error shows the server's own message (`the model server reported an error while generating: Failed to parse input at pos …`), or says the connection broke. Either way the history is kept: asking to continue picks the work up.
@@ -34,5 +38,5 @@
3438
3539 - The two configuration files that parametrise each provider are listed in the [configuration reference](../reference/configuration.md).
3640 - The window the probe learns is what [context compression](context-compression.md) measures against.
37-- Switching servers for one run is covered in [how to run on llama.cpp](../how-to/run-with-llama-cpp.md).
41+- Switching servers for one run is covered in [how to run on llama.cpp](../how-to/run-with-llama-cpp.md) [how to run on llmman](../how-to/run-with-llmman.md) and [how to run on Ollama](../how-to/run-with-ollama.md).
3842 - Checking the tool order and measuring the cache gain on your own server: [how to check that the tools go out in a stable order](../how-to/verify-tool-order.md).
@@ -2,7 +2,7 @@
2 2
3 ## What is this about?3 ## What is this about?
4 4
5-`mm` talks to a local model server through the OpenAI chat-completions API. Two servers are supported out of the box, Docker Model Runner and llama.cpp's `llama-server`, under the names `dmr` and `llamacpp`. The engine package used to be tied to Docker Model Runner; the provider abstraction is what generalised it.5+`mm` talks to a local model server through the OpenAI chat-completions API. Four servers are supported out of the box: Docker Model Runner, llama.cpp's `llama-server`, llmman and Ollama, under the names `dmr`, `llamacpp`, `llmman` and `ollama`. The engine package used to be tied to Docker Model Runner; the provider abstraction is what generalised it.
6 6
7 ## Why it is designed this way7 ## Why it is designed this way
8 8
@@ -16,6 +16,10 @@
16 16
17 **The probe is advice, never a failure.** On a demo machine the server is often started after the agent. The probe checks reachability, warns when the configured model is not in DMR's list, and on llama.cpp reads the served context size from `/props`, one level above `/v1`, because that endpoint reports what the server serves, not what the model was trained with. When the window is still unknown and compression is on, the REPL probes again before the first question.17 **The probe is advice, never a failure.** On a demo machine the server is often started after the agent. The probe checks reachability, warns when the configured model is not in DMR's list, and on llama.cpp reads the served context size from `/props`, one level above `/v1`, because that endpoint reports what the server serves, not what the model was trained with. When the window is still unknown and compression is on, the REPL probes again before the first question.
18 18
19+**llmman is a third registry entry, with its own probe.** [llmman](https://github.com/llmmanorg/llmman) (`llmman serve`, port 17434) is a daemon in front of one `llama-server` per loaded model, and speaks several APIs; `mm` uses only its OpenAI one. Everything llama.cpp-specific passes through it — prompt progress, reasoning, `enable_thinking: false` for summaries — so it gets the same treatment as `llamacpp`. Its `/props`, though, is a stub (`n_ctx` 4096, no chat template, measured on llmman 0.1.498), so the probe reads the served window from `/api/ps` instead, for the configured model, once it is loaded. The trained context from `/api/show` is not used: `LLMMAN_CONTEXT_LENGTH` can serve less, and a window too large means compression that never triggers before the server refuses the prompt. A model missing from `/v1/models` is only a warning: llmman pulls it on the first request.
20+
21+**Ollama is a fourth entry, and shows why each server keeps its own.** Ollama's `/v1` looks like the others, but measured on Ollama 0.35.0 it differs in three ways. Reasoning streams as `delta.reasoning`, which the tap already reads. No prompt progress comes back, so `return_progress` is not sent. And `chat_template_kwargs` is ignored: the summary of a compaction reasoned until its token limit and came back empty; `reasoning_effort: "none"` is what Ollama obeys, and with it the same compaction succeeded. That is why how to turn reasoning off is a per-provider field, not something implied by "is llama.cpp". Like llmman, Ollama reports the served window on `/api/ps` once the model is loaded (the two share that reading); unlike llmman, it does not pull a missing model, so the probe's warning says `ollama pull`.
22+
19 **Errors are rewritten in the server's own words.** A stack of wrapped messages is noise; "tool calls need llama-server started with --jinja" is the whole diagnosis. The provider recognises connection refusals, the missing `--jinja` flag, 401/403, 404 with a pull or alias hint, 429 and 5xx. Unknown errors come back unchanged: a raw message beats a wrong hint.23 **Errors are rewritten in the server's own words.** A stack of wrapped messages is noise; "tool calls need llama-server started with --jinja" is the whole diagnosis. The provider recognises connection refusals, the missing `--jinja` flag, 401/403, 404 with a pull or alias hint, 429 and 5xx. Unknown errors come back unchanged: a raw message beats a wrong hint.
20 24
21 **A stream the server breaks is retried, and its words are kept.** llama.cpp reports a failure in the middle of a streamed answer as an SSE `error:` event (the 200 status is already sent). openai-go only reads `data:` lines, so the message used to vanish behind `unexpected end of JSON input`. The tap that reads the stream for reasoning also picks up those events, and the model call that broke is retried once — that call only: the tools that ran earlier in the turn are not replayed, and text the broken attempt already streamed is not taken back. If it breaks again, the error shows the server's own message (`the model server reported an error while generating: Failed to parse input at pos …`), or says the connection broke. Either way the history is kept: asking to continue picks the work up.25 **A stream the server breaks is retried, and its words are kept.** llama.cpp reports a failure in the middle of a streamed answer as an SSE `error:` event (the 200 status is already sent). openai-go only reads `data:` lines, so the message used to vanish behind `unexpected end of JSON input`. The tap that reads the stream for reasoning also picks up those events, and the model call that broke is retried once — that call only: the tools that ran earlier in the turn are not replayed, and text the broken attempt already streamed is not taken back. If it breaks again, the error shows the server's own message (`the model server reported an error while generating: Failed to parse input at pos …`), or says the connection broke. Either way the history is kept: asking to continue picks the work up.
@@ -34,5 +38,5 @@
34 38
35 - The two configuration files that parametrise each provider are listed in the [configuration reference](../reference/configuration.md).39 - The two configuration files that parametrise each provider are listed in the [configuration reference](../reference/configuration.md).
36 - The window the probe learns is what [context compression](context-compression.md) measures against.40 - The window the probe learns is what [context compression](context-compression.md) measures against.
37-- Switching servers for one run is covered in [how to run on llama.cpp](../how-to/run-with-llama-cpp.md).41+- Switching servers for one run is covered in [how to run on llama.cpp](../how-to/run-with-llama-cpp.md) [how to run on llmman](../how-to/run-with-llmman.md) and [how to run on Ollama](../how-to/run-with-ollama.md).
38 - Checking the tool order and measuring the cache gain on your own server: [how to check that the tools go out in a stable order](../how-to/verify-tool-order.md).42 - Checking the tool order and measuring the cache gain on your own server: [how to check that the tools go out in a stable order](../how-to/verify-tool-order.md).
added docs/en/how-to/run-with-llmman.md +39 -0
new file mode 100644
@@ -0,0 +1,39 @@
1+# How to run mini-me on llmman
2+
3+This guide shows how to point `mm` at [llmman](https://github.com/llmmanorg/llmman) through its OpenAI-compatible endpoint. It assumes llmman is installed and you know how to build `mm`.
4+
5+## Steps
6+
7+1. Start the daemon. It listens on `127.0.0.1:17434`:
8+
9+ ```bash
10+ llmman serve
11+ ```
12+
13+2. Optionally pull the model first. Otherwise llmman pulls it on the first question, and that question waits for the download:
14+
15+ ```bash
16+ llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
17+ ```
18+
19+3. Run the demo configuration from its directory (its `skillsDir` is `../skills`):
20+
21+ ```bash
22+ cd demo-llmman
23+ ../mm agent.llmman.yaml
24+ ```
25+
26+4. Check the banner. `provider: llmman` confirms the provider. `ctx: 262144 (/api/ps)` means the model was already loaded and `mm` read the window it is served with; `ctx: unknown` means it was not loaded yet — `mm` asks again before the next question and prints `ctx: … (/api/ps)` once it knows.
27+
28+## Variants
29+
30+- **Another model.** Set `model:` to the id exactly as `llmman list` (or `GET /v1/models`) spells it.
31+- **From a Docker sandbox.** Nothing to change: when `127.0.0.1:17434` does not answer, `mm` falls back to `http://host.docker.internal:17434/v1`, the daemon on the host.
32+- **Daemon started with `LLMMAN_API_KEYS`** (mandatory when it listens beyond loopback). Export one of the keys in `LLMMAN_API_KEY`, the provider's default variable, or name another variable under `apiKeyEnv`.
33+- **`LLMMAN_CONTEXT_LENGTH` set on the daemon.** `/api/ps` already reports it once the model is loaded; set `contextWindow` to the same value if you want compression to know it from the first question.
34+- **Keep your own `agent.yaml`.** `AGENT_PROVIDER=llmman ./mm` works as long as the file leaves `baseUrl` empty.
35+
36+## See also
37+
38+- [Providers: one implementation per protocol](../explanation/providers.md) — why llmman's window comes from `/api/ps`, not `/props`.
39+- [Configuration reference](../reference/configuration.md) — provider defaults.
new file mode 100644
@@ -0,0 +1,39 @@
1+# How to run mini-me on llmman
2+
3+This guide shows how to point `mm` at [llmman](https://github.com/llmmanorg/llmman) through its OpenAI-compatible endpoint. It assumes llmman is installed and you know how to build `mm`.
4+
5+## Steps
6+
7+1. Start the daemon. It listens on `127.0.0.1:17434`:
8+
9+ ```bash
10+ llmman serve
11+ ```
12+
13+2. Optionally pull the model first. Otherwise llmman pulls it on the first question, and that question waits for the download:
14+
15+ ```bash
16+ llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
17+ ```
18+
19+3. Run the demo configuration from its directory (its `skillsDir` is `../skills`):
20+
21+ ```bash
22+ cd demo-llmman
23+ ../mm agent.llmman.yaml
24+ ```
25+
26+4. Check the banner. `provider: llmman` confirms the provider. `ctx: 262144 (/api/ps)` means the model was already loaded and `mm` read the window it is served with; `ctx: unknown` means it was not loaded yet — `mm` asks again before the next question and prints `ctx: … (/api/ps)` once it knows.
27+
28+## Variants
29+
30+- **Another model.** Set `model:` to the id exactly as `llmman list` (or `GET /v1/models`) spells it.
31+- **From a Docker sandbox.** Nothing to change: when `127.0.0.1:17434` does not answer, `mm` falls back to `http://host.docker.internal:17434/v1`, the daemon on the host.
32+- **Daemon started with `LLMMAN_API_KEYS`** (mandatory when it listens beyond loopback). Export one of the keys in `LLMMAN_API_KEY`, the provider's default variable, or name another variable under `apiKeyEnv`.
33+- **`LLMMAN_CONTEXT_LENGTH` set on the daemon.** `/api/ps` already reports it once the model is loaded; set `contextWindow` to the same value if you want compression to know it from the first question.
34+- **Keep your own `agent.yaml`.** `AGENT_PROVIDER=llmman ./mm` works as long as the file leaves `baseUrl` empty.
35+
36+## See also
37+
38+- [Providers: one implementation per protocol](../explanation/providers.md) — why llmman's window comes from `/api/ps`, not `/props`.
39+- [Configuration reference](../reference/configuration.md) — provider defaults.
added docs/en/how-to/run-with-ollama.md +40 -0
new file mode 100644
@@ -0,0 +1,40 @@
1+# How to run mini-me on Ollama
2+
3+This guide shows how to point `mm` at [Ollama](https://ollama.com) through its OpenAI-compatible endpoint. It assumes Ollama is installed and you know how to build `mm`.
4+
5+## Steps
6+
7+1. Pull the model. Ollama's OpenAI API does not pull a missing model: a question to it fails with `model '…' not found`:
8+
9+ ```bash
10+ ollama pull qwen3.5:2b
11+ ```
12+
13+2. Make sure the server runs on `localhost:11434` (the desktop app starts it; otherwise `ollama serve`).
14+
15+3. Run the demo configuration from its directory (its `skillsDir` is `../skills`):
16+
17+ ```bash
18+ cd demo-ollama
19+ ../mm agent.ollama.yaml
20+ ```
21+
22+4. Check the banner. `provider: ollama` confirms the provider. `ctx: 32768 (/api/ps)` means the model was already loaded and `mm` read the window it is served with; `ctx: unknown` means it was not loaded yet — `mm` asks again before the next question.
23+
24+## Variants
25+
26+- **Another model.** Set `model:` to the name exactly as `ollama list` spells it (`qwen3.5:9b`, …).
27+- **From a Docker sandbox.** Nothing to change: when `localhost:11434` does not answer, `mm` falls back to `http://host.docker.internal:11434/v1`, the Ollama on the host.
28+- **A larger window.** The OpenAI API cannot set `num_ctx` per request: start Ollama with `OLLAMA_CONTEXT_LENGTH`, and put the same value in `contextWindow` if you want compression to know it from the first question.
29+- **ollama.com's API.** Set `baseUrl` to its OpenAI endpoint and export the key in `OLLAMA_API_KEY` (or name another variable under `apiKeyEnv`).
30+- **Keep your own `agent.yaml`.** `AGENT_PROVIDER=ollama ./mm` works as long as the file leaves `baseUrl` empty.
31+
32+## What differs from the llama.cpp servers
33+
34+- No `Reading the prompt …` line: Ollama sends no progress while it reads, so `firstTokenTimeout` has to cover the whole prompt and the model's loading.
35+- The summary of a compaction turns reasoning off with `reasoning_effort: "none"` — Ollama ignores `chat_template_kwargs`.
36+
37+## See also
38+
39+- [Providers: one implementation per protocol](../explanation/providers.md) — what was measured on Ollama, and why each server keeps its own entry.
40+- [Configuration reference](../reference/configuration.md) — provider defaults.
new file mode 100644
@@ -0,0 +1,40 @@
1+# How to run mini-me on Ollama
2+
3+This guide shows how to point `mm` at [Ollama](https://ollama.com) through its OpenAI-compatible endpoint. It assumes Ollama is installed and you know how to build `mm`.
4+
5+## Steps
6+
7+1. Pull the model. Ollama's OpenAI API does not pull a missing model: a question to it fails with `model '…' not found`:
8+
9+ ```bash
10+ ollama pull qwen3.5:2b
11+ ```
12+
13+2. Make sure the server runs on `localhost:11434` (the desktop app starts it; otherwise `ollama serve`).
14+
15+3. Run the demo configuration from its directory (its `skillsDir` is `../skills`):
16+
17+ ```bash
18+ cd demo-ollama
19+ ../mm agent.ollama.yaml
20+ ```
21+
22+4. Check the banner. `provider: ollama` confirms the provider. `ctx: 32768 (/api/ps)` means the model was already loaded and `mm` read the window it is served with; `ctx: unknown` means it was not loaded yet — `mm` asks again before the next question.
23+
24+## Variants
25+
26+- **Another model.** Set `model:` to the name exactly as `ollama list` spells it (`qwen3.5:9b`, …).
27+- **From a Docker sandbox.** Nothing to change: when `localhost:11434` does not answer, `mm` falls back to `http://host.docker.internal:11434/v1`, the Ollama on the host.
28+- **A larger window.** The OpenAI API cannot set `num_ctx` per request: start Ollama with `OLLAMA_CONTEXT_LENGTH`, and put the same value in `contextWindow` if you want compression to know it from the first question.
29+- **ollama.com's API.** Set `baseUrl` to its OpenAI endpoint and export the key in `OLLAMA_API_KEY` (or name another variable under `apiKeyEnv`).
30+- **Keep your own `agent.yaml`.** `AGENT_PROVIDER=ollama ./mm` works as long as the file leaves `baseUrl` empty.
31+
32+## What differs from the llama.cpp servers
33+
34+- No `Reading the prompt …` line: Ollama sends no progress while it reads, so `firstTokenTimeout` has to cover the whole prompt and the model's loading.
35+- The summary of a compaction turns reasoning off with `reasoning_effort: "none"` — Ollama ignores `chat_template_kwargs`.
36+
37+## See also
38+
39+- [Providers: one implementation per protocol](../explanation/providers.md) — what was measured on Ollama, and why each server keeps its own entry.
40+- [Configuration reference](../reference/configuration.md) — provider defaults.
modified docs/en/reference/cli.md +1 -1
@@ -55,7 +55,7 @@ The first existing source wins.
5555 | `AGENT_MODEL` | Overrides the `model` key. |
5656 | `AGENT_BASE_URL` | Overrides `baseUrl` and disables the fallback probe. |
5757 | `DMR_BASE_URL` | Same as `AGENT_BASE_URL`, for the `dmr` provider only; `AGENT_BASE_URL` wins when both are set. |
58-| Variable named by `apiKeyEnv` | Holds the API key. Default name: none for `dmr`, `LLAMA_API_KEY` for `llamacpp`. |
58+| Variable named by `apiKeyEnv` | Holds the API key. Default name: none for `dmr`, `LLAMA_API_KEY` for `llamacpp`, `LLMMAN_API_KEY` for `llmman`, `OLLAMA_API_KEY` for `ollama`. |
5959
6060 ## Start-up output
6161
@@ -55,7 +55,7 @@ The first existing source wins.
55 | `AGENT_MODEL` | Overrides the `model` key. |55 | `AGENT_MODEL` | Overrides the `model` key. |
56 | `AGENT_BASE_URL` | Overrides `baseUrl` and disables the fallback probe. |56 | `AGENT_BASE_URL` | Overrides `baseUrl` and disables the fallback probe. |
57 | `DMR_BASE_URL` | Same as `AGENT_BASE_URL`, for the `dmr` provider only; `AGENT_BASE_URL` wins when both are set. |57 | `DMR_BASE_URL` | Same as `AGENT_BASE_URL`, for the `dmr` provider only; `AGENT_BASE_URL` wins when both are set. |
58-| Variable named by `apiKeyEnv` | Holds the API key. Default name: none for `dmr`, `LLAMA_API_KEY` for `llamacpp`. |58+| Variable named by `apiKeyEnv` | Holds the API key. Default name: none for `dmr`, `LLAMA_API_KEY` for `llamacpp`, `LLMMAN_API_KEY` for `llmman`, `OLLAMA_API_KEY` for `ollama`. |
59 59
60 ## Start-up output60 ## Start-up output
61 61
modified docs/en/reference/configuration.md +6 -4
@@ -6,7 +6,7 @@
66
77 | Key | Type | Default | Description |
88 |-----|------|---------|-------------|
9-| `provider` | string | `dmr` | Registry key of the LLM server: `dmr` or `llamacpp`. Overridden by `AGENT_PROVIDER`. |
9+| `provider` | string | `dmr` | Registry key of the LLM server: `dmr`, `llamacpp`, `llmman` or `ollama`. Overridden by `AGENT_PROVIDER`. |
1010 | `model` | string | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Model identifier in the provider's naming. Overridden by `AGENT_MODEL`. |
1111 | `baseUrl` | string | `""` (provider default) | OpenAI-compatible endpoint tried first. Overridden by `AGENT_BASE_URL`. |
1212 | `fallback` | string or absent | absent (provider default) | Endpoint used when `baseUrl` does not answer `/models` within 2 s. Absent = provider default; `""` = no fallback. |
@@ -24,7 +24,7 @@
2424 | `fetchContentTool` | bool | `false` | Declare `fetch_content`, which fetches web pages as Markdown. Off by default: it is the only built-in tool that reaches the network. |
2525 | `sampling` | map | `temperature: 0.0`, `top_p: 0.9`, `max_tokens: 4096` | Generation settings, OpenAI API key names, passed to the server. The shipped `agent.yaml` adds `parallel_tool_calls: false`. `max_tokens` also bounds one tool call: a `write_file` of a long document (or a thinking model's reasoning plus the call) can hit it, and the call is then cut off. mm hands it back to the model twice, asking for smaller pieces, then stops with `the model hit max_tokens (N) while writing a tool call`; raise the value if that happens. With streaming, mm also asks the server for token counts (`stream_options.include_usage`). |
2626 | `watchdogTimeout` | duration | `20s` | Silence tolerated between two streamed tokens before the connection is declared hung. Doubled for summary requests. |
27-| `firstTokenTimeout` | duration | `3m` | Silence tolerated before a model call's first token, while the server reads the prompt (every round trip of a turn has one). Never shorter than `watchdogTimeout`; must be >= 0. With a llama.cpp server (`dmr`, `llamacpp`) mm asks for `return_progress`: each progress report restarts this clock, and the spinner shows `Reading the prompt 12.0k/25.3k tokens`. On a timeout the watchdog says `no first token from the server after …` (before the first token) or `connection hang detected after …` (after). |
27+| `firstTokenTimeout` | duration | `3m` | Silence tolerated before a model call's first token, while the server reads the prompt (every round trip of a turn has one). Never shorter than `watchdogTimeout`; must be >= 0. With a llama.cpp server (`dmr`, `llamacpp`, `llmman` — whose local GGUF models run in llama-server) mm asks for `return_progress`: each progress report restarts this clock, and the spinner shows `Reading the prompt 12.0k/25.3k tokens`. Ollama sends no progress: this one wait covers the whole prompt (and the model's loading). On a timeout the watchdog says `no first token from the server after …` (before the first token) or `connection hang detected after …` (after). |
2828 | `context` | map | see below | Automatic history compression. |
2929 | `sessions` | map | see below | Persistent, per-project sessions. |
3030 | `acp` | map | see below | How `mm -acp` introduces itself to the editor. |
@@ -40,7 +40,7 @@
4040 | `threshold` | int | `75` | Percentage of the context window beyond which the history is compressed before the next question. Between 1 and 100. |
4141 | `maxMessages` | int | `80` | Message count that triggers a compression when the window is unknown or the estimate is low. `0` disables. Must be ≥ 0. |
4242 | `keepLastTurns` | int | `3` | Question turns kept raw at the end of the history. Must be ≥ 1. |
43-| `summaryMaxTokens` | int | `1200` | `max_tokens` of the summary request. Must be > 0. A thinking model's reasoning counts against it: on `dmr` and `llamacpp` the summary request turns reasoning off (`chat_template_kwargs.enable_thinking: false`); elsewhere, if the error says the model "reasoned until the limit", raise it. |
43+| `summaryMaxTokens` | int | `1200` | `max_tokens` of the summary request. Must be > 0. A thinking model's reasoning counts against it: on `dmr`, `llamacpp` and `llmman` the summary request turns reasoning off (`chat_template_kwargs.enable_thinking: false`), on `ollama` too (`reasoning_effort: "none"`, the one switch Ollama obeys); elsewhere, if the error says the model "reasoned until the limit", raise it. |
4444 | `prompt` | string | `""` | Replaces the built-in summary prompt when not empty. |
4545 | `showStats` | bool | `true` | Print the one-line `🗜️` report after each compression. |
4646
@@ -90,6 +90,8 @@ The shipped `agent.lsp.yaml` is `agent.dmr.yaml` with this block on (gopls activ
9090 |----------|-----------|------------|-------------|--------------|
9191 | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | none | no |
9292 | `llamacpp` | `http://127.0.0.1:8080/v1` | none | `LLAMA_API_KEY` | no |
93+| `llmman` | `http://127.0.0.1:17434/v1` | `http://host.docker.internal:17434/v1` | `LLMMAN_API_KEY` | no |
94+| `ollama` | `http://localhost:11434/v1` | `http://host.docker.internal:11434/v1` | `OLLAMA_API_KEY` | no |
9395
9496 ## Validation errors
9597
@@ -115,7 +117,7 @@ Reported as `[config error: <path>: <message>]`, exit code 1.
115117 | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | an extension without its dot |
116118 | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | two servers on one extension |
117119 | `permissions must be "auto" or "ask"` | any other value |
118-| `unknown provider "x" (known: dmr, llamacpp)` | Reported by the engine, as `[engine error: …]`. |
120+| `unknown provider "x" (known: dmr, llamacpp, llmman, ollama)` | Reported by the engine, as `[engine error: …]`. |
119121
120122 A start-up warning, not an error, is printed when `context.enabled` is `true`, the window is unknown and `context.maxMessages` is `0`.
121123
@@ -6,7 +6,7 @@
6 6
7 | Key | Type | Default | Description |7 | Key | Type | Default | Description |
8 |-----|------|---------|-------------|8 |-----|------|---------|-------------|
9-| `provider` | string | `dmr` | Registry key of the LLM server: `dmr` or `llamacpp`. Overridden by `AGENT_PROVIDER`. |9+| `provider` | string | `dmr` | Registry key of the LLM server: `dmr`, `llamacpp`, `llmman` or `ollama`. Overridden by `AGENT_PROVIDER`. |
10 | `model` | string | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Model identifier in the provider's naming. Overridden by `AGENT_MODEL`. |10 | `model` | string | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Model identifier in the provider's naming. Overridden by `AGENT_MODEL`. |
11 | `baseUrl` | string | `""` (provider default) | OpenAI-compatible endpoint tried first. Overridden by `AGENT_BASE_URL`. |11 | `baseUrl` | string | `""` (provider default) | OpenAI-compatible endpoint tried first. Overridden by `AGENT_BASE_URL`. |
12 | `fallback` | string or absent | absent (provider default) | Endpoint used when `baseUrl` does not answer `/models` within 2 s. Absent = provider default; `""` = no fallback. |12 | `fallback` | string or absent | absent (provider default) | Endpoint used when `baseUrl` does not answer `/models` within 2 s. Absent = provider default; `""` = no fallback. |
@@ -24,7 +24,7 @@
24 | `fetchContentTool` | bool | `false` | Declare `fetch_content`, which fetches web pages as Markdown. Off by default: it is the only built-in tool that reaches the network. |24 | `fetchContentTool` | bool | `false` | Declare `fetch_content`, which fetches web pages as Markdown. Off by default: it is the only built-in tool that reaches the network. |
25 | `sampling` | map | `temperature: 0.0`, `top_p: 0.9`, `max_tokens: 4096` | Generation settings, OpenAI API key names, passed to the server. The shipped `agent.yaml` adds `parallel_tool_calls: false`. `max_tokens` also bounds one tool call: a `write_file` of a long document (or a thinking model's reasoning plus the call) can hit it, and the call is then cut off. mm hands it back to the model twice, asking for smaller pieces, then stops with `the model hit max_tokens (N) while writing a tool call`; raise the value if that happens. With streaming, mm also asks the server for token counts (`stream_options.include_usage`). |25 | `sampling` | map | `temperature: 0.0`, `top_p: 0.9`, `max_tokens: 4096` | Generation settings, OpenAI API key names, passed to the server. The shipped `agent.yaml` adds `parallel_tool_calls: false`. `max_tokens` also bounds one tool call: a `write_file` of a long document (or a thinking model's reasoning plus the call) can hit it, and the call is then cut off. mm hands it back to the model twice, asking for smaller pieces, then stops with `the model hit max_tokens (N) while writing a tool call`; raise the value if that happens. With streaming, mm also asks the server for token counts (`stream_options.include_usage`). |
26 | `watchdogTimeout` | duration | `20s` | Silence tolerated between two streamed tokens before the connection is declared hung. Doubled for summary requests. |26 | `watchdogTimeout` | duration | `20s` | Silence tolerated between two streamed tokens before the connection is declared hung. Doubled for summary requests. |
27-| `firstTokenTimeout` | duration | `3m` | Silence tolerated before a model call's first token, while the server reads the prompt (every round trip of a turn has one). Never shorter than `watchdogTimeout`; must be >= 0. With a llama.cpp server (`dmr`, `llamacpp`) mm asks for `return_progress`: each progress report restarts this clock, and the spinner shows `Reading the prompt 12.0k/25.3k tokens`. On a timeout the watchdog says `no first token from the server after …` (before the first token) or `connection hang detected after …` (after). |27+| `firstTokenTimeout` | duration | `3m` | Silence tolerated before a model call's first token, while the server reads the prompt (every round trip of a turn has one). Never shorter than `watchdogTimeout`; must be >= 0. With a llama.cpp server (`dmr`, `llamacpp`, `llmman` — whose local GGUF models run in llama-server) mm asks for `return_progress`: each progress report restarts this clock, and the spinner shows `Reading the prompt 12.0k/25.3k tokens`. Ollama sends no progress: this one wait covers the whole prompt (and the model's loading). On a timeout the watchdog says `no first token from the server after …` (before the first token) or `connection hang detected after …` (after). |
28 | `context` | map | see below | Automatic history compression. |28 | `context` | map | see below | Automatic history compression. |
29 | `sessions` | map | see below | Persistent, per-project sessions. |29 | `sessions` | map | see below | Persistent, per-project sessions. |
30 | `acp` | map | see below | How `mm -acp` introduces itself to the editor. |30 | `acp` | map | see below | How `mm -acp` introduces itself to the editor. |
@@ -40,7 +40,7 @@
40 | `threshold` | int | `75` | Percentage of the context window beyond which the history is compressed before the next question. Between 1 and 100. |40 | `threshold` | int | `75` | Percentage of the context window beyond which the history is compressed before the next question. Between 1 and 100. |
41 | `maxMessages` | int | `80` | Message count that triggers a compression when the window is unknown or the estimate is low. `0` disables. Must be ≥ 0. |41 | `maxMessages` | int | `80` | Message count that triggers a compression when the window is unknown or the estimate is low. `0` disables. Must be ≥ 0. |
42 | `keepLastTurns` | int | `3` | Question turns kept raw at the end of the history. Must be ≥ 1. |42 | `keepLastTurns` | int | `3` | Question turns kept raw at the end of the history. Must be ≥ 1. |
43-| `summaryMaxTokens` | int | `1200` | `max_tokens` of the summary request. Must be > 0. A thinking model's reasoning counts against it: on `dmr` and `llamacpp` the summary request turns reasoning off (`chat_template_kwargs.enable_thinking: false`); elsewhere, if the error says the model "reasoned until the limit", raise it. |43+| `summaryMaxTokens` | int | `1200` | `max_tokens` of the summary request. Must be > 0. A thinking model's reasoning counts against it: on `dmr`, `llamacpp` and `llmman` the summary request turns reasoning off (`chat_template_kwargs.enable_thinking: false`), on `ollama` too (`reasoning_effort: "none"`, the one switch Ollama obeys); elsewhere, if the error says the model "reasoned until the limit", raise it. |
44 | `prompt` | string | `""` | Replaces the built-in summary prompt when not empty. |44 | `prompt` | string | `""` | Replaces the built-in summary prompt when not empty. |
45 | `showStats` | bool | `true` | Print the one-line `🗜️` report after each compression. |45 | `showStats` | bool | `true` | Print the one-line `🗜️` report after each compression. |
46 46
@@ -90,6 +90,8 @@ The shipped `agent.lsp.yaml` is `agent.dmr.yaml` with this block on (gopls activ
90 |----------|-----------|------------|-------------|--------------|90 |----------|-----------|------------|-------------|--------------|
91 | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | none | no |91 | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | none | no |
92 | `llamacpp` | `http://127.0.0.1:8080/v1` | none | `LLAMA_API_KEY` | no |92 | `llamacpp` | `http://127.0.0.1:8080/v1` | none | `LLAMA_API_KEY` | no |
93+| `llmman` | `http://127.0.0.1:17434/v1` | `http://host.docker.internal:17434/v1` | `LLMMAN_API_KEY` | no |
94+| `ollama` | `http://localhost:11434/v1` | `http://host.docker.internal:11434/v1` | `OLLAMA_API_KEY` | no |
93 95
94 ## Validation errors96 ## Validation errors
95 97
@@ -115,7 +117,7 @@ Reported as `[config error: <path>: <message>]`, exit code 1.
115 | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | an extension without its dot |117 | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | an extension without its dot |
116 | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | two servers on one extension |118 | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | two servers on one extension |
117 | `permissions must be "auto" or "ask"` | any other value |119 | `permissions must be "auto" or "ask"` | any other value |
118-| `unknown provider "x" (known: dmr, llamacpp)` | Reported by the engine, as `[engine error: …]`. |120+| `unknown provider "x" (known: dmr, llamacpp, llmman, ollama)` | Reported by the engine, as `[engine error: …]`. |
119 121
120 A start-up warning, not an error, is printed when `context.enabled` is `true`, the window is unknown and `context.maxMessages` is `0`.122 A start-up warning, not an error, is printed when `context.enabled` is `true`, the window is unknown and `context.maxMessages` is `0`.
121 123
modified docs/en/reference/mcp-configuration.md +4 -2
@@ -6,7 +6,7 @@
66
77 | Key | Type | Default | Description |
88 |-----|------|---------|-------------|
9-| `provider` | string | `dmr` | Registry key of the LLM server: `dmr` or `llamacpp`. Overridden by `AGENT_PROVIDER`. |
9+| `provider` | string | `dmr` | Registry key of the LLM server: `dmr`, `llamacpp`, `llmman` or `ollama`. Overridden by `AGENT_PROVIDER`. |
1010 | `model` | string | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Model identifier in the provider's naming. Overridden by `AGENT_MODEL`. |
1111 | `baseUrl` | string | `""` (provider default) | OpenAI-compatible endpoint tried first. Overridden by `AGENT_BASE_URL`. |
1212 | `fallback` | string or absent | absent (provider default) | Endpoint used when `baseUrl` does not answer `/models` within 2 s. Absent = provider default; `""` = no fallback. |
@@ -78,6 +78,8 @@ The shipped `agent.lsp.yaml` is `agent.dmr.yaml` with this block on (gopls activ
7878 |----------|-----------|------------|-------------|--------------|
7979 | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | none | no |
8080 | `llamacpp` | `http://127.0.0.1:8080/v1` | none | `LLAMA_API_KEY` | no |
81+| `llmman` | `http://127.0.0.1:17434/v1` | `http://host.docker.internal:17434/v1` | `LLMMAN_API_KEY` | no |
82+| `ollama` | `http://localhost:11434/v1` | `http://host.docker.internal:11434/v1` | `OLLAMA_API_KEY` | no |
8183
8284 ## Validation errors
8385
@@ -102,7 +104,7 @@ Reported as `[config error: <path>: <message>]`, exit code 1.
102104 | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | an extension without its dot |
103105 | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | two servers on one extension |
104106 | `permissions must be "auto" or "ask"` | any other value |
105-| `unknown provider "x" (known: dmr, llamacpp)` | Reported by the engine, as `[engine error: …]`. |
107+| `unknown provider "x" (known: dmr, llamacpp, llmman, ollama)` | Reported by the engine, as `[engine error: …]`. |
106108
107109 A start-up warning, not an error, is printed when `context.enabled` is `true`, the window is unknown and `context.maxMessages` is `0`.
108110
@@ -6,7 +6,7 @@
6 6
7 | Key | Type | Default | Description |7 | Key | Type | Default | Description |
8 |-----|------|---------|-------------|8 |-----|------|---------|-------------|
9-| `provider` | string | `dmr` | Registry key of the LLM server: `dmr` or `llamacpp`. Overridden by `AGENT_PROVIDER`. |9+| `provider` | string | `dmr` | Registry key of the LLM server: `dmr`, `llamacpp`, `llmman` or `ollama`. Overridden by `AGENT_PROVIDER`. |
10 | `model` | string | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Model identifier in the provider's naming. Overridden by `AGENT_MODEL`. |10 | `model` | string | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Model identifier in the provider's naming. Overridden by `AGENT_MODEL`. |
11 | `baseUrl` | string | `""` (provider default) | OpenAI-compatible endpoint tried first. Overridden by `AGENT_BASE_URL`. |11 | `baseUrl` | string | `""` (provider default) | OpenAI-compatible endpoint tried first. Overridden by `AGENT_BASE_URL`. |
12 | `fallback` | string or absent | absent (provider default) | Endpoint used when `baseUrl` does not answer `/models` within 2 s. Absent = provider default; `""` = no fallback. |12 | `fallback` | string or absent | absent (provider default) | Endpoint used when `baseUrl` does not answer `/models` within 2 s. Absent = provider default; `""` = no fallback. |
@@ -78,6 +78,8 @@ The shipped `agent.lsp.yaml` is `agent.dmr.yaml` with this block on (gopls activ
78 |----------|-----------|------------|-------------|--------------|78 |----------|-----------|------------|-------------|--------------|
79 | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | none | no |79 | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | none | no |
80 | `llamacpp` | `http://127.0.0.1:8080/v1` | none | `LLAMA_API_KEY` | no |80 | `llamacpp` | `http://127.0.0.1:8080/v1` | none | `LLAMA_API_KEY` | no |
81+| `llmman` | `http://127.0.0.1:17434/v1` | `http://host.docker.internal:17434/v1` | `LLMMAN_API_KEY` | no |
82+| `ollama` | `http://localhost:11434/v1` | `http://host.docker.internal:11434/v1` | `OLLAMA_API_KEY` | no |
81 83
82 ## Validation errors84 ## Validation errors
83 85
@@ -102,7 +104,7 @@ Reported as `[config error: <path>: <message>]`, exit code 1.
102 | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | an extension without its dot |104 | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | an extension without its dot |
103 | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | two servers on one extension |105 | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | two servers on one extension |
104 | `permissions must be "auto" or "ask"` | any other value |106 | `permissions must be "auto" or "ask"` | any other value |
105-| `unknown provider "x" (known: dmr, llamacpp)` | Reported by the engine, as `[engine error: …]`. |107+| `unknown provider "x" (known: dmr, llamacpp, llmman, ollama)` | Reported by the engine, as `[engine error: …]`. |
106 108
107 A start-up warning, not an error, is printed when `context.enabled` is `true`, the window is unknown and `context.maxMessages` is `0`.109 A start-up warning, not an error, is printed when `context.enabled` is `true`, the window is unknown and `context.maxMessages` is `0`.
108 110
modified docs/fr/README.md +3 -1
@@ -1,6 +1,6 @@
11 # mini-me — documentation (français)
22
3-`mm` est un agent de code qui tourne dans votre terminal — ligne à ligne, ou dans une fenêtre unique avec `mm -tui` — ou dans un éditeur via l'Agent Client Protocol. Il dialogue avec un serveur de modèle local (Docker Model Runner ou `llama-server` de llama.cpp) et travaille avec une poignée d'outils intégrés : `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — et, quand ils sont activés, `lsp` et `fetch_content`.
3+`mm` est un agent de code qui tourne dans votre terminal — ligne à ligne, ou dans une fenêtre unique avec `mm -tui` — ou dans un éditeur via l'Agent Client Protocol. Il dialogue avec un serveur de modèle local (Docker Model Runner, `llama-server` de llama.cpp, llmman ou Ollama) et travaille avec une poignée d'outils intégrés : `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — et, quand ils sont activés, `lsp` et `fetch_content`.
44
55 Cette documentation est organisée selon les quatre quadrants [Diátaxis](https://diataxis.fr). Choisissez celui qui correspond à votre besoin du moment.
66
@@ -11,6 +11,8 @@ Cette documentation est organisée selon les quatre quadrants [Diátaxis](https:
1111 ## Guides pratiques — accomplir une tâche
1212
1313 - [Faire tourner mini-me sur llama.cpp plutôt que Docker Model Runner](how-to/run-with-llama-cpp.md)
14+- [Faire tourner mini-me sur llmman](how-to/run-with-llmman.md) — `provider: llmman`, la config de démo dans `demo-llmman/`
15+- [Faire tourner mini-me sur Ollama](how-to/run-with-ollama.md) — `provider: ollama`, la config de démo dans `demo-ollama/`
1416 - [Utiliser mini-me depuis un éditeur (ACP)](how-to/use-from-an-editor.md)
1517 - [Faire tourner mini-me dans un Docker Sandbox](how-to/run-in-a-docker-sandbox.md) — le template et les deux kits sous `docker-sbx/`, mode fenêtre et éditeur
1618 - [Faire tourner mini-me avec les kits sandbox v3](how-to/run-with-sandbox-kits-v3.md) — les deux mêmes kits pour la spécification de kit v3, sous `docker-sbx-v3/`
@@ -1,6 +1,6 @@
1 # mini-me — documentation (français)1 # mini-me — documentation (français)
2 2
3-`mm` est un agent de code qui tourne dans votre terminal — ligne à ligne, ou dans une fenêtre unique avec `mm -tui` — ou dans un éditeur via l'Agent Client Protocol. Il dialogue avec un serveur de modèle local (Docker Model Runner ou `llama-server` de llama.cpp) et travaille avec une poignée d'outils intégrés : `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — et, quand ils sont activés, `lsp` et `fetch_content`.3+`mm` est un agent de code qui tourne dans votre terminal — ligne à ligne, ou dans une fenêtre unique avec `mm -tui` — ou dans un éditeur via l'Agent Client Protocol. Il dialogue avec un serveur de modèle local (Docker Model Runner, `llama-server` de llama.cpp, llmman ou Ollama) et travaille avec une poignée d'outils intégrés : `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — et, quand ils sont activés, `lsp` et `fetch_content`.
4 4
5 Cette documentation est organisée selon les quatre quadrants [Diátaxis](https://diataxis.fr). Choisissez celui qui correspond à votre besoin du moment.5 Cette documentation est organisée selon les quatre quadrants [Diátaxis](https://diataxis.fr). Choisissez celui qui correspond à votre besoin du moment.
6 6
@@ -11,6 +11,8 @@ Cette documentation est organisée selon les quatre quadrants [Diátaxis](https:
11 ## Guides pratiques — accomplir une tâche11 ## Guides pratiques — accomplir une tâche
12 12
13 - [Faire tourner mini-me sur llama.cpp plutôt que Docker Model Runner](how-to/run-with-llama-cpp.md)13 - [Faire tourner mini-me sur llama.cpp plutôt que Docker Model Runner](how-to/run-with-llama-cpp.md)
14+- [Faire tourner mini-me sur llmman](how-to/run-with-llmman.md) — `provider: llmman`, la config de démo dans `demo-llmman/`
15+- [Faire tourner mini-me sur Ollama](how-to/run-with-ollama.md) — `provider: ollama`, la config de démo dans `demo-ollama/`
14 - [Utiliser mini-me depuis un éditeur (ACP)](how-to/use-from-an-editor.md)16 - [Utiliser mini-me depuis un éditeur (ACP)](how-to/use-from-an-editor.md)
15 - [Faire tourner mini-me dans un Docker Sandbox](how-to/run-in-a-docker-sandbox.md) — le template et les deux kits sous `docker-sbx/`, mode fenêtre et éditeur17 - [Faire tourner mini-me dans un Docker Sandbox](how-to/run-in-a-docker-sandbox.md) — le template et les deux kits sous `docker-sbx/`, mode fenêtre et éditeur
16 - [Faire tourner mini-me avec les kits sandbox v3](how-to/run-with-sandbox-kits-v3.md) — les deux mêmes kits pour la spécification de kit v3, sous `docker-sbx-v3/`18 - [Faire tourner mini-me avec les kits sandbox v3](how-to/run-with-sandbox-kits-v3.md) — les deux mêmes kits pour la spécification de kit v3, sous `docker-sbx-v3/`
modified docs/fr/explanation/providers.md +6 -2
@@ -2,7 +2,7 @@
22
33 ## De quoi s'agit-il ?
44
5-`mm` dialogue avec un serveur de modèle local via l'API chat-completions d'OpenAI. Deux serveurs sont pris en charge d'origine, Docker Model Runner et `llama-server` de llama.cpp, sous les noms `dmr` et `llamacpp`. Le paquet moteur était autrefois lié à Docker Model Runner ; l'abstraction de fournisseur est ce qui l'a généralisé.
5+`mm` dialogue avec un serveur de modèle local via l'API chat-completions d'OpenAI. Quatre serveurs sont pris en charge d'origine : Docker Model Runner, `llama-server` de llama.cpp, llmman et Ollama, sous les noms `dmr`, `llamacpp`, `llmman` et `ollama`. Le paquet moteur était autrefois lié à Docker Model Runner ; l'abstraction de fournisseur est ce qui l'a généralisé.
66
77 ## Pourquoi c'est conçu ainsi
88
@@ -16,6 +16,10 @@
1616
1717 **La sonde est un conseil, jamais un échec.** Sur une machine de démo, le serveur est souvent démarré après l'agent. La sonde vérifie la joignabilité, avertit quand le modèle configuré n'est pas dans la liste de DMR, et sur llama.cpp lit la taille de contexte servie sur `/props`, un niveau au-dessus de `/v1`, parce que ce point d'accès rapporte ce que le serveur sert, pas ce sur quoi le modèle a été entraîné. Quand la fenêtre est encore inconnue et que la compression est activée, le REPL sonde à nouveau avant la première question.
1818
19+**llmman est une troisième entrée du registre, avec sa propre sonde.** [llmman](https://github.com/llmmanorg/llmman) (`llmman serve`, port 17434) est un démon placé devant un `llama-server` par modèle chargé, et parle plusieurs API ; `mm` n'utilise que celle d'OpenAI. Tout ce qui est propre à llama.cpp la traverse — avancement de la lecture du prompt, raisonnement, `enable_thinking: false` pour les résumés — d'où le même traitement que `llamacpp`. Son `/props`, en revanche, est un bouchon (`n_ctx` 4096, pas de chat template, mesuré sur llmman 0.1.498) : la sonde lit donc la fenêtre servie sur `/api/ps`, pour le modèle configuré, une fois qu'il est chargé. Le contexte d'entraînement de `/api/show` n'est pas utilisé : `LLMMAN_CONTEXT_LENGTH` peut servir moins, et une fenêtre trop grande donne une compression qui ne se déclenche jamais avant que le serveur refuse le prompt. Un modèle absent de `/v1/models` n'est qu'un avertissement : llmman le télécharge à la première requête.
20+
21+**Ollama est une quatrième entrée, et montre pourquoi chaque serveur garde la sienne.** Le `/v1` d'Ollama ressemble aux autres, mais, mesuré sur Ollama 0.35.0, il diffère sur trois points. Le raisonnement arrive dans `delta.reasoning`, que le tap lit déjà. Aucun avancement de lecture du prompt ne revient, donc `return_progress` n'est pas envoyé. Et `chat_template_kwargs` est ignoré : le résumé d'une compaction a raisonné jusqu'à sa limite de tokens et est revenu vide ; c'est `reasoning_effort: "none"` qu'Ollama respecte, et avec lui la même compaction a réussi. Voilà pourquoi la façon de couper le raisonnement est un champ par fournisseur, et non une conséquence de « est llama.cpp ». Comme llmman, Ollama rapporte la fenêtre servie sur `/api/ps` une fois le modèle chargé (les deux partagent cette lecture) ; contrairement à llmman, il ne télécharge pas un modèle absent, d'où l'avertissement de la sonde qui dit `ollama pull`.
22+
1923 **Les erreurs sont réécrites dans les mots du serveur.** Une pile de messages imbriqués est du bruit ; « tool calls need llama-server started with --jinja » est tout le diagnostic. Le fournisseur reconnaît les refus de connexion, l'option `--jinja` manquante, les 401/403, les 404 avec une indication de pull ou d'alias, les 429 et les 5xx. Les erreurs inconnues reviennent inchangées : un message brut vaut mieux qu'une mauvaise piste.
2024
2125 **Un flux que le serveur interrompt est relancé, et ses mots sont gardés.** llama.cpp signale un échec au milieu d'une réponse diffusée par un événement SSE `error:` (le statut 200 est déjà parti). openai-go ne lit que les lignes `data:` : le message disparaissait derrière `unexpected end of JSON input`. Le tap qui lit déjà le flux pour le raisonnement récupère aussi ces événements, et l'appel au modèle qui a échoué est relancé une fois — cet appel seulement : les outils déjà exécutés dans le tour ne sont pas rejoués, et le texte déjà diffusé par la tentative interrompue n'est pas retiré. S'il échoue encore, l'erreur donne le message du serveur (`the model server reported an error while generating: Failed to parse input at pos …`), ou indique que la connexion a été coupée. Dans les deux cas l'historique est conservé : demander de continuer reprend le travail.
@@ -34,5 +38,5 @@
3438
3539 - Les deux fichiers de configuration qui paramètrent chaque fournisseur sont listés dans la [référence de la configuration](../reference/configuration.md).
3640 - La fenêtre apprise par la sonde est ce contre quoi la [compression du contexte](context-compression.md) mesure.
37-- Changer de serveur pour une exécution est couvert dans [comment faire tourner sur llama.cpp](../how-to/run-with-llama-cpp.md).
41+- Changer de serveur pour une exécution est couvert dans [comment faire tourner sur llama.cpp](../how-to/run-with-llama-cpp.md) [comment faire tourner sur llmman](../how-to/run-with-llmman.md) et [comment faire tourner sur Ollama](../how-to/run-with-ollama.md).
3842 - Vérifier l'ordre des outils et mesurer le gain de cache sur votre propre serveur : [vérifier que les outils partent dans un ordre stable](../how-to/verify-tool-order.md).
@@ -2,7 +2,7 @@
2 2
3 ## De quoi s'agit-il ?3 ## De quoi s'agit-il ?
4 4
5-`mm` dialogue avec un serveur de modèle local via l'API chat-completions d'OpenAI. Deux serveurs sont pris en charge d'origine, Docker Model Runner et `llama-server` de llama.cpp, sous les noms `dmr` et `llamacpp`. Le paquet moteur était autrefois lié à Docker Model Runner ; l'abstraction de fournisseur est ce qui l'a généralisé.5+`mm` dialogue avec un serveur de modèle local via l'API chat-completions d'OpenAI. Quatre serveurs sont pris en charge d'origine : Docker Model Runner, `llama-server` de llama.cpp, llmman et Ollama, sous les noms `dmr`, `llamacpp`, `llmman` et `ollama`. Le paquet moteur était autrefois lié à Docker Model Runner ; l'abstraction de fournisseur est ce qui l'a généralisé.
6 6
7 ## Pourquoi c'est conçu ainsi7 ## Pourquoi c'est conçu ainsi
8 8
@@ -16,6 +16,10 @@
16 16
17 **La sonde est un conseil, jamais un échec.** Sur une machine de démo, le serveur est souvent démarré après l'agent. La sonde vérifie la joignabilité, avertit quand le modèle configuré n'est pas dans la liste de DMR, et sur llama.cpp lit la taille de contexte servie sur `/props`, un niveau au-dessus de `/v1`, parce que ce point d'accès rapporte ce que le serveur sert, pas ce sur quoi le modèle a été entraîné. Quand la fenêtre est encore inconnue et que la compression est activée, le REPL sonde à nouveau avant la première question.17 **La sonde est un conseil, jamais un échec.** Sur une machine de démo, le serveur est souvent démarré après l'agent. La sonde vérifie la joignabilité, avertit quand le modèle configuré n'est pas dans la liste de DMR, et sur llama.cpp lit la taille de contexte servie sur `/props`, un niveau au-dessus de `/v1`, parce que ce point d'accès rapporte ce que le serveur sert, pas ce sur quoi le modèle a été entraîné. Quand la fenêtre est encore inconnue et que la compression est activée, le REPL sonde à nouveau avant la première question.
18 18
19+**llmman est une troisième entrée du registre, avec sa propre sonde.** [llmman](https://github.com/llmmanorg/llmman) (`llmman serve`, port 17434) est un démon placé devant un `llama-server` par modèle chargé, et parle plusieurs API ; `mm` n'utilise que celle d'OpenAI. Tout ce qui est propre à llama.cpp la traverse — avancement de la lecture du prompt, raisonnement, `enable_thinking: false` pour les résumés — d'où le même traitement que `llamacpp`. Son `/props`, en revanche, est un bouchon (`n_ctx` 4096, pas de chat template, mesuré sur llmman 0.1.498) : la sonde lit donc la fenêtre servie sur `/api/ps`, pour le modèle configuré, une fois qu'il est chargé. Le contexte d'entraînement de `/api/show` n'est pas utilisé : `LLMMAN_CONTEXT_LENGTH` peut servir moins, et une fenêtre trop grande donne une compression qui ne se déclenche jamais avant que le serveur refuse le prompt. Un modèle absent de `/v1/models` n'est qu'un avertissement : llmman le télécharge à la première requête.
20+
21+**Ollama est une quatrième entrée, et montre pourquoi chaque serveur garde la sienne.** Le `/v1` d'Ollama ressemble aux autres, mais, mesuré sur Ollama 0.35.0, il diffère sur trois points. Le raisonnement arrive dans `delta.reasoning`, que le tap lit déjà. Aucun avancement de lecture du prompt ne revient, donc `return_progress` n'est pas envoyé. Et `chat_template_kwargs` est ignoré : le résumé d'une compaction a raisonné jusqu'à sa limite de tokens et est revenu vide ; c'est `reasoning_effort: "none"` qu'Ollama respecte, et avec lui la même compaction a réussi. Voilà pourquoi la façon de couper le raisonnement est un champ par fournisseur, et non une conséquence de « est llama.cpp ». Comme llmman, Ollama rapporte la fenêtre servie sur `/api/ps` une fois le modèle chargé (les deux partagent cette lecture) ; contrairement à llmman, il ne télécharge pas un modèle absent, d'où l'avertissement de la sonde qui dit `ollama pull`.
22+
19 **Les erreurs sont réécrites dans les mots du serveur.** Une pile de messages imbriqués est du bruit ; « tool calls need llama-server started with --jinja » est tout le diagnostic. Le fournisseur reconnaît les refus de connexion, l'option `--jinja` manquante, les 401/403, les 404 avec une indication de pull ou d'alias, les 429 et les 5xx. Les erreurs inconnues reviennent inchangées : un message brut vaut mieux qu'une mauvaise piste.23 **Les erreurs sont réécrites dans les mots du serveur.** Une pile de messages imbriqués est du bruit ; « tool calls need llama-server started with --jinja » est tout le diagnostic. Le fournisseur reconnaît les refus de connexion, l'option `--jinja` manquante, les 401/403, les 404 avec une indication de pull ou d'alias, les 429 et les 5xx. Les erreurs inconnues reviennent inchangées : un message brut vaut mieux qu'une mauvaise piste.
20 24
21 **Un flux que le serveur interrompt est relancé, et ses mots sont gardés.** llama.cpp signale un échec au milieu d'une réponse diffusée par un événement SSE `error:` (le statut 200 est déjà parti). openai-go ne lit que les lignes `data:` : le message disparaissait derrière `unexpected end of JSON input`. Le tap qui lit déjà le flux pour le raisonnement récupère aussi ces événements, et l'appel au modèle qui a échoué est relancé une fois — cet appel seulement : les outils déjà exécutés dans le tour ne sont pas rejoués, et le texte déjà diffusé par la tentative interrompue n'est pas retiré. S'il échoue encore, l'erreur donne le message du serveur (`the model server reported an error while generating: Failed to parse input at pos …`), ou indique que la connexion a été coupée. Dans les deux cas l'historique est conservé : demander de continuer reprend le travail.25 **Un flux que le serveur interrompt est relancé, et ses mots sont gardés.** llama.cpp signale un échec au milieu d'une réponse diffusée par un événement SSE `error:` (le statut 200 est déjà parti). openai-go ne lit que les lignes `data:` : le message disparaissait derrière `unexpected end of JSON input`. Le tap qui lit déjà le flux pour le raisonnement récupère aussi ces événements, et l'appel au modèle qui a échoué est relancé une fois — cet appel seulement : les outils déjà exécutés dans le tour ne sont pas rejoués, et le texte déjà diffusé par la tentative interrompue n'est pas retiré. S'il échoue encore, l'erreur donne le message du serveur (`the model server reported an error while generating: Failed to parse input at pos …`), ou indique que la connexion a été coupée. Dans les deux cas l'historique est conservé : demander de continuer reprend le travail.
@@ -34,5 +38,5 @@
34 38
35 - Les deux fichiers de configuration qui paramètrent chaque fournisseur sont listés dans la [référence de la configuration](../reference/configuration.md).39 - Les deux fichiers de configuration qui paramètrent chaque fournisseur sont listés dans la [référence de la configuration](../reference/configuration.md).
36 - La fenêtre apprise par la sonde est ce contre quoi la [compression du contexte](context-compression.md) mesure.40 - La fenêtre apprise par la sonde est ce contre quoi la [compression du contexte](context-compression.md) mesure.
37-- Changer de serveur pour une exécution est couvert dans [comment faire tourner sur llama.cpp](../how-to/run-with-llama-cpp.md).41+- Changer de serveur pour une exécution est couvert dans [comment faire tourner sur llama.cpp](../how-to/run-with-llama-cpp.md) [comment faire tourner sur llmman](../how-to/run-with-llmman.md) et [comment faire tourner sur Ollama](../how-to/run-with-ollama.md).
38 - Vérifier l'ordre des outils et mesurer le gain de cache sur votre propre serveur : [vérifier que les outils partent dans un ordre stable](../how-to/verify-tool-order.md).42 - Vérifier l'ordre des outils et mesurer le gain de cache sur votre propre serveur : [vérifier que les outils partent dans un ordre stable](../how-to/verify-tool-order.md).
added docs/fr/how-to/run-with-llmman.md +39 -0
new file mode 100644
@@ -0,0 +1,39 @@
1+# Comment faire tourner mini-me sur llmman
2+
3+Ce guide montre comment brancher `mm` sur [llmman](https://github.com/llmmanorg/llmman) via son point d'accès compatible OpenAI. Il suppose que llmman est installé et que vous savez compiler `mm`.
4+
5+## Étapes
6+
7+1. Démarrez le démon. Il écoute sur `127.0.0.1:17434` :
8+
9+ ```bash
10+ llmman serve
11+ ```
12+
13+2. Téléchargez le modèle au préalable si vous le souhaitez. Sinon llmman le télécharge à la première question, qui attend la fin du téléchargement :
14+
15+ ```bash
16+ llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
17+ ```
18+
19+3. Lancez la configuration de démo depuis son dossier (son `skillsDir` vaut `../skills`) :
20+
21+ ```bash
22+ cd demo-llmman
23+ ../mm agent.llmman.yaml
24+ ```
25+
26+4. Vérifiez la bannière. `provider: llmman` confirme le fournisseur. `ctx: 262144 (/api/ps)` signifie que le modèle était déjà chargé et que `mm` a lu la fenêtre avec laquelle il est servi ; `ctx: unknown` signifie qu'il ne l'était pas encore — `mm` redemande avant la question suivante et affiche `ctx: … (/api/ps)` dès qu'il la connaît.
27+
28+## Variantes
29+
30+- **Un autre modèle.** Mettez dans `model:` l'identifiant exactement tel que `llmman list` (ou `GET /v1/models`) l'écrit.
31+- **Depuis un Docker Sandbox.** Rien à changer : quand `127.0.0.1:17434` ne répond pas, `mm` se replie sur `http://host.docker.internal:17434/v1`, le démon de l'hôte.
32+- **Démon lancé avec `LLMMAN_API_KEYS`** (obligatoire quand il écoute au-delà de la boucle locale). Exportez une des clés dans `LLMMAN_API_KEY`, la variable par défaut du fournisseur, ou nommez une autre variable dans `apiKeyEnv`.
33+- **`LLMMAN_CONTEXT_LENGTH` réglé sur le démon.** `/api/ps` la rapporte déjà une fois le modèle chargé ; mettez la même valeur dans `contextWindow` si vous voulez que la compression la connaisse dès la première question.
34+- **Garder votre propre `agent.yaml`.** `AGENT_PROVIDER=llmman ./mm` fonctionne tant que le fichier laisse `baseUrl` vide.
35+
36+## Voir aussi
37+
38+- [Fournisseurs : une implémentation par protocole](../explanation/providers.md) — pourquoi la fenêtre de llmman vient de `/api/ps` et non de `/props`.
39+- [Référence de configuration](../reference/configuration.md) — valeurs par défaut des fournisseurs.
new file mode 100644
@@ -0,0 +1,39 @@
1+# Comment faire tourner mini-me sur llmman
2+
3+Ce guide montre comment brancher `mm` sur [llmman](https://github.com/llmmanorg/llmman) via son point d'accès compatible OpenAI. Il suppose que llmman est installé et que vous savez compiler `mm`.
4+
5+## Étapes
6+
7+1. Démarrez le démon. Il écoute sur `127.0.0.1:17434` :
8+
9+ ```bash
10+ llmman serve
11+ ```
12+
13+2. Téléchargez le modèle au préalable si vous le souhaitez. Sinon llmman le télécharge à la première question, qui attend la fin du téléchargement :
14+
15+ ```bash
16+ llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
17+ ```
18+
19+3. Lancez la configuration de démo depuis son dossier (son `skillsDir` vaut `../skills`) :
20+
21+ ```bash
22+ cd demo-llmman
23+ ../mm agent.llmman.yaml
24+ ```
25+
26+4. Vérifiez la bannière. `provider: llmman` confirme le fournisseur. `ctx: 262144 (/api/ps)` signifie que le modèle était déjà chargé et que `mm` a lu la fenêtre avec laquelle il est servi ; `ctx: unknown` signifie qu'il ne l'était pas encore — `mm` redemande avant la question suivante et affiche `ctx: … (/api/ps)` dès qu'il la connaît.
27+
28+## Variantes
29+
30+- **Un autre modèle.** Mettez dans `model:` l'identifiant exactement tel que `llmman list` (ou `GET /v1/models`) l'écrit.
31+- **Depuis un Docker Sandbox.** Rien à changer : quand `127.0.0.1:17434` ne répond pas, `mm` se replie sur `http://host.docker.internal:17434/v1`, le démon de l'hôte.
32+- **Démon lancé avec `LLMMAN_API_KEYS`** (obligatoire quand il écoute au-delà de la boucle locale). Exportez une des clés dans `LLMMAN_API_KEY`, la variable par défaut du fournisseur, ou nommez une autre variable dans `apiKeyEnv`.
33+- **`LLMMAN_CONTEXT_LENGTH` réglé sur le démon.** `/api/ps` la rapporte déjà une fois le modèle chargé ; mettez la même valeur dans `contextWindow` si vous voulez que la compression la connaisse dès la première question.
34+- **Garder votre propre `agent.yaml`.** `AGENT_PROVIDER=llmman ./mm` fonctionne tant que le fichier laisse `baseUrl` vide.
35+
36+## Voir aussi
37+
38+- [Fournisseurs : une implémentation par protocole](../explanation/providers.md) — pourquoi la fenêtre de llmman vient de `/api/ps` et non de `/props`.
39+- [Référence de configuration](../reference/configuration.md) — valeurs par défaut des fournisseurs.
added docs/fr/how-to/run-with-ollama.md +40 -0
new file mode 100644
@@ -0,0 +1,40 @@
1+# Comment faire tourner mini-me sur Ollama
2+
3+Ce guide montre comment brancher `mm` sur [Ollama](https://ollama.com) via son point d'accès compatible OpenAI. Il suppose qu'Ollama est installé et que vous savez compiler `mm`.
4+
5+## Étapes
6+
7+1. Téléchargez le modèle. L'API OpenAI d'Ollama ne télécharge pas un modèle absent : une question échoue alors avec `model '…' not found` :
8+
9+ ```bash
10+ ollama pull qwen3.5:2b
11+ ```
12+
13+2. Vérifiez que le serveur tourne sur `localhost:11434` (l'application de bureau le démarre ; sinon `ollama serve`).
14+
15+3. Lancez la configuration de démo depuis son dossier (son `skillsDir` vaut `../skills`) :
16+
17+ ```bash
18+ cd demo-ollama
19+ ../mm agent.ollama.yaml
20+ ```
21+
22+4. Vérifiez la bannière. `provider: ollama` confirme le fournisseur. `ctx: 32768 (/api/ps)` signifie que le modèle était déjà chargé et que `mm` a lu la fenêtre avec laquelle il est servi ; `ctx: unknown` signifie qu'il ne l'était pas encore — `mm` redemande avant la question suivante.
23+
24+## Variantes
25+
26+- **Un autre modèle.** Mettez dans `model:` le nom exactement tel que `ollama list` l'écrit (`qwen3.5:9b`, …).
27+- **Depuis un Docker Sandbox.** Rien à changer : quand `localhost:11434` ne répond pas, `mm` se replie sur `http://host.docker.internal:11434/v1`, l'Ollama de l'hôte.
28+- **Une fenêtre plus grande.** L'API OpenAI ne peut pas régler `num_ctx` par requête : démarrez Ollama avec `OLLAMA_CONTEXT_LENGTH`, et mettez la même valeur dans `contextWindow` si vous voulez que la compression la connaisse dès la première question.
29+- **L'API d'ollama.com.** Mettez son point d'accès OpenAI dans `baseUrl` et exportez la clé dans `OLLAMA_API_KEY` (ou nommez une autre variable dans `apiKeyEnv`).
30+- **Garder votre propre `agent.yaml`.** `AGENT_PROVIDER=ollama ./mm` fonctionne tant que le fichier laisse `baseUrl` vide.
31+
32+## Ce qui diffère des serveurs llama.cpp
33+
34+- Pas de ligne `Reading the prompt …` : Ollama n'envoie pas d'avancement pendant la lecture, donc `firstTokenTimeout` doit couvrir tout le prompt et le chargement du modèle.
35+- Le résumé d'une compaction coupe le raisonnement avec `reasoning_effort: "none"` — Ollama ignore `chat_template_kwargs`.
36+
37+## Voir aussi
38+
39+- [Fournisseurs : une implémentation par protocole](../explanation/providers.md) — ce qui a été mesuré sur Ollama, et pourquoi chaque serveur garde sa propre entrée.
40+- [Référence de configuration](../reference/configuration.md) — valeurs par défaut des fournisseurs.
new file mode 100644
@@ -0,0 +1,40 @@
1+# Comment faire tourner mini-me sur Ollama
2+
3+Ce guide montre comment brancher `mm` sur [Ollama](https://ollama.com) via son point d'accès compatible OpenAI. Il suppose qu'Ollama est installé et que vous savez compiler `mm`.
4+
5+## Étapes
6+
7+1. Téléchargez le modèle. L'API OpenAI d'Ollama ne télécharge pas un modèle absent : une question échoue alors avec `model '…' not found` :
8+
9+ ```bash
10+ ollama pull qwen3.5:2b
11+ ```
12+
13+2. Vérifiez que le serveur tourne sur `localhost:11434` (l'application de bureau le démarre ; sinon `ollama serve`).
14+
15+3. Lancez la configuration de démo depuis son dossier (son `skillsDir` vaut `../skills`) :
16+
17+ ```bash
18+ cd demo-ollama
19+ ../mm agent.ollama.yaml
20+ ```
21+
22+4. Vérifiez la bannière. `provider: ollama` confirme le fournisseur. `ctx: 32768 (/api/ps)` signifie que le modèle était déjà chargé et que `mm` a lu la fenêtre avec laquelle il est servi ; `ctx: unknown` signifie qu'il ne l'était pas encore — `mm` redemande avant la question suivante.
23+
24+## Variantes
25+
26+- **Un autre modèle.** Mettez dans `model:` le nom exactement tel que `ollama list` l'écrit (`qwen3.5:9b`, …).
27+- **Depuis un Docker Sandbox.** Rien à changer : quand `localhost:11434` ne répond pas, `mm` se replie sur `http://host.docker.internal:11434/v1`, l'Ollama de l'hôte.
28+- **Une fenêtre plus grande.** L'API OpenAI ne peut pas régler `num_ctx` par requête : démarrez Ollama avec `OLLAMA_CONTEXT_LENGTH`, et mettez la même valeur dans `contextWindow` si vous voulez que la compression la connaisse dès la première question.
29+- **L'API d'ollama.com.** Mettez son point d'accès OpenAI dans `baseUrl` et exportez la clé dans `OLLAMA_API_KEY` (ou nommez une autre variable dans `apiKeyEnv`).
30+- **Garder votre propre `agent.yaml`.** `AGENT_PROVIDER=ollama ./mm` fonctionne tant que le fichier laisse `baseUrl` vide.
31+
32+## Ce qui diffère des serveurs llama.cpp
33+
34+- Pas de ligne `Reading the prompt …` : Ollama n'envoie pas d'avancement pendant la lecture, donc `firstTokenTimeout` doit couvrir tout le prompt et le chargement du modèle.
35+- Le résumé d'une compaction coupe le raisonnement avec `reasoning_effort: "none"` — Ollama ignore `chat_template_kwargs`.
36+
37+## Voir aussi
38+
39+- [Fournisseurs : une implémentation par protocole](../explanation/providers.md) — ce qui a été mesuré sur Ollama, et pourquoi chaque serveur garde sa propre entrée.
40+- [Référence de configuration](../reference/configuration.md) — valeurs par défaut des fournisseurs.
modified docs/fr/reference/cli.md +1 -1
@@ -55,7 +55,7 @@ La première source existante gagne.
5555 | `AGENT_MODEL` | Surcharge la clé `model`. |
5656 | `AGENT_BASE_URL` | Surcharge `baseUrl` et désactive la sonde de repli. |
5757 | `DMR_BASE_URL` | Comme `AGENT_BASE_URL`, pour le fournisseur `dmr` seulement ; `AGENT_BASE_URL` gagne quand les deux sont définies. |
58-| Variable nommée par `apiKeyEnv` | Contient la clé d'API. Nom par défaut : aucun pour `dmr`, `LLAMA_API_KEY` pour `llamacpp`. |
58+| Variable nommée par `apiKeyEnv` | Contient la clé d'API. Nom par défaut : aucun pour `dmr`, `LLAMA_API_KEY` pour `llamacpp`, `LLMMAN_API_KEY` pour `llmman`, `OLLAMA_API_KEY` pour `ollama`. |
5959
6060 ## Sortie de démarrage
6161
@@ -55,7 +55,7 @@ La première source existante gagne.
55 | `AGENT_MODEL` | Surcharge la clé `model`. |55 | `AGENT_MODEL` | Surcharge la clé `model`. |
56 | `AGENT_BASE_URL` | Surcharge `baseUrl` et désactive la sonde de repli. |56 | `AGENT_BASE_URL` | Surcharge `baseUrl` et désactive la sonde de repli. |
57 | `DMR_BASE_URL` | Comme `AGENT_BASE_URL`, pour le fournisseur `dmr` seulement ; `AGENT_BASE_URL` gagne quand les deux sont définies. |57 | `DMR_BASE_URL` | Comme `AGENT_BASE_URL`, pour le fournisseur `dmr` seulement ; `AGENT_BASE_URL` gagne quand les deux sont définies. |
58-| Variable nommée par `apiKeyEnv` | Contient la clé d'API. Nom par défaut : aucun pour `dmr`, `LLAMA_API_KEY` pour `llamacpp`. |58+| Variable nommée par `apiKeyEnv` | Contient la clé d'API. Nom par défaut : aucun pour `dmr`, `LLAMA_API_KEY` pour `llamacpp`, `LLMMAN_API_KEY` pour `llmman`, `OLLAMA_API_KEY` pour `ollama`. |
59 59
60 ## Sortie de démarrage60 ## Sortie de démarrage
61 61
modified docs/fr/reference/configuration.md +6 -4
@@ -6,7 +6,7 @@
66
77 | Clé | Type | Défaut | Description |
88 |-----|------|--------|-------------|
9-| `provider` | chaîne | `dmr` | Clé du registre du serveur LLM : `dmr` ou `llamacpp`. Surchargée par `AGENT_PROVIDER`. |
9+| `provider` | chaîne | `dmr` | Clé du registre du serveur LLM : `dmr`, `llamacpp`, `llmman` ou `ollama`. Surchargée par `AGENT_PROVIDER`. |
1010 | `model` | chaîne | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Identifiant du modèle dans la nomenclature du fournisseur. Surchargé par `AGENT_MODEL`. |
1111 | `baseUrl` | chaîne | `""` (défaut du fournisseur) | Point d'accès compatible OpenAI essayé en premier. Surchargé par `AGENT_BASE_URL`. |
1212 | `fallback` | chaîne ou absente | absente (défaut du fournisseur) | Point d'accès utilisé quand `baseUrl` ne répond pas sur `/models` en 2 s. Absente = défaut du fournisseur ; `""` = pas de repli. |
@@ -24,7 +24,7 @@
2424 | `fetchContentTool` | booléen | `false` | Déclare `fetch_content`, qui récupère des pages web en Markdown. Désactivé par défaut : c'est le seul outil intégré qui accède au réseau. |
2525 | `sampling` | dictionnaire | `temperature: 0.0`, `top_p: 0.9`, `max_tokens: 4096` | Paramètres de génération, noms de l'API OpenAI, transmis au serveur. L'`agent.yaml` livré ajoute `parallel_tool_calls: false`. `max_tokens` borne aussi un appel d'outil : un `write_file` de long document (ou le raisonnement d'un modèle « thinking » plus l'appel) peut l'atteindre, et l'appel est alors coupé. mm le rend deux fois au modèle en lui demandant de procéder par morceaux, puis s'arrête avec `the model hit max_tokens (N) while writing a tool call` ; augmentez la valeur si cela arrive. En streaming, mm demande aussi au serveur le compte des tokens (`stream_options.include_usage`). |
2626 | `watchdogTimeout` | durée | `20s` | Silence toléré entre deux tokens diffusés avant de déclarer la connexion bloquée. Doublé pour les requêtes de résumé. |
27-| `firstTokenTimeout` | durée | `3m` | Silence toléré avant le premier token d'un appel au modèle, pendant que le serveur lit le prompt (chaque aller-retour d'un tour en a un). Jamais plus court que `watchdogTimeout` ; doit être >= 0. Avec un serveur llama.cpp (`dmr`, `llamacpp`), mm demande `return_progress` : chaque rapport d'avancement relance ce délai, et le spinner affiche `Reading the prompt 12.0k/25.3k tokens`. À l'expiration, le watchdog dit `no first token from the server after …` (avant le premier token) ou `connection hang detected after …` (après). |
27+| `firstTokenTimeout` | durée | `3m` | Silence toléré avant le premier token d'un appel au modèle, pendant que le serveur lit le prompt (chaque aller-retour d'un tour en a un). Jamais plus court que `watchdogTimeout` ; doit être >= 0. Avec un serveur llama.cpp (`dmr`, `llamacpp`, `llmman` — dont les modèles GGUF locaux tournent dans llama-server), mm demande `return_progress` : chaque rapport d'avancement relance ce délai, et le spinner affiche `Reading the prompt 12.0k/25.3k tokens`. Ollama n'envoie pas d'avancement : cette seule attente couvre tout le prompt (et le chargement du modèle). À l'expiration, le watchdog dit `no first token from the server after …` (avant le premier token) ou `connection hang detected after …` (après). |
2828 | `context` | dictionnaire | voir ci-dessous | Compression automatique de l'historique. |
2929 | `sessions` | dictionnaire | voir ci-dessous | Sessions persistantes, par projet. |
3030 | `acp` | dictionnaire | voir ci-dessous | La façon dont `mm -acp` se présente à l'éditeur. |
@@ -40,7 +40,7 @@
4040 | `threshold` | entier | `75` | Pourcentage de la fenêtre de contexte au-delà duquel l'historique est compressé avant la question suivante. Entre 1 et 100. |
4141 | `maxMessages` | entier | `80` | Nombre de messages qui déclenche une compression quand la fenêtre est inconnue ou l'estimation trop basse. `0` désactive. Doit être ≥ 0. |
4242 | `keepLastTurns` | entier | `3` | Tours de question conservés bruts en fin d'historique. Doit être ≥ 1. |
43-| `summaryMaxTokens` | entier | `1200` | `max_tokens` de la requête de résumé. Doit être > 0. Le raisonnement d'un modèle qui réfléchit est décompté de ce budget : sur `dmr` et `llamacpp`, la requête de résumé désactive le raisonnement (`chat_template_kwargs.enable_thinking: false`) ; ailleurs, si l'erreur dit que le modèle « reasoned until the limit », augmentez-le. |
43+| `summaryMaxTokens` | entier | `1200` | `max_tokens` de la requête de résumé. Doit être > 0. Le raisonnement d'un modèle qui réfléchit est décompté de ce budget : sur `dmr`, `llamacpp` et `llmman`, la requête de résumé désactive le raisonnement (`chat_template_kwargs.enable_thinking: false`), sur `ollama` aussi (`reasoning_effort: "none"`, le seul interrupteur qu'Ollama respecte) ; ailleurs, si l'erreur dit que le modèle « reasoned until the limit », augmentez-le. |
4444 | `prompt` | chaîne | `""` | Remplace le prompt de résumé intégré quand non vide. |
4545 | `showStats` | booléen | `true` | Affiche le rapport `🗜️` d'une ligne après chaque compression. |
4646
@@ -90,6 +90,8 @@ Le fichier livré `agent.lsp.yaml` est `agent.dmr.yaml` avec ce bloc actif (gopl
9090 |-------------|-----------|------------|-------------|-------------|
9191 | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | aucune | non |
9292 | `llamacpp` | `http://127.0.0.1:8080/v1` | aucun | `LLAMA_API_KEY` | non |
93+| `llmman` | `http://127.0.0.1:17434/v1` | `http://host.docker.internal:17434/v1` | `LLMMAN_API_KEY` | non |
94+| `ollama` | `http://localhost:11434/v1` | `http://host.docker.internal:11434/v1` | `OLLAMA_API_KEY` | non |
9395
9496 ## Erreurs de validation
9597
@@ -115,7 +117,7 @@ Signalées sous la forme `[config error: <chemin>: <message>]`, code de sortie 1
115117 | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | une extension sans son point |
116118 | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | deux serveurs sur une même extension |
117119 | `permissions must be "auto" or "ask"` | toute autre valeur |
118-| `unknown provider "x" (known: dmr, llamacpp)` | Signalée par le moteur, sous la forme `[engine error: …]`. |
120+| `unknown provider "x" (known: dmr, llamacpp, llmman, ollama)` | Signalée par le moteur, sous la forme `[engine error: …]`. |
119121
120122 Un avertissement au démarrage, et non une erreur, est affiché quand `context.enabled` vaut `true`, que la fenêtre est inconnue et que `context.maxMessages` vaut `0`.
121123
@@ -6,7 +6,7 @@
6 6
7 | Clé | Type | Défaut | Description |7 | Clé | Type | Défaut | Description |
8 |-----|------|--------|-------------|8 |-----|------|--------|-------------|
9-| `provider` | chaîne | `dmr` | Clé du registre du serveur LLM : `dmr` ou `llamacpp`. Surchargée par `AGENT_PROVIDER`. |9+| `provider` | chaîne | `dmr` | Clé du registre du serveur LLM : `dmr`, `llamacpp`, `llmman` ou `ollama`. Surchargée par `AGENT_PROVIDER`. |
10 | `model` | chaîne | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Identifiant du modèle dans la nomenclature du fournisseur. Surchargé par `AGENT_MODEL`. |10 | `model` | chaîne | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Identifiant du modèle dans la nomenclature du fournisseur. Surchargé par `AGENT_MODEL`. |
11 | `baseUrl` | chaîne | `""` (défaut du fournisseur) | Point d'accès compatible OpenAI essayé en premier. Surchargé par `AGENT_BASE_URL`. |11 | `baseUrl` | chaîne | `""` (défaut du fournisseur) | Point d'accès compatible OpenAI essayé en premier. Surchargé par `AGENT_BASE_URL`. |
12 | `fallback` | chaîne ou absente | absente (défaut du fournisseur) | Point d'accès utilisé quand `baseUrl` ne répond pas sur `/models` en 2 s. Absente = défaut du fournisseur ; `""` = pas de repli. |12 | `fallback` | chaîne ou absente | absente (défaut du fournisseur) | Point d'accès utilisé quand `baseUrl` ne répond pas sur `/models` en 2 s. Absente = défaut du fournisseur ; `""` = pas de repli. |
@@ -24,7 +24,7 @@
24 | `fetchContentTool` | booléen | `false` | Déclare `fetch_content`, qui récupère des pages web en Markdown. Désactivé par défaut : c'est le seul outil intégré qui accède au réseau. |24 | `fetchContentTool` | booléen | `false` | Déclare `fetch_content`, qui récupère des pages web en Markdown. Désactivé par défaut : c'est le seul outil intégré qui accède au réseau. |
25 | `sampling` | dictionnaire | `temperature: 0.0`, `top_p: 0.9`, `max_tokens: 4096` | Paramètres de génération, noms de l'API OpenAI, transmis au serveur. L'`agent.yaml` livré ajoute `parallel_tool_calls: false`. `max_tokens` borne aussi un appel d'outil : un `write_file` de long document (ou le raisonnement d'un modèle « thinking » plus l'appel) peut l'atteindre, et l'appel est alors coupé. mm le rend deux fois au modèle en lui demandant de procéder par morceaux, puis s'arrête avec `the model hit max_tokens (N) while writing a tool call` ; augmentez la valeur si cela arrive. En streaming, mm demande aussi au serveur le compte des tokens (`stream_options.include_usage`). |25 | `sampling` | dictionnaire | `temperature: 0.0`, `top_p: 0.9`, `max_tokens: 4096` | Paramètres de génération, noms de l'API OpenAI, transmis au serveur. L'`agent.yaml` livré ajoute `parallel_tool_calls: false`. `max_tokens` borne aussi un appel d'outil : un `write_file` de long document (ou le raisonnement d'un modèle « thinking » plus l'appel) peut l'atteindre, et l'appel est alors coupé. mm le rend deux fois au modèle en lui demandant de procéder par morceaux, puis s'arrête avec `the model hit max_tokens (N) while writing a tool call` ; augmentez la valeur si cela arrive. En streaming, mm demande aussi au serveur le compte des tokens (`stream_options.include_usage`). |
26 | `watchdogTimeout` | durée | `20s` | Silence toléré entre deux tokens diffusés avant de déclarer la connexion bloquée. Doublé pour les requêtes de résumé. |26 | `watchdogTimeout` | durée | `20s` | Silence toléré entre deux tokens diffusés avant de déclarer la connexion bloquée. Doublé pour les requêtes de résumé. |
27-| `firstTokenTimeout` | durée | `3m` | Silence toléré avant le premier token d'un appel au modèle, pendant que le serveur lit le prompt (chaque aller-retour d'un tour en a un). Jamais plus court que `watchdogTimeout` ; doit être >= 0. Avec un serveur llama.cpp (`dmr`, `llamacpp`), mm demande `return_progress` : chaque rapport d'avancement relance ce délai, et le spinner affiche `Reading the prompt 12.0k/25.3k tokens`. À l'expiration, le watchdog dit `no first token from the server after …` (avant le premier token) ou `connection hang detected after …` (après). |27+| `firstTokenTimeout` | durée | `3m` | Silence toléré avant le premier token d'un appel au modèle, pendant que le serveur lit le prompt (chaque aller-retour d'un tour en a un). Jamais plus court que `watchdogTimeout` ; doit être >= 0. Avec un serveur llama.cpp (`dmr`, `llamacpp`, `llmman` — dont les modèles GGUF locaux tournent dans llama-server), mm demande `return_progress` : chaque rapport d'avancement relance ce délai, et le spinner affiche `Reading the prompt 12.0k/25.3k tokens`. Ollama n'envoie pas d'avancement : cette seule attente couvre tout le prompt (et le chargement du modèle). À l'expiration, le watchdog dit `no first token from the server after …` (avant le premier token) ou `connection hang detected after …` (après). |
28 | `context` | dictionnaire | voir ci-dessous | Compression automatique de l'historique. |28 | `context` | dictionnaire | voir ci-dessous | Compression automatique de l'historique. |
29 | `sessions` | dictionnaire | voir ci-dessous | Sessions persistantes, par projet. |29 | `sessions` | dictionnaire | voir ci-dessous | Sessions persistantes, par projet. |
30 | `acp` | dictionnaire | voir ci-dessous | La façon dont `mm -acp` se présente à l'éditeur. |30 | `acp` | dictionnaire | voir ci-dessous | La façon dont `mm -acp` se présente à l'éditeur. |
@@ -40,7 +40,7 @@
40 | `threshold` | entier | `75` | Pourcentage de la fenêtre de contexte au-delà duquel l'historique est compressé avant la question suivante. Entre 1 et 100. |40 | `threshold` | entier | `75` | Pourcentage de la fenêtre de contexte au-delà duquel l'historique est compressé avant la question suivante. Entre 1 et 100. |
41 | `maxMessages` | entier | `80` | Nombre de messages qui déclenche une compression quand la fenêtre est inconnue ou l'estimation trop basse. `0` désactive. Doit être ≥ 0. |41 | `maxMessages` | entier | `80` | Nombre de messages qui déclenche une compression quand la fenêtre est inconnue ou l'estimation trop basse. `0` désactive. Doit être ≥ 0. |
42 | `keepLastTurns` | entier | `3` | Tours de question conservés bruts en fin d'historique. Doit être ≥ 1. |42 | `keepLastTurns` | entier | `3` | Tours de question conservés bruts en fin d'historique. Doit être ≥ 1. |
43-| `summaryMaxTokens` | entier | `1200` | `max_tokens` de la requête de résumé. Doit être > 0. Le raisonnement d'un modèle qui réfléchit est décompté de ce budget : sur `dmr` et `llamacpp`, la requête de résumé désactive le raisonnement (`chat_template_kwargs.enable_thinking: false`) ; ailleurs, si l'erreur dit que le modèle « reasoned until the limit », augmentez-le. |43+| `summaryMaxTokens` | entier | `1200` | `max_tokens` de la requête de résumé. Doit être > 0. Le raisonnement d'un modèle qui réfléchit est décompté de ce budget : sur `dmr`, `llamacpp` et `llmman`, la requête de résumé désactive le raisonnement (`chat_template_kwargs.enable_thinking: false`), sur `ollama` aussi (`reasoning_effort: "none"`, le seul interrupteur qu'Ollama respecte) ; ailleurs, si l'erreur dit que le modèle « reasoned until the limit », augmentez-le. |
44 | `prompt` | chaîne | `""` | Remplace le prompt de résumé intégré quand non vide. |44 | `prompt` | chaîne | `""` | Remplace le prompt de résumé intégré quand non vide. |
45 | `showStats` | booléen | `true` | Affiche le rapport `🗜️` d'une ligne après chaque compression. |45 | `showStats` | booléen | `true` | Affiche le rapport `🗜️` d'une ligne après chaque compression. |
46 46
@@ -90,6 +90,8 @@ Le fichier livré `agent.lsp.yaml` est `agent.dmr.yaml` avec ce bloc actif (gopl
90 |-------------|-----------|------------|-------------|-------------|90 |-------------|-----------|------------|-------------|-------------|
91 | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | aucune | non |91 | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | aucune | non |
92 | `llamacpp` | `http://127.0.0.1:8080/v1` | aucun | `LLAMA_API_KEY` | non |92 | `llamacpp` | `http://127.0.0.1:8080/v1` | aucun | `LLAMA_API_KEY` | non |
93+| `llmman` | `http://127.0.0.1:17434/v1` | `http://host.docker.internal:17434/v1` | `LLMMAN_API_KEY` | non |
94+| `ollama` | `http://localhost:11434/v1` | `http://host.docker.internal:11434/v1` | `OLLAMA_API_KEY` | non |
93 95
94 ## Erreurs de validation96 ## Erreurs de validation
95 97
@@ -115,7 +117,7 @@ Signalées sous la forme `[config error: <chemin>: <message>]`, code de sortie 1
115 | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | une extension sans son point |117 | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | une extension sans son point |
116 | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | deux serveurs sur une même extension |118 | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | deux serveurs sur une même extension |
117 | `permissions must be "auto" or "ask"` | toute autre valeur |119 | `permissions must be "auto" or "ask"` | toute autre valeur |
118-| `unknown provider "x" (known: dmr, llamacpp)` | Signalée par le moteur, sous la forme `[engine error: …]`. |120+| `unknown provider "x" (known: dmr, llamacpp, llmman, ollama)` | Signalée par le moteur, sous la forme `[engine error: …]`. |
119 121
120 Un avertissement au démarrage, et non une erreur, est affiché quand `context.enabled` vaut `true`, que la fenêtre est inconnue et que `context.maxMessages` vaut `0`.122 Un avertissement au démarrage, et non une erreur, est affiché quand `context.enabled` vaut `true`, que la fenêtre est inconnue et que `context.maxMessages` vaut `0`.
121 123
modified internal/config/config.go +1 -1
@@ -21,7 +21,7 @@ import (
2121 // Config: everything that can be tuned without touching the code.
2222 type Config struct {
2323 // Provider: which LLM server is behind BaseURL — a key of the engine
24- // registry ("dmr", "llamacpp"). Defaults to "dmr" so that every agent.yaml
24+ // registry ("dmr", "llamacpp", "llmman", "ollama"). Defaults to "dmr" so that every agent.yaml
2525 // written before this key existed keeps its meaning.
2626 Provider string `yaml:"provider"`
2727
@@ -21,7 +21,7 @@ import (
21 // Config: everything that can be tuned without touching the code.21 // Config: everything that can be tuned without touching the code.
22 type Config struct {22 type Config struct {
23 // Provider: which LLM server is behind BaseURL — a key of the engine23 // Provider: which LLM server is behind BaseURL — a key of the engine
24- // registry ("dmr", "llamacpp"). Defaults to "dmr" so that every agent.yaml24+ // registry ("dmr", "llamacpp", "llmman", "ollama"). Defaults to "dmr" so that every agent.yaml
25 // written before this key existed keeps its meaning.25 // written before this key existed keeps its meaning.
26 Provider string `yaml:"provider"`26 Provider string `yaml:"provider"`
27 27
added internal/engine/apips.go +51 -0
new file mode 100644
@@ -0,0 +1,51 @@
1+package engine
2+
3+import "strings"
4+
5+// Ollama-style servers (Ollama itself, llmman, which copies its API) report,
6+// on GET /api/ps, every LOADED model with the `context_length` its runner was
7+// started with: the served window, the number the compression needs. Models
8+// load on the first request and unload after five idle minutes, so at
9+// start-up the window is usually unknown; EnsureContextWindow asks again
10+// before the next question and learns it. /api/show's trained context is
11+// deliberately not used: the server can serve less (Ollama's num_ctx /
12+// OLLAMA_CONTEXT_LENGTH, LLMMAN_CONTEXT_LENGTH), and a window too large means
13+// a compression that never triggers before the server refuses the prompt.
14+
15+// modelList is the body of GET /v1/models.
16+type modelList struct {
17+ Data []struct {
18+ ID string `json:"id"`
19+ } `json:"data"`
20+}
21+
22+// has reports whether id is among the listed models.
23+func (l modelList) has(id string) bool {
24+ for _, m := range l.Data {
25+ if m.ID == id {
26+ return true
27+ }
28+ }
29+ return false
30+}
31+
32+// loadedWindow reads /api/ps, one level above /v1, and returns the served
33+// context of `model` — 0 when it is not loaded or the route does not answer.
34+func loadedWindow(baseURL, model string) int {
35+ var ps struct {
36+ Models []struct {
37+ Name string `json:"name"`
38+ ContextLength int `json:"context_length"`
39+ } `json:"models"`
40+ }
41+ root := strings.TrimSuffix(strings.TrimSuffix(baseURL, "/"), "/v1")
42+ if !getJSON(root+"/api/ps", &ps) {
43+ return 0
44+ }
45+ for _, m := range ps.Models {
46+ if m.Name == model && m.ContextLength > 0 {
47+ return m.ContextLength
48+ }
49+ }
50+ return 0
51+}
new file mode 100644
@@ -0,0 +1,51 @@
1+package engine
2+
3+import "strings"
4+
5+// Ollama-style servers (Ollama itself, llmman, which copies its API) report,
6+// on GET /api/ps, every LOADED model with the `context_length` its runner was
7+// started with: the served window, the number the compression needs. Models
8+// load on the first request and unload after five idle minutes, so at
9+// start-up the window is usually unknown; EnsureContextWindow asks again
10+// before the next question and learns it. /api/show's trained context is
11+// deliberately not used: the server can serve less (Ollama's num_ctx /
12+// OLLAMA_CONTEXT_LENGTH, LLMMAN_CONTEXT_LENGTH), and a window too large means
13+// a compression that never triggers before the server refuses the prompt.
14+
15+// modelList is the body of GET /v1/models.
16+type modelList struct {
17+ Data []struct {
18+ ID string `json:"id"`
19+ } `json:"data"`
20+}
21+
22+// has reports whether id is among the listed models.
23+func (l modelList) has(id string) bool {
24+ for _, m := range l.Data {
25+ if m.ID == id {
26+ return true
27+ }
28+ }
29+ return false
30+}
31+
32+// loadedWindow reads /api/ps, one level above /v1, and returns the served
33+// context of `model` — 0 when it is not loaded or the route does not answer.
34+func loadedWindow(baseURL, model string) int {
35+ var ps struct {
36+ Models []struct {
37+ Name string `json:"name"`
38+ ContextLength int `json:"context_length"`
39+ } `json:"models"`
40+ }
41+ root := strings.TrimSuffix(strings.TrimSuffix(baseURL, "/"), "/v1")
42+ if !getJSON(root+"/api/ps", &ps) {
43+ return 0
44+ }
45+ for _, m := range ps.Models {
46+ if m.Name == model && m.ContextLength > 0 {
47+ return m.ContextLength
48+ }
49+ }
50+ return 0
51+}
added internal/engine/llmman.go +33 -0
new file mode 100644
@@ -0,0 +1,33 @@
1+package engine
2+
3+import (
4+ "context"
5+ "fmt"
6+)
7+
8+// probeLlmman: llmman (`llmman serve`) is a daemon in front of one
9+// llama-server per loaded model, so the llama.cpp probe does not apply as is.
10+// Measured on llmman 0.1.498: its /props is a stub — n_ctx 4096 and no chat
11+// template, model loaded or not, `?model=` or not — which would report a wrong
12+// window and a false "no chat template" warning. It has Ollama's /api/ps
13+// instead (apips.go).
14+//
15+// A model missing from /v1/models is NOT an error there: the first request
16+// pulls it, which can take minutes — worth a warning, because the wait looks
17+// like a stalled server.
18+func probeLlmman(_ context.Context, b Backend) Info {
19+ var list modelList
20+ info := Info{Reachable: getJSON(b.BaseURL+"/models", &list)}
21+ if !info.Reachable {
22+ return info
23+ }
24+ if !list.has(b.Model) {
25+ info.Warnings = append(info.Warnings,
26+ fmt.Sprintf("llmman: model %q is not in the store — the first question will pull it (or: llmman pull %s)", b.Model, b.Model))
27+ return info
28+ }
29+ if w := loadedWindow(b.BaseURL, b.Model); w > 0 {
30+ info.ContextWindow, info.ContextSource = w, "/api/ps"
31+ }
32+ return info
33+}
new file mode 100644
@@ -0,0 +1,33 @@
1+package engine
2+
3+import (
4+ "context"
5+ "fmt"
6+)
7+
8+// probeLlmman: llmman (`llmman serve`) is a daemon in front of one
9+// llama-server per loaded model, so the llama.cpp probe does not apply as is.
10+// Measured on llmman 0.1.498: its /props is a stub — n_ctx 4096 and no chat
11+// template, model loaded or not, `?model=` or not — which would report a wrong
12+// window and a false "no chat template" warning. It has Ollama's /api/ps
13+// instead (apips.go).
14+//
15+// A model missing from /v1/models is NOT an error there: the first request
16+// pulls it, which can take minutes — worth a warning, because the wait looks
17+// like a stalled server.
18+func probeLlmman(_ context.Context, b Backend) Info {
19+ var list modelList
20+ info := Info{Reachable: getJSON(b.BaseURL+"/models", &list)}
21+ if !info.Reachable {
22+ return info
23+ }
24+ if !list.has(b.Model) {
25+ info.Warnings = append(info.Warnings,
26+ fmt.Sprintf("llmman: model %q is not in the store — the first question will pull it (or: llmman pull %s)", b.Model, b.Model))
27+ return info
28+ }
29+ if w := loadedWindow(b.BaseURL, b.Model); w > 0 {
30+ info.ContextWindow, info.ContextSource = w, "/api/ps"
31+ }
32+ return info
33+}
added internal/engine/ollama.go +27 -0
new file mode 100644
@@ -0,0 +1,27 @@
1+package engine
2+
3+import (
4+ "context"
5+ "fmt"
6+)
7+
8+// probeOllama: Ollama has no /props; its served window is on /api/ps
9+// (apips.go), measured 2026-10-01 on Ollama 0.35.0 (`context_length: 32768`
10+// for a loaded qwen3.5:2b). Unlike llmman, Ollama's OpenAI API does NOT pull
11+// a missing model: the request fails with a 404, so the warning says to pull.
12+func probeOllama(_ context.Context, b Backend) Info {
13+ var list modelList
14+ info := Info{Reachable: getJSON(b.BaseURL+"/models", &list)}
15+ if !info.Reachable {
16+ return info
17+ }
18+ if !list.has(b.Model) {
19+ info.Warnings = append(info.Warnings,
20+ fmt.Sprintf("ollama: model %q is not in the server's list — ollama pull %s", b.Model, b.Model))
21+ return info
22+ }
23+ if w := loadedWindow(b.BaseURL, b.Model); w > 0 {
24+ info.ContextWindow, info.ContextSource = w, "/api/ps"
25+ }
26+ return info
27+}
new file mode 100644
@@ -0,0 +1,27 @@
1+package engine
2+
3+import (
4+ "context"
5+ "fmt"
6+)
7+
8+// probeOllama: Ollama has no /props; its served window is on /api/ps
9+// (apips.go), measured 2026-10-01 on Ollama 0.35.0 (`context_length: 32768`
10+// for a loaded qwen3.5:2b). Unlike llmman, Ollama's OpenAI API does NOT pull
11+// a missing model: the request fails with a 404, so the warning says to pull.
12+func probeOllama(_ context.Context, b Backend) Info {
13+ var list modelList
14+ info := Info{Reachable: getJSON(b.BaseURL+"/models", &list)}
15+ if !info.Reachable {
16+ return info
17+ }
18+ if !list.has(b.Model) {
19+ info.Warnings = append(info.Warnings,
20+ fmt.Sprintf("ollama: model %q is not in the server's list — ollama pull %s", b.Model, b.Model))
21+ return info
22+ }
23+ if w := loadedWindow(b.BaseURL, b.Model); w > 0 {
24+ info.ContextWindow, info.ContextSource = w, "/api/ps"
25+ }
26+ return info
27+}
modified internal/engine/openai_compat.go +48 -6
@@ -18,9 +18,12 @@ import (
1818 )
1919
2020 // openaiCompat is the Provider for every server that speaks the OpenAI
21-// chat-completions protocol. Docker Model Runner and llama-server are two
22-// parametrisations of it; Ollama's /v1, vLLM and the remote APIs would be more
23-// entries in init() below — the point of keeping the struct data-only.
21+// chat-completions protocol. Docker Model Runner, llama-server, llmman and
22+// Ollama's /v1 are four parametrisations of it; vLLM and the remote APIs would
23+// be more entries in init() below — the point of keeping the struct data-only.
24+// Each server keeps its own entry even when two look alike today: a difference
25+// found later (how to turn reasoning off, where the window is) is then one
26+// field to change, not a provider to split.
2427 type openaiCompat struct {
2528 name string
2629
@@ -48,9 +51,12 @@ type openaiCompat struct {
4851 probe func(ctx context.Context, b Backend) Info
4952
5053 // progress: the server is llama.cpp and understands return_progress
51- // (usage.go, progress.go) and chat_template_kwargs, which turns the
52- // summary's reasoning off (usage.go). DMR's default backend is llama.cpp.
54+ // (usage.go, progress.go). DMR's default backend is llama.cpp.
5355 progress bool
56+
57+ // thinkingOff: how this server is told not to reason, for the summary of
58+ // a compaction (usage.go). Zero value = no known way.
59+ thinkingOff thinkingOff
5460 }
5561
5662 func init() {
@@ -64,6 +70,7 @@ func init() {
6470 notFoundHint: "docker model pull %s",
6571 probe: probeModels,
6672 progress: true,
73+ thinkingOff: thinkingOffTemplate,
6774 })
6875 register(&openaiCompat{
6976 name: "llamacpp",
@@ -76,6 +83,41 @@ func init() {
7683 notFoundHint: "llama-server serves one model; check baseUrl ends with /v1 (model: %s)",
7784 probe: probeLlamaCpp,
7885 progress: true,
86+ thinkingOff: thinkingOffTemplate,
87+ })
88+ register(&openaiCompat{
89+ name: "llmman",
90+ baseURL: "http://127.0.0.1:17434/v1", // `llmman serve`'s default bind, plus the OpenAI prefix
91+ fallback: "http://host.docker.internal:17434/v1",
92+ // The daemon checks a key only when LLMMAN_API_KEYS is set (mandatory
93+ // off loopback); its own CLI reads LLMMAN_API_KEY, so do we.
94+ apiKeyEnv: "LLMMAN_API_KEY",
95+ dummyKey: "not-needed",
96+ startHint: "start it: llmman serve",
97+ notFoundHint: "llmman pull %s",
98+ probe: probeLlmman,
99+ // Local GGUF models are served by an upstream llama-server behind the
100+ // daemon, which passes return_progress and chat_template_kwargs through
101+ // (verified 2026-10-01: prompt_progress chunks stream back,
102+ // enable_thinking:false answers without reasoning).
103+ progress: true,
104+ thinkingOff: thinkingOffTemplate,
105+ })
106+ register(&openaiCompat{
107+ name: "ollama",
108+ baseURL: "http://localhost:11434/v1", // `ollama serve`'s default, plus the OpenAI prefix
109+ fallback: "http://host.docker.internal:11434/v1",
110+ // A local Ollama checks no key; ollama.com's API wants OLLAMA_API_KEY.
111+ apiKeyEnv: "OLLAMA_API_KEY",
112+ dummyKey: "not-needed",
113+ startHint: "start it: ollama serve",
114+ notFoundHint: "ollama pull %s",
115+ probe: probeOllama,
116+ // Measured 2026-10-01 on Ollama 0.35.0: no prompt_progress (the field
117+ // is ignored), reasoning streamed as delta.reasoning (reasoning.go reads
118+ // it), chat_template_kwargs ignored but reasoning_effort "none" obeyed.
119+ progress: false,
120+ thinkingOff: thinkingOffEffort,
79121 })
80122 }
81123
@@ -141,7 +183,7 @@ func (p *openaiCompat) Open(ctx context.Context, b Backend) (*genkit.Genkit, str
141183 // on the HTTP client, so a thinking model's reasoning can be read off
142184 // the wire the plugin drops it from (reasoning.go), and a streamed
143185 // request asks for the token counts it would otherwise lack (usage.go).
144- Opts: []option.RequestOption{option.WithMiddleware(streamOptions(p.progress), reasoningTap)},
186+ Opts: []option.RequestOption{option.WithMiddleware(streamOptions(p.progress, p.thinkingOff), reasoningTap)},
145187 }
146188 g := genkit.Init(ctx, genkit.WithPlugins(client))
147189 client.DefineModel(p.name, b.Model, ai.ModelOptions{
@@ -18,9 +18,12 @@ import (
18 )18 )
19 19
20 // openaiCompat is the Provider for every server that speaks the OpenAI20 // openaiCompat is the Provider for every server that speaks the OpenAI
21-// chat-completions protocol. Docker Model Runner and llama-server are two21+// chat-completions protocol. Docker Model Runner, llama-server, llmman and
22-// parametrisations of it; Ollama's /v1, vLLM and the remote APIs would be more22+// Ollama's /v1 are four parametrisations of it; vLLM and the remote APIs would
23-// entries in init() below — the point of keeping the struct data-only.23+// be more entries in init() below — the point of keeping the struct data-only.
24+// Each server keeps its own entry even when two look alike today: a difference
25+// found later (how to turn reasoning off, where the window is) is then one
26+// field to change, not a provider to split.
24 type openaiCompat struct {27 type openaiCompat struct {
25 name string28 name string
26 29
@@ -48,9 +51,12 @@ type openaiCompat struct {
48 probe func(ctx context.Context, b Backend) Info51 probe func(ctx context.Context, b Backend) Info
49 52
50 // progress: the server is llama.cpp and understands return_progress53 // progress: the server is llama.cpp and understands return_progress
51- // (usage.go, progress.go) and chat_template_kwargs, which turns the54+ // (usage.go, progress.go). DMR's default backend is llama.cpp.
52- // summary's reasoning off (usage.go). DMR's default backend is llama.cpp.
53 progress bool55 progress bool
56+
57+ // thinkingOff: how this server is told not to reason, for the summary of
58+ // a compaction (usage.go). Zero value = no known way.
59+ thinkingOff thinkingOff
54 }60 }
55 61
56 func init() {62 func init() {
@@ -64,6 +70,7 @@ func init() {
64 notFoundHint: "docker model pull %s",70 notFoundHint: "docker model pull %s",
65 probe: probeModels,71 probe: probeModels,
66 progress: true,72 progress: true,
73+ thinkingOff: thinkingOffTemplate,
67 })74 })
68 register(&openaiCompat{75 register(&openaiCompat{
69 name: "llamacpp",76 name: "llamacpp",
@@ -76,6 +83,41 @@ func init() {
76 notFoundHint: "llama-server serves one model; check baseUrl ends with /v1 (model: %s)",83 notFoundHint: "llama-server serves one model; check baseUrl ends with /v1 (model: %s)",
77 probe: probeLlamaCpp,84 probe: probeLlamaCpp,
78 progress: true,85 progress: true,
86+ thinkingOff: thinkingOffTemplate,
87+ })
88+ register(&openaiCompat{
89+ name: "llmman",
90+ baseURL: "http://127.0.0.1:17434/v1", // `llmman serve`'s default bind, plus the OpenAI prefix
91+ fallback: "http://host.docker.internal:17434/v1",
92+ // The daemon checks a key only when LLMMAN_API_KEYS is set (mandatory
93+ // off loopback); its own CLI reads LLMMAN_API_KEY, so do we.
94+ apiKeyEnv: "LLMMAN_API_KEY",
95+ dummyKey: "not-needed",
96+ startHint: "start it: llmman serve",
97+ notFoundHint: "llmman pull %s",
98+ probe: probeLlmman,
99+ // Local GGUF models are served by an upstream llama-server behind the
100+ // daemon, which passes return_progress and chat_template_kwargs through
101+ // (verified 2026-10-01: prompt_progress chunks stream back,
102+ // enable_thinking:false answers without reasoning).
103+ progress: true,
104+ thinkingOff: thinkingOffTemplate,
105+ })
106+ register(&openaiCompat{
107+ name: "ollama",
108+ baseURL: "http://localhost:11434/v1", // `ollama serve`'s default, plus the OpenAI prefix
109+ fallback: "http://host.docker.internal:11434/v1",
110+ // A local Ollama checks no key; ollama.com's API wants OLLAMA_API_KEY.
111+ apiKeyEnv: "OLLAMA_API_KEY",
112+ dummyKey: "not-needed",
113+ startHint: "start it: ollama serve",
114+ notFoundHint: "ollama pull %s",
115+ probe: probeOllama,
116+ // Measured 2026-10-01 on Ollama 0.35.0: no prompt_progress (the field
117+ // is ignored), reasoning streamed as delta.reasoning (reasoning.go reads
118+ // it), chat_template_kwargs ignored but reasoning_effort "none" obeyed.
119+ progress: false,
120+ thinkingOff: thinkingOffEffort,
79 })121 })
80 }122 }
81 123
@@ -141,7 +183,7 @@ func (p *openaiCompat) Open(ctx context.Context, b Backend) (*genkit.Genkit, str
141 // on the HTTP client, so a thinking model's reasoning can be read off183 // on the HTTP client, so a thinking model's reasoning can be read off
142 // the wire the plugin drops it from (reasoning.go), and a streamed184 // the wire the plugin drops it from (reasoning.go), and a streamed
143 // request asks for the token counts it would otherwise lack (usage.go).185 // request asks for the token counts it would otherwise lack (usage.go).
144- Opts: []option.RequestOption{option.WithMiddleware(streamOptions(p.progress), reasoningTap)},186+ Opts: []option.RequestOption{option.WithMiddleware(streamOptions(p.progress, p.thinkingOff), reasoningTap)},
145 }187 }
146 g := genkit.Init(ctx, genkit.WithPlugins(client))188 g := genkit.Init(ctx, genkit.WithPlugins(client))
147 client.DefineModel(p.name, b.Model, ai.ModelOptions{189 client.DefineModel(p.name, b.Model, ai.ModelOptions{
modified internal/engine/provider_test.go +85 -6
@@ -72,15 +72,15 @@ func ptr(s string) *string { return &s }
7272 // where DMR_BASE_URL is often exported.
7373 func clearEnv(t *testing.T) {
7474 t.Helper()
75- for _, k := range []string{"AGENT_BASE_URL", "DMR_BASE_URL", "LLAMA_API_KEY", "AGENT_PROVIDER", "AGENT_MODEL"} {
75+ for _, k := range []string{"AGENT_BASE_URL", "DMR_BASE_URL", "LLAMA_API_KEY", "LLMMAN_API_KEY", "OLLAMA_API_KEY", "AGENT_PROVIDER", "AGENT_MODEL"} {
7676 t.Setenv(k, "")
7777 }
7878 }
7979
8080 func TestLookupListsKnownProviders(t *testing.T) {
81- _, err := Lookup("ollama")
82- if err == nil || !strings.Contains(err.Error(), "dmr, llamacpp") {
83- t.Fatalf("Lookup(ollama) = %v, want an error naming the known providers", err)
81+ _, err := Lookup("vllm")
82+ if err == nil || !strings.Contains(err.Error(), "dmr, llamacpp, llmman, ollama") {
83+ t.Fatalf("Lookup(vllm) = %v, want an error naming the known providers", err)
8484 }
8585 }
8686
@@ -109,6 +109,12 @@ func TestResolveURLPrecedence(t *testing.T) {
109109 {"llamacpp ignores DMR_BASE_URL", "llamacpp", config.Config{}, map[string]string{"DMR_BASE_URL": down}, "http://127.0.0.1:8080/v1", "not-needed"},
110110 {"llamacpp reads LLAMA_API_KEY when set", "llamacpp", config.Config{}, map[string]string{"LLAMA_API_KEY": "secret"}, "http://127.0.0.1:8080/v1", "secret"},
111111 {"apiKeyEnv names another variable", "llamacpp", config.Config{APIKeyEnv: "MY_KEY"}, map[string]string{"MY_KEY": "k2"}, "http://127.0.0.1:8080/v1", "k2"},
112+ {"llmman falls back to the host's daemon", "llmman", config.Config{BaseURL: down}, nil, "http://host.docker.internal:17434/v1", "not-needed"},
113+ {"llmman keeps a reachable baseUrl", "llmman", config.Config{BaseURL: up.URL}, nil, up.URL, "not-needed"},
114+ {"llmman reads LLMMAN_API_KEY when set", "llmman", config.Config{BaseURL: up.URL}, map[string]string{"LLMMAN_API_KEY": "k3", "LLAMA_API_KEY": "secret"}, up.URL, "k3"},
115+ {"ollama falls back to the host's server", "ollama", config.Config{BaseURL: down}, nil, "http://host.docker.internal:11434/v1", "not-needed"},
116+ {"ollama keeps a reachable baseUrl", "ollama", config.Config{BaseURL: up.URL}, nil, up.URL, "not-needed"},
117+ {"ollama reads OLLAMA_API_KEY when set", "ollama", config.Config{BaseURL: up.URL}, map[string]string{"OLLAMA_API_KEY": "k4", "LLMMAN_API_KEY": "k3"}, up.URL, "k4"},
112118 }
113119 for _, tc := range cases {
114120 t.Run(tc.name, func(t *testing.T) {
@@ -447,11 +453,84 @@ func TestConfigCompat(t *testing.T) {
447453 }
448454
449455 config.Cfg = saved
450- bad := write("bad.yaml", "provider: ollama\nmodel: m\n")
456+ bad := write("bad.yaml", "provider: vllm\nmodel: m\n")
451457 if _, err := config.Load(bad); err != nil {
452458 t.Fatalf("Load must accept an unknown provider (engine.New refuses it): %v", err)
453459 }
454460 if _, err := Lookup(config.Cfg.Provider); err == nil {
455- t.Error("Lookup(ollama) must fail")
461+ t.Error("Lookup(vllm) must fail")
462+ }
463+}
464+
465+// TestProbeLlmman: llmman's /props is a stub (n_ctx 4096, no template), so the
466+// served window comes from /api/ps, and only for the configured model once it
467+// is loaded; a model missing from /v1/models is a warning (the first question
468+// pulls it), never an error. The stub /props is served here on purpose: the
469+// probe must not read it.
470+func TestProbeLlmman(t *testing.T) {
471+ loaded := false
472+ mux := http.NewServeMux()
473+ mux.HandleFunc("/props", func(w http.ResponseWriter, _ *http.Request) {
474+ _, _ = w.Write([]byte(`{"default_generation_settings":{"n_ctx":4096},"chat_template":""}`))
475+ })
476+ mux.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) {
477+ _, _ = w.Write([]byte(`{"object":"list","data":[{"id":"hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M","object":"model"}]}`))
478+ })
479+ mux.HandleFunc("/api/ps", func(w http.ResponseWriter, _ *http.Request) {
480+ if !loaded {
481+ _, _ = w.Write([]byte(`{"models":[]}`))
482+ return
483+ }
484+ _, _ = w.Write([]byte(`{"models":[{"name":"other","context_length":8192},{"name":"hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M","context_length":32768}]}`))
485+ })
486+ srv := httptest.NewServer(mux)
487+ defer srv.Close()
488+
489+ p, _ := Lookup("llmman")
490+ b := Backend{Provider: "llmman", BaseURL: srv.URL + "/v1", Model: "hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M"}
491+
492+ info := p.Probe(context.Background(), b)
493+ if !info.Reachable || info.ContextWindow != 0 || len(info.Warnings) != 0 {
494+ t.Errorf("not loaded: %+v, want reachable, window unknown, no warning", info)
495+ }
496+
497+ loaded = true
498+ info = p.Probe(context.Background(), b)
499+ if info.ContextWindow != 32768 || info.ContextSource != "/api/ps" {
500+ t.Errorf("loaded: %+v, want 32768 from /api/ps", info)
501+ }
502+
503+ b.Model = "hf.co/unsloth/Missing-GGUF:Q4_K_M"
504+ info = p.Probe(context.Background(), b)
505+ if !info.Reachable || len(info.Warnings) != 1 || !strings.Contains(info.Warnings[0], "llmman pull") {
506+ t.Errorf("missing model: %+v, want reachable and one warning naming llmman pull", info)
507+ }
508+}
509+
510+// TestProbeOllama: same /api/ps reading as llmman (measured shape, Ollama
511+// 0.35.0), but a missing model is a pull to do — Ollama's OpenAI API answers
512+// 404 instead of pulling it.
513+func TestProbeOllama(t *testing.T) {
514+ mux := http.NewServeMux()
515+ mux.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) {
516+ _, _ = w.Write([]byte(`{"object":"list","data":[{"id":"qwen3.5:2b","object":"model","owned_by":"library"}]}`))
517+ })
518+ mux.HandleFunc("/api/ps", func(w http.ResponseWriter, _ *http.Request) {
519+ _, _ = w.Write([]byte(`{"models":[{"name":"qwen3.5:2b","model":"qwen3.5:2b","context_length":32768}]}`))
520+ })
521+ srv := httptest.NewServer(mux)
522+ defer srv.Close()
523+
524+ p, _ := Lookup("ollama")
525+ b := Backend{Provider: "ollama", BaseURL: srv.URL + "/v1", Model: "qwen3.5:2b"}
526+ info := p.Probe(context.Background(), b)
527+ if !info.Reachable || info.ContextWindow != 32768 || info.ContextSource != "/api/ps" || len(info.Warnings) != 0 {
528+ t.Errorf("loaded: %+v, want reachable, 32768 from /api/ps, no warning", info)
529+ }
530+
531+ b.Model = "qwen3.5:27b"
532+ info = p.Probe(context.Background(), b)
533+ if len(info.Warnings) != 1 || !strings.Contains(info.Warnings[0], "ollama pull qwen3.5:27b") || info.ContextWindow != 0 {
534+ t.Errorf("missing model: %+v, want one warning naming ollama pull, window unknown", info)
456535 }
457536 }
@@ -72,15 +72,15 @@ func ptr(s string) *string { return &s }
72 // where DMR_BASE_URL is often exported.72 // where DMR_BASE_URL is often exported.
73 func clearEnv(t *testing.T) {73 func clearEnv(t *testing.T) {
74 t.Helper()74 t.Helper()
75- for _, k := range []string{"AGENT_BASE_URL", "DMR_BASE_URL", "LLAMA_API_KEY", "AGENT_PROVIDER", "AGENT_MODEL"} {75+ for _, k := range []string{"AGENT_BASE_URL", "DMR_BASE_URL", "LLAMA_API_KEY", "LLMMAN_API_KEY", "OLLAMA_API_KEY", "AGENT_PROVIDER", "AGENT_MODEL"} {
76 t.Setenv(k, "")76 t.Setenv(k, "")
77 }77 }
78 }78 }
79 79
80 func TestLookupListsKnownProviders(t *testing.T) {80 func TestLookupListsKnownProviders(t *testing.T) {
81- _, err := Lookup("ollama")81+ _, err := Lookup("vllm")
82- if err == nil || !strings.Contains(err.Error(), "dmr, llamacpp") {82+ if err == nil || !strings.Contains(err.Error(), "dmr, llamacpp, llmman, ollama") {
83- t.Fatalf("Lookup(ollama) = %v, want an error naming the known providers", err)83+ t.Fatalf("Lookup(vllm) = %v, want an error naming the known providers", err)
84 }84 }
85 }85 }
86 86
@@ -109,6 +109,12 @@ func TestResolveURLPrecedence(t *testing.T) {
109 {"llamacpp ignores DMR_BASE_URL", "llamacpp", config.Config{}, map[string]string{"DMR_BASE_URL": down}, "http://127.0.0.1:8080/v1", "not-needed"},109 {"llamacpp ignores DMR_BASE_URL", "llamacpp", config.Config{}, map[string]string{"DMR_BASE_URL": down}, "http://127.0.0.1:8080/v1", "not-needed"},
110 {"llamacpp reads LLAMA_API_KEY when set", "llamacpp", config.Config{}, map[string]string{"LLAMA_API_KEY": "secret"}, "http://127.0.0.1:8080/v1", "secret"},110 {"llamacpp reads LLAMA_API_KEY when set", "llamacpp", config.Config{}, map[string]string{"LLAMA_API_KEY": "secret"}, "http://127.0.0.1:8080/v1", "secret"},
111 {"apiKeyEnv names another variable", "llamacpp", config.Config{APIKeyEnv: "MY_KEY"}, map[string]string{"MY_KEY": "k2"}, "http://127.0.0.1:8080/v1", "k2"},111 {"apiKeyEnv names another variable", "llamacpp", config.Config{APIKeyEnv: "MY_KEY"}, map[string]string{"MY_KEY": "k2"}, "http://127.0.0.1:8080/v1", "k2"},
112+ {"llmman falls back to the host's daemon", "llmman", config.Config{BaseURL: down}, nil, "http://host.docker.internal:17434/v1", "not-needed"},
113+ {"llmman keeps a reachable baseUrl", "llmman", config.Config{BaseURL: up.URL}, nil, up.URL, "not-needed"},
114+ {"llmman reads LLMMAN_API_KEY when set", "llmman", config.Config{BaseURL: up.URL}, map[string]string{"LLMMAN_API_KEY": "k3", "LLAMA_API_KEY": "secret"}, up.URL, "k3"},
115+ {"ollama falls back to the host's server", "ollama", config.Config{BaseURL: down}, nil, "http://host.docker.internal:11434/v1", "not-needed"},
116+ {"ollama keeps a reachable baseUrl", "ollama", config.Config{BaseURL: up.URL}, nil, up.URL, "not-needed"},
117+ {"ollama reads OLLAMA_API_KEY when set", "ollama", config.Config{BaseURL: up.URL}, map[string]string{"OLLAMA_API_KEY": "k4", "LLMMAN_API_KEY": "k3"}, up.URL, "k4"},
112 }118 }
113 for _, tc := range cases {119 for _, tc := range cases {
114 t.Run(tc.name, func(t *testing.T) {120 t.Run(tc.name, func(t *testing.T) {
@@ -447,11 +453,84 @@ func TestConfigCompat(t *testing.T) {
447 }453 }
448 454
449 config.Cfg = saved455 config.Cfg = saved
450- bad := write("bad.yaml", "provider: ollama\nmodel: m\n")456+ bad := write("bad.yaml", "provider: vllm\nmodel: m\n")
451 if _, err := config.Load(bad); err != nil {457 if _, err := config.Load(bad); err != nil {
452 t.Fatalf("Load must accept an unknown provider (engine.New refuses it): %v", err)458 t.Fatalf("Load must accept an unknown provider (engine.New refuses it): %v", err)
453 }459 }
454 if _, err := Lookup(config.Cfg.Provider); err == nil {460 if _, err := Lookup(config.Cfg.Provider); err == nil {
455- t.Error("Lookup(ollama) must fail")461+ t.Error("Lookup(vllm) must fail")
462+ }
463+}
464+
465+// TestProbeLlmman: llmman's /props is a stub (n_ctx 4096, no template), so the
466+// served window comes from /api/ps, and only for the configured model once it
467+// is loaded; a model missing from /v1/models is a warning (the first question
468+// pulls it), never an error. The stub /props is served here on purpose: the
469+// probe must not read it.
470+func TestProbeLlmman(t *testing.T) {
471+ loaded := false
472+ mux := http.NewServeMux()
473+ mux.HandleFunc("/props", func(w http.ResponseWriter, _ *http.Request) {
474+ _, _ = w.Write([]byte(`{"default_generation_settings":{"n_ctx":4096},"chat_template":""}`))
475+ })
476+ mux.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) {
477+ _, _ = w.Write([]byte(`{"object":"list","data":[{"id":"hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M","object":"model"}]}`))
478+ })
479+ mux.HandleFunc("/api/ps", func(w http.ResponseWriter, _ *http.Request) {
480+ if !loaded {
481+ _, _ = w.Write([]byte(`{"models":[]}`))
482+ return
483+ }
484+ _, _ = w.Write([]byte(`{"models":[{"name":"other","context_length":8192},{"name":"hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M","context_length":32768}]}`))
485+ })
486+ srv := httptest.NewServer(mux)
487+ defer srv.Close()
488+
489+ p, _ := Lookup("llmman")
490+ b := Backend{Provider: "llmman", BaseURL: srv.URL + "/v1", Model: "hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M"}
491+
492+ info := p.Probe(context.Background(), b)
493+ if !info.Reachable || info.ContextWindow != 0 || len(info.Warnings) != 0 {
494+ t.Errorf("not loaded: %+v, want reachable, window unknown, no warning", info)
495+ }
496+
497+ loaded = true
498+ info = p.Probe(context.Background(), b)
499+ if info.ContextWindow != 32768 || info.ContextSource != "/api/ps" {
500+ t.Errorf("loaded: %+v, want 32768 from /api/ps", info)
501+ }
502+
503+ b.Model = "hf.co/unsloth/Missing-GGUF:Q4_K_M"
504+ info = p.Probe(context.Background(), b)
505+ if !info.Reachable || len(info.Warnings) != 1 || !strings.Contains(info.Warnings[0], "llmman pull") {
506+ t.Errorf("missing model: %+v, want reachable and one warning naming llmman pull", info)
507+ }
508+}
509+
510+// TestProbeOllama: same /api/ps reading as llmman (measured shape, Ollama
511+// 0.35.0), but a missing model is a pull to do — Ollama's OpenAI API answers
512+// 404 instead of pulling it.
513+func TestProbeOllama(t *testing.T) {
514+ mux := http.NewServeMux()
515+ mux.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) {
516+ _, _ = w.Write([]byte(`{"object":"list","data":[{"id":"qwen3.5:2b","object":"model","owned_by":"library"}]}`))
517+ })
518+ mux.HandleFunc("/api/ps", func(w http.ResponseWriter, _ *http.Request) {
519+ _, _ = w.Write([]byte(`{"models":[{"name":"qwen3.5:2b","model":"qwen3.5:2b","context_length":32768}]}`))
520+ })
521+ srv := httptest.NewServer(mux)
522+ defer srv.Close()
523+
524+ p, _ := Lookup("ollama")
525+ b := Backend{Provider: "ollama", BaseURL: srv.URL + "/v1", Model: "qwen3.5:2b"}
526+ info := p.Probe(context.Background(), b)
527+ if !info.Reachable || info.ContextWindow != 32768 || info.ContextSource != "/api/ps" || len(info.Warnings) != 0 {
528+ t.Errorf("loaded: %+v, want reachable, 32768 from /api/ps, no warning", info)
529+ }
530+
531+ b.Model = "qwen3.5:27b"
532+ info = p.Probe(context.Background(), b)
533+ if len(info.Warnings) != 1 || !strings.Contains(info.Warnings[0], "ollama pull qwen3.5:27b") || info.ContextWindow != 0 {
534+ t.Errorf("missing model: %+v, want one warning naming ollama pull, window unknown", info)
456 }535 }
457 }536 }
modified internal/engine/summary_test.go +36 -4
@@ -102,13 +102,17 @@ func TestEmptySummaryWithoutReasoning(t *testing.T) {
102102 }
103103 }
104104
105-// The middleware turns the reasoning off only for a llama.cpp provider, only
106-// under a marked context, and never over a request that chose for itself.
105+// The middleware turns the reasoning off only for a provider that knows how,
106+// only under a marked context, and never over a request that chose for itself.
107107 func TestStreamOptionsWithoutThinking(t *testing.T) {
108- send := func(ctx context.Context, progress bool, body string) string {
108+ send := func(ctx context.Context, llamacpp bool, body string) string {
109+ off := thinkingOffUnknown
110+ if llamacpp {
111+ off = thinkingOffTemplate
112+ }
109113 req, _ := http.NewRequestWithContext(ctx, http.MethodPost, "http://x/v1/chat/completions", strings.NewReader(body))
110114 var seen string
111- _, _ = streamOptions(progress)(req, func(r *http.Request) (*http.Response, error) {
115+ _, _ = streamOptions(llamacpp, off)(req, func(r *http.Request) (*http.Response, error) {
112116 b, _ := io.ReadAll(r.Body)
113117 seen = string(b)
114118 return &http.Response{StatusCode: 200, Body: http.NoBody}, nil
@@ -135,3 +139,31 @@ func TestStreamOptionsWithoutThinking(t *testing.T) {
135139 t.Errorf("a request that chose was changed: %s", got)
136140 }
137141 }
142+
143+// Ollama ignores chat_template_kwargs (measured: qwen3.5:2b reasoned through
144+// the whole budget) and obeys reasoning_effort "none": its provider must send
145+// that, and only that, under a marked context — and leave a request that chose.
146+func TestStreamOptionsWithoutThinkingOllama(t *testing.T) {
147+ send := func(ctx context.Context, body string) string {
148+ req, _ := http.NewRequestWithContext(ctx, http.MethodPost, "http://x/v1/chat/completions", strings.NewReader(body))
149+ var seen string
150+ _, _ = streamOptions(false, thinkingOffEffort)(req, func(r *http.Request) (*http.Response, error) {
151+ b, _ := io.ReadAll(r.Body)
152+ seen = string(b)
153+ return &http.Response{StatusCode: 200, Body: http.NoBody}, nil
154+ })
155+ return seen
156+ }
157+ marked := withoutThinking(context.Background())
158+ got := send(marked, `{"stream":true}`)
159+ if !strings.Contains(got, `"reasoning_effort":"none"`) || strings.Contains(got, "chat_template_kwargs") || strings.Contains(got, "return_progress") {
160+ t.Errorf("ollama, marked: %s", got)
161+ }
162+ if got := send(context.Background(), `{"stream":true}`); strings.Contains(got, "reasoning_effort") {
163+ t.Errorf("ollama, not marked: %s", got)
164+ }
165+ chose := `{"stream":false,"reasoning_effort":"high"}`
166+ if got := send(marked, chose); got != chose {
167+ t.Errorf("a request that chose was changed: %s", got)
168+ }
169+}
@@ -102,13 +102,17 @@ func TestEmptySummaryWithoutReasoning(t *testing.T) {
102 }102 }
103 }103 }
104 104
105-// The middleware turns the reasoning off only for a llama.cpp provider, only105+// The middleware turns the reasoning off only for a provider that knows how,
106-// under a marked context, and never over a request that chose for itself.106+// only under a marked context, and never over a request that chose for itself.
107 func TestStreamOptionsWithoutThinking(t *testing.T) {107 func TestStreamOptionsWithoutThinking(t *testing.T) {
108- send := func(ctx context.Context, progress bool, body string) string {108+ send := func(ctx context.Context, llamacpp bool, body string) string {
109+ off := thinkingOffUnknown
110+ if llamacpp {
111+ off = thinkingOffTemplate
112+ }
109 req, _ := http.NewRequestWithContext(ctx, http.MethodPost, "http://x/v1/chat/completions", strings.NewReader(body))113 req, _ := http.NewRequestWithContext(ctx, http.MethodPost, "http://x/v1/chat/completions", strings.NewReader(body))
110 var seen string114 var seen string
111- _, _ = streamOptions(progress)(req, func(r *http.Request) (*http.Response, error) {115+ _, _ = streamOptions(llamacpp, off)(req, func(r *http.Request) (*http.Response, error) {
112 b, _ := io.ReadAll(r.Body)116 b, _ := io.ReadAll(r.Body)
113 seen = string(b)117 seen = string(b)
114 return &http.Response{StatusCode: 200, Body: http.NoBody}, nil118 return &http.Response{StatusCode: 200, Body: http.NoBody}, nil
@@ -135,3 +139,31 @@ func TestStreamOptionsWithoutThinking(t *testing.T) {
135 t.Errorf("a request that chose was changed: %s", got)139 t.Errorf("a request that chose was changed: %s", got)
136 }140 }
137 }141 }
142+
143+// Ollama ignores chat_template_kwargs (measured: qwen3.5:2b reasoned through
144+// the whole budget) and obeys reasoning_effort "none": its provider must send
145+// that, and only that, under a marked context — and leave a request that chose.
146+func TestStreamOptionsWithoutThinkingOllama(t *testing.T) {
147+ send := func(ctx context.Context, body string) string {
148+ req, _ := http.NewRequestWithContext(ctx, http.MethodPost, "http://x/v1/chat/completions", strings.NewReader(body))
149+ var seen string
150+ _, _ = streamOptions(false, thinkingOffEffort)(req, func(r *http.Request) (*http.Response, error) {
151+ b, _ := io.ReadAll(r.Body)
152+ seen = string(b)
153+ return &http.Response{StatusCode: 200, Body: http.NoBody}, nil
154+ })
155+ return seen
156+ }
157+ marked := withoutThinking(context.Background())
158+ got := send(marked, `{"stream":true}`)
159+ if !strings.Contains(got, `"reasoning_effort":"none"`) || strings.Contains(got, "chat_template_kwargs") || strings.Contains(got, "return_progress") {
160+ t.Errorf("ollama, marked: %s", got)
161+ }
162+ if got := send(context.Background(), `{"stream":true}`); strings.Contains(got, "reasoning_effort") {
163+ t.Errorf("ollama, not marked: %s", got)
164+ }
165+ chose := `{"stream":false,"reasoning_effort":"high"}`
166+ if got := send(marked, chose); got != chose {
167+ t.Errorf("a request that chose was changed: %s", got)
168+ }
169+}
modified internal/engine/usage.go +41 -15
@@ -46,8 +46,25 @@ import (
4646 // summaryMaxTokens and every compaction failed with "empty summary". Notes
4747 // need no reasoning. It cannot go in the sampling config either: compat_oai
4848 // maps the config onto openai.ChatCompletionNewParams, which has no such
49-// field. llama.cpp providers only, like return_progress; a template without
50-// the variable ignores it.
49+// field. How to say it depends on the server (thinkingOff below): llama.cpp
50+// reads the template variable, and a template without it ignores it; Ollama
51+// ignores chat_template_kwargs entirely (measured 2026-10-01, Ollama 0.35.0,
52+// qwen3.5:2b: it reasoned through the whole budget) and obeys
53+// `reasoning_effort: "none"` instead (same model: the answer, no reasoning).
54+
55+// thinkingOff is how a provider's server is told not to reason.
56+type thinkingOff int
57+
58+const (
59+ // thinkingOffUnknown: no known way — the request goes as is, and an empty
60+ // summary says "raise context.summaryMaxTokens".
61+ thinkingOffUnknown thinkingOff = iota
62+ // thinkingOffTemplate: chat_template_kwargs.enable_thinking = false
63+ // (llama.cpp, so DMR and llmman's local models).
64+ thinkingOffTemplate
65+ // thinkingOffEffort: reasoning_effort = "none" (Ollama).
66+ thinkingOffEffort
67+)
5168
5269 // noThinkKey is the context key marking a request whose reasoning is off.
5370 type noThinkKey struct{}
@@ -60,16 +77,16 @@ func withoutThinking(ctx context.Context) context.Context {
6077
6178 // streamOptions returns the openai-go middleware that adds those options to a
6279 // request that streams and has not chosen for itself; `progress` — the server
63-// is llama.cpp — adds return_progress, and turns the reasoning off when the
64-// request's context asks. Anything else — not JSON, not streamed, the options
65-// already set — goes through untouched.
66-func streamOptions(progress bool) option.Middleware {
80+// is llama.cpp — adds return_progress; `off` turns the reasoning off, the
81+// server's way, when the request's context asks. Anything else — not JSON,
82+// not streamed, the options already set — goes through untouched.
83+func streamOptions(progress bool, off thinkingOff) option.Middleware {
6784 return func(req *http.Request, next option.MiddlewareNext) (*http.Response, error) {
6885 if req.Method == http.MethodPost && req.Body != nil {
6986 if body, err := io.ReadAll(req.Body); err == nil {
7087 _ = req.Body.Close()
71- if noThink, _ := req.Context().Value(noThinkKey{}).(bool); progress && noThink {
72- body = withoutReasoning(body)
88+ if noThink, _ := req.Context().Value(noThinkKey{}).(bool); noThink {
89+ body = withoutReasoning(body, off)
7390 }
7491 body = withStreamOptions(body, progress)
7592 req.Body = io.NopCloser(bytes.NewReader(body))
@@ -113,19 +130,28 @@ func withStreamOptions(body []byte, progress bool) []byte {
113130 return out
114131 }
115132
116-// withoutReasoning returns body with chat_template_kwargs.enable_thinking set
117-// to false, streamed or not. A request that already carries
118-// chat_template_kwargs has chosen for itself and is left as is, like a body
119-// that is not JSON.
120-func withoutReasoning(body []byte) []byte {
133+// withoutReasoning returns body with the reasoning turned off the way `off`
134+// says, streamed or not. A request that already carries the field has chosen
135+// for itself and is left as is, like a body that is not JSON, or a server
136+// with no known way (thinkingOffUnknown).
137+func withoutReasoning(body []byte, off thinkingOff) []byte {
138+ key, value := "", ""
139+ switch off {
140+ case thinkingOffTemplate:
141+ key, value = "chat_template_kwargs", `{"enable_thinking":false}`
142+ case thinkingOffEffort:
143+ key, value = "reasoning_effort", `"none"`
144+ default:
145+ return body
146+ }
121147 var m map[string]json.RawMessage
122148 if json.Unmarshal(body, &m) != nil {
123149 return body
124150 }
125- if _, set := m["chat_template_kwargs"]; set {
151+ if _, set := m[key]; set {
126152 return body
127153 }
128- m["chat_template_kwargs"] = json.RawMessage(`{"enable_thinking":false}`)
154+ m[key] = json.RawMessage(value)
129155 out, err := json.Marshal(m)
130156 if err != nil {
131157 return body
@@ -46,8 +46,25 @@ import (
46 // summaryMaxTokens and every compaction failed with "empty summary". Notes46 // summaryMaxTokens and every compaction failed with "empty summary". Notes
47 // need no reasoning. It cannot go in the sampling config either: compat_oai47 // need no reasoning. It cannot go in the sampling config either: compat_oai
48 // maps the config onto openai.ChatCompletionNewParams, which has no such48 // maps the config onto openai.ChatCompletionNewParams, which has no such
49-// field. llama.cpp providers only, like return_progress; a template without49+// field. How to say it depends on the server (thinkingOff below): llama.cpp
50-// the variable ignores it.50+// reads the template variable, and a template without it ignores it; Ollama
51+// ignores chat_template_kwargs entirely (measured 2026-10-01, Ollama 0.35.0,
52+// qwen3.5:2b: it reasoned through the whole budget) and obeys
53+// `reasoning_effort: "none"` instead (same model: the answer, no reasoning).
54+
55+// thinkingOff is how a provider's server is told not to reason.
56+type thinkingOff int
57+
58+const (
59+ // thinkingOffUnknown: no known way — the request goes as is, and an empty
60+ // summary says "raise context.summaryMaxTokens".
61+ thinkingOffUnknown thinkingOff = iota
62+ // thinkingOffTemplate: chat_template_kwargs.enable_thinking = false
63+ // (llama.cpp, so DMR and llmman's local models).
64+ thinkingOffTemplate
65+ // thinkingOffEffort: reasoning_effort = "none" (Ollama).
66+ thinkingOffEffort
67+)
51 68
52 // noThinkKey is the context key marking a request whose reasoning is off.69 // noThinkKey is the context key marking a request whose reasoning is off.
53 type noThinkKey struct{}70 type noThinkKey struct{}
@@ -60,16 +77,16 @@ func withoutThinking(ctx context.Context) context.Context {
60 77
61 // streamOptions returns the openai-go middleware that adds those options to a78 // streamOptions returns the openai-go middleware that adds those options to a
62 // request that streams and has not chosen for itself; `progress` — the server79 // request that streams and has not chosen for itself; `progress` — the server
63-// is llama.cpp — adds return_progress, and turns the reasoning off when the80+// is llama.cpp — adds return_progress; `off` turns the reasoning off, the
64-// request's context asks. Anything else — not JSON, not streamed, the options81+// server's way, when the request's context asks. Anything else — not JSON,
65-// already set — goes through untouched.82+// not streamed, the options already set — goes through untouched.
66-func streamOptions(progress bool) option.Middleware {83+func streamOptions(progress bool, off thinkingOff) option.Middleware {
67 return func(req *http.Request, next option.MiddlewareNext) (*http.Response, error) {84 return func(req *http.Request, next option.MiddlewareNext) (*http.Response, error) {
68 if req.Method == http.MethodPost && req.Body != nil {85 if req.Method == http.MethodPost && req.Body != nil {
69 if body, err := io.ReadAll(req.Body); err == nil {86 if body, err := io.ReadAll(req.Body); err == nil {
70 _ = req.Body.Close()87 _ = req.Body.Close()
71- if noThink, _ := req.Context().Value(noThinkKey{}).(bool); progress && noThink {88+ if noThink, _ := req.Context().Value(noThinkKey{}).(bool); noThink {
72- body = withoutReasoning(body)89+ body = withoutReasoning(body, off)
73 }90 }
74 body = withStreamOptions(body, progress)91 body = withStreamOptions(body, progress)
75 req.Body = io.NopCloser(bytes.NewReader(body))92 req.Body = io.NopCloser(bytes.NewReader(body))
@@ -113,19 +130,28 @@ func withStreamOptions(body []byte, progress bool) []byte {
113 return out130 return out
114 }131 }
115 132
116-// withoutReasoning returns body with chat_template_kwargs.enable_thinking set133+// withoutReasoning returns body with the reasoning turned off the way `off`
117-// to false, streamed or not. A request that already carries134+// says, streamed or not. A request that already carries the field has chosen
118-// chat_template_kwargs has chosen for itself and is left as is, like a body135+// for itself and is left as is, like a body that is not JSON, or a server
119-// that is not JSON.136+// with no known way (thinkingOffUnknown).
120-func withoutReasoning(body []byte) []byte {137+func withoutReasoning(body []byte, off thinkingOff) []byte {
138+ key, value := "", ""
139+ switch off {
140+ case thinkingOffTemplate:
141+ key, value = "chat_template_kwargs", `{"enable_thinking":false}`
142+ case thinkingOffEffort:
143+ key, value = "reasoning_effort", `"none"`
144+ default:
145+ return body
146+ }
121 var m map[string]json.RawMessage147 var m map[string]json.RawMessage
122 if json.Unmarshal(body, &m) != nil {148 if json.Unmarshal(body, &m) != nil {
123 return body149 return body
124 }150 }
125- if _, set := m["chat_template_kwargs"]; set {151+ if _, set := m[key]; set {
126 return body152 return body
127 }153 }
128- m["chat_template_kwargs"] = json.RawMessage(`{"enable_thinking":false}`)154+ m[key] = json.RawMessage(value)
129 out, err := json.Marshal(m)155 out, err := json.Marshal(m)
130 if err != nil {156 if err != nil {
131 return body157 return body
modified internal/engine/usage_test.go +1 -1
@@ -42,7 +42,7 @@ func TestWithIncludeUsage(t *testing.T) {
4242 func TestStreamUsageRewritesTheRequest(t *testing.T) {
4343 req, _ := http.NewRequest(http.MethodPost, "http://x/v1/chat/completions", strings.NewReader(`{"stream":true}`))
4444 var seen string
45- _, _ = streamOptions(false)(req, func(r *http.Request) (*http.Response, error) {
45+ _, _ = streamOptions(false, thinkingOffUnknown)(req, func(r *http.Request) (*http.Response, error) {
4646 b, _ := io.ReadAll(r.Body)
4747 seen = string(b)
4848 if r.ContentLength != int64(len(b)) {
@@ -42,7 +42,7 @@ func TestWithIncludeUsage(t *testing.T) {
42 func TestStreamUsageRewritesTheRequest(t *testing.T) {42 func TestStreamUsageRewritesTheRequest(t *testing.T) {
43 req, _ := http.NewRequest(http.MethodPost, "http://x/v1/chat/completions", strings.NewReader(`{"stream":true}`))43 req, _ := http.NewRequest(http.MethodPost, "http://x/v1/chat/completions", strings.NewReader(`{"stream":true}`))
44 var seen string44 var seen string
45- _, _ = streamOptions(false)(req, func(r *http.Request) (*http.Response, error) {45+ _, _ = streamOptions(false, thinkingOffUnknown)(req, func(r *http.Request) (*http.Response, error) {
46 b, _ := io.ReadAll(r.Body)46 b, _ := io.ReadAll(r.Body)
47 seen = string(b)47 seen = string(b)
48 if r.ContentLength != int64(len(b)) {48 if r.ContentLength != int64(len(b)) {
modified main.go +2 -1
@@ -2,7 +2,8 @@
22 // `read_skill`, and (switchable) `read_file`, `write_file`, `edit_file`.
33 //
44 // LLM engine: any provider of internal/engine — Docker Model Runner by default,
5-// llama-server with `provider: llamacpp` — all through the OpenAI-compatible API.
5+// llama-server with `provider: llamacpp`, llmman with `provider: llmman`, Ollama
6+// with `provider: ollama` — all through the OpenAI-compatible API.
67 //
78 // The code is split into packages (internal/ directory):
89 // - config: every setting, with YAML overrides (see agent.yaml);
@@ -2,7 +2,8 @@
2 // `read_skill`, and (switchable) `read_file`, `write_file`, `edit_file`.2 // `read_skill`, and (switchable) `read_file`, `write_file`, `edit_file`.
3 //3 //
4 // LLM engine: any provider of internal/engine — Docker Model Runner by default,4 // LLM engine: any provider of internal/engine — Docker Model Runner by default,
5-// llama-server with `provider: llamacpp` — all through the OpenAI-compatible API.5+// llama-server with `provider: llamacpp`, llmman with `provider: llmman`, Ollama
6+// with `provider: ollama` — all through the OpenAI-compatible API.
6 //7 //
7 // The code is split into packages (internal/ directory):8 // The code is split into packages (internal/ directory):
8 // - config: every setting, with YAML overrides (see agent.yaml);9 // - config: every setting, with YAML overrides (see agent.yaml);