Add llmman and ollama providers #20
Merged@k33g wants to merge
feature/llmman-provider into mainMerged as
250d8d0121ef.30 files changed
added
.tickets/issues/0058-add-more-tools-to-the-sbx-template-redis-cli-to.yaml +13 -0 | new file mode 100644 | ||
| @@ -0,0 +1,13 @@ | ||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | |
| 2 | +id: 58 | |
| 3 | +title: Add more tools to the sbx template (redis, cli to parse source code ...) | |
| 4 | +state: open | |
| 5 | +author: | |
| 6 | + name: k33g | |
| 7 | + email: ph.charriere@gmail.com | |
| 8 | +createdAt: 2026-10-01T04:39:22.452Z | |
| 9 | +updatedAt: 2026-10-01T04:39:22.452Z | |
| 10 | +labels: [] | |
| 11 | +body: "" | |
| 12 | +tasks: [] | |
| 13 | +comments: [] | |
| new file mode 100644 | |||
| @@ -0,0 +1,13 @@ | |||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | ||
| 2 | +id: 58 | ||
| 3 | +title: Add more tools to the sbx template (redis, cli to parse source code ...) | ||
| 4 | +state: open | ||
| 5 | +author: | ||
| 6 | + name: k33g | ||
| 7 | + email: ph.charriere@gmail.com | ||
| 8 | +createdAt: 2026-10-01T04:39:22.452Z | ||
| 9 | +updatedAt: 2026-10-01T04:39:22.452Z | ||
| 10 | +labels: [] | ||
| 11 | +body: "" | ||
| 12 | +tasks: [] | ||
| 13 | +comments: [] | ||
added
.tickets/issues/0059-how-to-start-in-yolo-mode.yaml +106 -0 | new file mode 100644 | ||
| @@ -0,0 +1,106 @@ | ||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | |
| 2 | +id: 59 | |
| 3 | +title: How to start in yolo mode? | |
| 4 | +state: open | |
| 5 | +author: | |
| 6 | + name: k33g | |
| 7 | + email: ph.charriere@gmail.com | |
| 8 | +createdAt: 2026-10-01T04:39:53.878Z | |
| 9 | +updatedAt: 2026-10-01T06:30:00.000Z | |
| 10 | +labels: | |
| 11 | + - kind::feature | |
| 12 | + - priority::low | |
| 13 | +related: | |
| 14 | + - 57 | |
| 15 | +body: | | |
| 16 | + Question: is there a way to start mm in "yolo" mode (tools run without any | |
| 17 | + confirmation)? | |
| 18 | + | |
| 19 | + Answer as of 2026-10-01 (main 1d966fe): there is no --yolo flag and no | |
| 20 | + dedicated key, but it depends on the front end: | |
| 21 | + - Line mode (`mm agent.yaml`) and one-shot (`mm -p "…"`): never asks. | |
| 22 | + bash / write_file / edit_file run at once (no sink, no gate). | |
| 23 | + - Window mode (`mm -tui`): asks only with `permissions: ask`. Default is | |
| 24 | + `auto` = yolo. With `auto`, MCP tools marked `ask` do not ask either | |
| 25 | + (internal/tui/sink.go Allow returns true). | |
| 26 | + - Editor (`mm -acp`): ALWAYS asks through the editor's dialog | |
| 27 | + (internal/acp/sink.go Allow -> RequestPermission). No setting skips it; | |
| 28 | + only "allow always", remembered per tool for the session. | |
| 29 | + - MCP tools: `mcp.permissions` defaults to `ask` (also per server); set | |
| 30 | + `auto` to skip. | |
| 31 | + - `permissions` has no AGENT_* environment override: YAML only. | |
| 32 | + | |
| 33 | + Yolo in the window mode today: | |
| 34 | + permissions: auto | |
| 35 | + mcp: | |
| 36 | + permissions: auto | |
| 37 | + | |
| 38 | + Proposed feature (later): a real switch, `--yolo`, that forces `auto` | |
| 39 | + everywhere, ACP included (mm would approve without opening the editor's | |
| 40 | + dialog). Side benefit: no pending permission dialog, so ticket 57's | |
| 41 | + "watchdog kills the turn while a permission is pending" cannot happen. | |
| 42 | + | |
| 43 | + Open questions: | |
| 44 | + - Name: `--yolo` only, or also `AGENT_PERMISSIONS=auto` / a config key | |
| 45 | + (`permissions: yolo`?) so the sandbox kits can set it via env. | |
| 46 | + - ACP: should yolo be allowed there at all? The editor's dialog is the | |
| 47 | + main safety an editor adds; maybe require the flag explicitly (never | |
| 48 | + from config). | |
| 49 | + - Should yolo override a per-server MCP `permissions: ask`, or respect it? | |
| 50 | + - Make it visible: banner/status line "yolo" so nobody runs it by mistake. | |
| 51 | +tasks: | |
| 52 | + - id: 1 | |
| 53 | + title: "Decide the open questions below (flag name, ACP scope, MCP scope, banner)" | |
| 54 | + state: open | |
| 55 | + priority: none | |
| 56 | + author: | |
| 57 | + name: k33g | |
| 58 | + email: ph.charriere@gmail.com | |
| 59 | + createdAt: 2026-10-01T06:30:00.000Z | |
| 60 | + updatedAt: 2026-10-01T06:30:00.000Z | |
| 61 | + - id: 2 | |
| 62 | + title: "Failing tests first: with yolo on, window mode (permissions: ask) and ACP Allow() return true without asking; MCP ask-tools run" | |
| 63 | + state: open | |
| 64 | + priority: none | |
| 65 | + author: | |
| 66 | + name: k33g | |
| 67 | + email: ph.charriere@gmail.com | |
| 68 | + createdAt: 2026-10-01T06:30:00.000Z | |
| 69 | + updatedAt: 2026-10-01T06:30:00.000Z | |
| 70 | + - id: 3 | |
| 71 | + title: "Flag --yolo (main.go, alongside -p/-tui/-acp) + optional env/config override; forces Permissions=auto and mcp.permissions=auto, per-server MCP permissions included" | |
| 72 | + state: open | |
| 73 | + priority: none | |
| 74 | + author: | |
| 75 | + name: k33g | |
| 76 | + email: ph.charriere@gmail.com | |
| 77 | + createdAt: 2026-10-01T06:30:00.000Z | |
| 78 | + updatedAt: 2026-10-01T06:30:00.000Z | |
| 79 | + - id: 4 | |
| 80 | + title: "ACP sink Allow(): short-circuit when yolo is on (no RequestPermission sent to the editor)" | |
| 81 | + state: open | |
| 82 | + priority: none | |
| 83 | + author: | |
| 84 | + name: k33g | |
| 85 | + email: ph.charriere@gmail.com | |
| 86 | + createdAt: 2026-10-01T06:30:00.000Z | |
| 87 | + updatedAt: 2026-10-01T06:30:00.000Z | |
| 88 | + - id: 5 | |
| 89 | + title: "Banner/status line shows that yolo is on (terminal, window, ACP agent info)" | |
| 90 | + state: open | |
| 91 | + priority: none | |
| 92 | + author: | |
| 93 | + name: k33g | |
| 94 | + email: ph.charriere@gmail.com | |
| 95 | + createdAt: 2026-10-01T06:30:00.000Z | |
| 96 | + updatedAt: 2026-10-01T06:30:00.000Z | |
| 97 | + - id: 6 | |
| 98 | + title: "Docs EN+FR: reference/cli.md, reference/configuration.md, how-to (window mode, editor); answer the question in the how-to" | |
| 99 | + state: open | |
| 100 | + priority: none | |
| 101 | + author: | |
| 102 | + name: k33g | |
| 103 | + email: ph.charriere@gmail.com | |
| 104 | + createdAt: 2026-10-01T06:30:00.000Z | |
| 105 | + updatedAt: 2026-10-01T06:30:00.000Z | |
| 106 | +comments: [] | |
| new file mode 100644 | |||
| @@ -0,0 +1,106 @@ | |||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | ||
| 2 | +id: 59 | ||
| 3 | +title: How to start in yolo mode? | ||
| 4 | +state: open | ||
| 5 | +author: | ||
| 6 | + name: k33g | ||
| 7 | + email: ph.charriere@gmail.com | ||
| 8 | +createdAt: 2026-10-01T04:39:53.878Z | ||
| 9 | +updatedAt: 2026-10-01T06:30:00.000Z | ||
| 10 | +labels: | ||
| 11 | + - kind::feature | ||
| 12 | + - priority::low | ||
| 13 | +related: | ||
| 14 | + - 57 | ||
| 15 | +body: | | ||
| 16 | + Question: is there a way to start mm in "yolo" mode (tools run without any | ||
| 17 | + confirmation)? | ||
| 18 | + | ||
| 19 | + Answer as of 2026-10-01 (main 1d966fe): there is no --yolo flag and no | ||
| 20 | + dedicated key, but it depends on the front end: | ||
| 21 | + - Line mode (`mm agent.yaml`) and one-shot (`mm -p "…"`): never asks. | ||
| 22 | + bash / write_file / edit_file run at once (no sink, no gate). | ||
| 23 | + - Window mode (`mm -tui`): asks only with `permissions: ask`. Default is | ||
| 24 | + `auto` = yolo. With `auto`, MCP tools marked `ask` do not ask either | ||
| 25 | + (internal/tui/sink.go Allow returns true). | ||
| 26 | + - Editor (`mm -acp`): ALWAYS asks through the editor's dialog | ||
| 27 | + (internal/acp/sink.go Allow -> RequestPermission). No setting skips it; | ||
| 28 | + only "allow always", remembered per tool for the session. | ||
| 29 | + - MCP tools: `mcp.permissions` defaults to `ask` (also per server); set | ||
| 30 | + `auto` to skip. | ||
| 31 | + - `permissions` has no AGENT_* environment override: YAML only. | ||
| 32 | + | ||
| 33 | + Yolo in the window mode today: | ||
| 34 | + permissions: auto | ||
| 35 | + mcp: | ||
| 36 | + permissions: auto | ||
| 37 | + | ||
| 38 | + Proposed feature (later): a real switch, `--yolo`, that forces `auto` | ||
| 39 | + everywhere, ACP included (mm would approve without opening the editor's | ||
| 40 | + dialog). Side benefit: no pending permission dialog, so ticket 57's | ||
| 41 | + "watchdog kills the turn while a permission is pending" cannot happen. | ||
| 42 | + | ||
| 43 | + Open questions: | ||
| 44 | + - Name: `--yolo` only, or also `AGENT_PERMISSIONS=auto` / a config key | ||
| 45 | + (`permissions: yolo`?) so the sandbox kits can set it via env. | ||
| 46 | + - ACP: should yolo be allowed there at all? The editor's dialog is the | ||
| 47 | + main safety an editor adds; maybe require the flag explicitly (never | ||
| 48 | + from config). | ||
| 49 | + - Should yolo override a per-server MCP `permissions: ask`, or respect it? | ||
| 50 | + - Make it visible: banner/status line "yolo" so nobody runs it by mistake. | ||
| 51 | +tasks: | ||
| 52 | + - id: 1 | ||
| 53 | + title: "Decide the open questions below (flag name, ACP scope, MCP scope, banner)" | ||
| 54 | + state: open | ||
| 55 | + priority: none | ||
| 56 | + author: | ||
| 57 | + name: k33g | ||
| 58 | + email: ph.charriere@gmail.com | ||
| 59 | + createdAt: 2026-10-01T06:30:00.000Z | ||
| 60 | + updatedAt: 2026-10-01T06:30:00.000Z | ||
| 61 | + - id: 2 | ||
| 62 | + title: "Failing tests first: with yolo on, window mode (permissions: ask) and ACP Allow() return true without asking; MCP ask-tools run" | ||
| 63 | + state: open | ||
| 64 | + priority: none | ||
| 65 | + author: | ||
| 66 | + name: k33g | ||
| 67 | + email: ph.charriere@gmail.com | ||
| 68 | + createdAt: 2026-10-01T06:30:00.000Z | ||
| 69 | + updatedAt: 2026-10-01T06:30:00.000Z | ||
| 70 | + - id: 3 | ||
| 71 | + title: "Flag --yolo (main.go, alongside -p/-tui/-acp) + optional env/config override; forces Permissions=auto and mcp.permissions=auto, per-server MCP permissions included" | ||
| 72 | + state: open | ||
| 73 | + priority: none | ||
| 74 | + author: | ||
| 75 | + name: k33g | ||
| 76 | + email: ph.charriere@gmail.com | ||
| 77 | + createdAt: 2026-10-01T06:30:00.000Z | ||
| 78 | + updatedAt: 2026-10-01T06:30:00.000Z | ||
| 79 | + - id: 4 | ||
| 80 | + title: "ACP sink Allow(): short-circuit when yolo is on (no RequestPermission sent to the editor)" | ||
| 81 | + state: open | ||
| 82 | + priority: none | ||
| 83 | + author: | ||
| 84 | + name: k33g | ||
| 85 | + email: ph.charriere@gmail.com | ||
| 86 | + createdAt: 2026-10-01T06:30:00.000Z | ||
| 87 | + updatedAt: 2026-10-01T06:30:00.000Z | ||
| 88 | + - id: 5 | ||
| 89 | + title: "Banner/status line shows that yolo is on (terminal, window, ACP agent info)" | ||
| 90 | + state: open | ||
| 91 | + priority: none | ||
| 92 | + author: | ||
| 93 | + name: k33g | ||
| 94 | + email: ph.charriere@gmail.com | ||
| 95 | + createdAt: 2026-10-01T06:30:00.000Z | ||
| 96 | + updatedAt: 2026-10-01T06:30:00.000Z | ||
| 97 | + - id: 6 | ||
| 98 | + title: "Docs EN+FR: reference/cli.md, reference/configuration.md, how-to (window mode, editor); answer the question in the how-to" | ||
| 99 | + state: open | ||
| 100 | + priority: none | ||
| 101 | + author: | ||
| 102 | + name: k33g | ||
| 103 | + email: ph.charriere@gmail.com | ||
| 104 | + createdAt: 2026-10-01T06:30:00.000Z | ||
| 105 | + updatedAt: 2026-10-01T06:30:00.000Z | ||
| 106 | +comments: [] | ||
added
.tickets/issues/0060-add-llmman-provider.yaml +121 -0 | new file mode 100644 | ||
| @@ -0,0 +1,121 @@ | ||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | |
| 2 | +id: 60 | |
| 3 | +title: Add llmman provider | |
| 4 | +state: open | |
| 5 | +author: | |
| 6 | + name: k33g | |
| 7 | + email: ph.charriere@gmail.com | |
| 8 | +createdAt: 2026-10-01T04:48:02.829Z | |
| 9 | +updatedAt: 2026-10-01T07:00:00.000Z | |
| 10 | +labels: | |
| 11 | + - kind::feature | |
| 12 | + - priority::medium | |
| 13 | +related: | |
| 14 | + - 61 | |
| 15 | + - 62 | |
| 16 | +body: | | |
| 17 | + Add llmman (https://github.com/llmmanorg/llmman, `llmman serve`) as a | |
| 18 | + provider, OpenAI API only (/v1/chat/completions), and a demo config in | |
| 19 | + demo-llmman/ using hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M. | |
| 20 | +tasks: | |
| 21 | + - id: 1 | |
| 22 | + title: "Registry entry llmman (URL, fallback, key, progress, thinking off)" | |
| 23 | + state: closed | |
| 24 | + priority: none | |
| 25 | + author: | |
| 26 | + name: k33g | |
| 27 | + email: ph.charriere@gmail.com | |
| 28 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 29 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 30 | + - id: 2 | |
| 31 | + title: "Probe: /v1/models warning + served window from /api/ps (not the stub /props)" | |
| 32 | + state: closed | |
| 33 | + priority: none | |
| 34 | + author: | |
| 35 | + name: k33g | |
| 36 | + email: ph.charriere@gmail.com | |
| 37 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 38 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 39 | + - id: 3 | |
| 40 | + title: "Tests: Resolve rows + TestProbeLlmman" | |
| 41 | + state: closed | |
| 42 | + priority: none | |
| 43 | + author: | |
| 44 | + name: k33g | |
| 45 | + email: ph.charriere@gmail.com | |
| 46 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 47 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 48 | + - id: 4 | |
| 49 | + title: "demo-llmman/agent.llmman.yaml with hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M" | |
| 50 | + state: closed | |
| 51 | + priority: none | |
| 52 | + author: | |
| 53 | + name: k33g | |
| 54 | + email: ph.charriere@gmail.com | |
| 55 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 56 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 57 | + - id: 5 | |
| 58 | + title: "Docs EN+FR (how-to, reference, explanation, indexes)" | |
| 59 | + state: closed | |
| 60 | + priority: none | |
| 61 | + author: | |
| 62 | + name: k33g | |
| 63 | + email: ph.charriere@gmail.com | |
| 64 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 65 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 66 | + - id: 6 | |
| 67 | + title: "Live check against llmman serve (tool call, ctx, compaction)" | |
| 68 | + state: closed | |
| 69 | + priority: none | |
| 70 | + author: | |
| 71 | + name: k33g | |
| 72 | + email: ph.charriere@gmail.com | |
| 73 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 74 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 75 | + - id: 7 | |
| 76 | + title: "Quality gate (qlty) where available" | |
| 77 | + state: open | |
| 78 | + priority: none | |
| 79 | + author: | |
| 80 | + name: k33g | |
| 81 | + email: ph.charriere@gmail.com | |
| 82 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 83 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 84 | + - id: 8 | |
| 85 | + title: "Review, commit, merge" | |
| 86 | + state: open | |
| 87 | + priority: none | |
| 88 | + author: | |
| 89 | + name: k33g | |
| 90 | + email: ph.charriere@gmail.com | |
| 91 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 92 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 93 | +comments: | |
| 94 | + - id: 1 | |
| 95 | + author: | |
| 96 | + name: claude | |
| 97 | + email: noreply@anthropic.com | |
| 98 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 99 | + body: | | |
| 100 | + **Implemented** on branch `feature/llmman-provider` (uncommitted, 2026-10-01). | |
| 101 | + | |
| 102 | + - Registry entry `llmman` (internal/engine/openai_compat.go): base | |
| 103 | + `http://127.0.0.1:17434/v1`, fallback `http://host.docker.internal:17434/v1`, | |
| 104 | + key `LLMMAN_API_KEY` (optional), `progress: true`, | |
| 105 | + `thinkingOff: thinkingOffTemplate`. | |
| 106 | + - Probe `probeLlmman` (internal/engine/llmman.go): /v1/models (a missing | |
| 107 | + model is a warning: llmman pulls it on the first request) then the served | |
| 108 | + window from /api/ps (shared reading in apips.go, also used by Ollama #62). | |
| 109 | + - Measured on llmman 0.1.498: `/props` is a stub (n_ctx 4096, empty | |
| 110 | + chat_template, loaded or not) -> NOT used. prompt_progress, | |
| 111 | + reasoning_content, usage and `chat_template_kwargs.enable_thinking:false` | |
| 112 | + pass through to its llama-server. /api/show's trained context rejected | |
| 113 | + (LLMMAN_CONTEXT_LENGTH can serve less). | |
| 114 | + - Verified live: one-shot prompts (bash tool call, read_skill, reasoning | |
| 115 | + shown), REPL banner `ctx: 262144 (/api/ps)`, a real compaction | |
| 116 | + (`compressed 8 messages -> 1 summary + 2 kept`). | |
| 117 | + - Tests: Resolve rows, TestProbeLlmman; `go test ./...` green. | |
| 118 | + - Docs EN+FR: how-to/run-with-llmman.md, reference (configuration, cli, | |
| 119 | + mcp-configuration EN), explanation/providers.md, README indexes. | |
| 120 | + - Not done: automatic download with progress -> ticket 61. Quality gate (qlty) | |
| 121 | + not run: absent from the sandbox. | |
| new file mode 100644 | |||
| @@ -0,0 +1,121 @@ | |||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | ||
| 2 | +id: 60 | ||
| 3 | +title: Add llmman provider | ||
| 4 | +state: open | ||
| 5 | +author: | ||
| 6 | + name: k33g | ||
| 7 | + email: ph.charriere@gmail.com | ||
| 8 | +createdAt: 2026-10-01T04:48:02.829Z | ||
| 9 | +updatedAt: 2026-10-01T07:00:00.000Z | ||
| 10 | +labels: | ||
| 11 | + - kind::feature | ||
| 12 | + - priority::medium | ||
| 13 | +related: | ||
| 14 | + - 61 | ||
| 15 | + - 62 | ||
| 16 | +body: | | ||
| 17 | + Add llmman (https://github.com/llmmanorg/llmman, `llmman serve`) as a | ||
| 18 | + provider, OpenAI API only (/v1/chat/completions), and a demo config in | ||
| 19 | + demo-llmman/ using hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M. | ||
| 20 | +tasks: | ||
| 21 | + - id: 1 | ||
| 22 | + title: "Registry entry llmman (URL, fallback, key, progress, thinking off)" | ||
| 23 | + state: closed | ||
| 24 | + priority: none | ||
| 25 | + author: | ||
| 26 | + name: k33g | ||
| 27 | + email: ph.charriere@gmail.com | ||
| 28 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 29 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 30 | + - id: 2 | ||
| 31 | + title: "Probe: /v1/models warning + served window from /api/ps (not the stub /props)" | ||
| 32 | + state: closed | ||
| 33 | + priority: none | ||
| 34 | + author: | ||
| 35 | + name: k33g | ||
| 36 | + email: ph.charriere@gmail.com | ||
| 37 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 38 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 39 | + - id: 3 | ||
| 40 | + title: "Tests: Resolve rows + TestProbeLlmman" | ||
| 41 | + state: closed | ||
| 42 | + priority: none | ||
| 43 | + author: | ||
| 44 | + name: k33g | ||
| 45 | + email: ph.charriere@gmail.com | ||
| 46 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 47 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 48 | + - id: 4 | ||
| 49 | + title: "demo-llmman/agent.llmman.yaml with hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M" | ||
| 50 | + state: closed | ||
| 51 | + priority: none | ||
| 52 | + author: | ||
| 53 | + name: k33g | ||
| 54 | + email: ph.charriere@gmail.com | ||
| 55 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 56 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 57 | + - id: 5 | ||
| 58 | + title: "Docs EN+FR (how-to, reference, explanation, indexes)" | ||
| 59 | + state: closed | ||
| 60 | + priority: none | ||
| 61 | + author: | ||
| 62 | + name: k33g | ||
| 63 | + email: ph.charriere@gmail.com | ||
| 64 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 65 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 66 | + - id: 6 | ||
| 67 | + title: "Live check against llmman serve (tool call, ctx, compaction)" | ||
| 68 | + state: closed | ||
| 69 | + priority: none | ||
| 70 | + author: | ||
| 71 | + name: k33g | ||
| 72 | + email: ph.charriere@gmail.com | ||
| 73 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 74 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 75 | + - id: 7 | ||
| 76 | + title: "Quality gate (qlty) where available" | ||
| 77 | + state: open | ||
| 78 | + priority: none | ||
| 79 | + author: | ||
| 80 | + name: k33g | ||
| 81 | + email: ph.charriere@gmail.com | ||
| 82 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 83 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 84 | + - id: 8 | ||
| 85 | + title: "Review, commit, merge" | ||
| 86 | + state: open | ||
| 87 | + priority: none | ||
| 88 | + author: | ||
| 89 | + name: k33g | ||
| 90 | + email: ph.charriere@gmail.com | ||
| 91 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 92 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 93 | +comments: | ||
| 94 | + - id: 1 | ||
| 95 | + author: | ||
| 96 | + name: claude | ||
| 97 | + email: noreply@anthropic.com | ||
| 98 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 99 | + body: | | ||
| 100 | + **Implemented** on branch `feature/llmman-provider` (uncommitted, 2026-10-01). | ||
| 101 | + | ||
| 102 | + - Registry entry `llmman` (internal/engine/openai_compat.go): base | ||
| 103 | + `http://127.0.0.1:17434/v1`, fallback `http://host.docker.internal:17434/v1`, | ||
| 104 | + key `LLMMAN_API_KEY` (optional), `progress: true`, | ||
| 105 | + `thinkingOff: thinkingOffTemplate`. | ||
| 106 | + - Probe `probeLlmman` (internal/engine/llmman.go): /v1/models (a missing | ||
| 107 | + model is a warning: llmman pulls it on the first request) then the served | ||
| 108 | + window from /api/ps (shared reading in apips.go, also used by Ollama #62). | ||
| 109 | + - Measured on llmman 0.1.498: `/props` is a stub (n_ctx 4096, empty | ||
| 110 | + chat_template, loaded or not) -> NOT used. prompt_progress, | ||
| 111 | + reasoning_content, usage and `chat_template_kwargs.enable_thinking:false` | ||
| 112 | + pass through to its llama-server. /api/show's trained context rejected | ||
| 113 | + (LLMMAN_CONTEXT_LENGTH can serve less). | ||
| 114 | + - Verified live: one-shot prompts (bash tool call, read_skill, reasoning | ||
| 115 | + shown), REPL banner `ctx: 262144 (/api/ps)`, a real compaction | ||
| 116 | + (`compressed 8 messages -> 1 summary + 2 kept`). | ||
| 117 | + - Tests: Resolve rows, TestProbeLlmman; `go test ./...` green. | ||
| 118 | + - Docs EN+FR: how-to/run-with-llmman.md, reference (configuration, cli, | ||
| 119 | + mcp-configuration EN), explanation/providers.md, README indexes. | ||
| 120 | + - Not done: automatic download with progress -> ticket 61. Quality gate (qlty) | ||
| 121 | + not run: absent from the sandbox. | ||
added
.tickets/issues/0061-llmman-provider-download-the-model-automatically.yaml +13 -0 | new file mode 100644 | ||
| @@ -0,0 +1,13 @@ | ||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | |
| 2 | +id: 61 | |
| 3 | +title: "llmman provider: download the model automatically if does not exist" | |
| 4 | +state: open | |
| 5 | +author: | |
| 6 | + name: k33g | |
| 7 | + email: ph.charriere@gmail.com | |
| 8 | +createdAt: 2026-10-01T05:48:44.919Z | |
| 9 | +updatedAt: 2026-10-01T05:48:44.919Z | |
| 10 | +labels: [] | |
| 11 | +body: "" | |
| 12 | +tasks: [] | |
| 13 | +comments: [] | |
| new file mode 100644 | |||
| @@ -0,0 +1,13 @@ | |||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | ||
| 2 | +id: 61 | ||
| 3 | +title: "llmman provider: download the model automatically if does not exist" | ||
| 4 | +state: open | ||
| 5 | +author: | ||
| 6 | + name: k33g | ||
| 7 | + email: ph.charriere@gmail.com | ||
| 8 | +createdAt: 2026-10-01T05:48:44.919Z | ||
| 9 | +updatedAt: 2026-10-01T05:48:44.919Z | ||
| 10 | +labels: [] | ||
| 11 | +body: "" | ||
| 12 | +tasks: [] | ||
| 13 | +comments: [] | ||
added
.tickets/issues/0062-add-ollama-provider.yaml +146 -0 | new file mode 100644 | ||
| @@ -0,0 +1,146 @@ | ||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | |
| 2 | +id: 62 | |
| 3 | +title: Add Ollama provider | |
| 4 | +state: open | |
| 5 | +author: | |
| 6 | + name: k33g | |
| 7 | + email: ph.charriere@gmail.com | |
| 8 | +createdAt: 2026-10-01T06:10:07.295Z | |
| 9 | +updatedAt: 2026-10-01T07:00:00.000Z | |
| 10 | +labels: | |
| 11 | + - kind::feature | |
| 12 | + - priority::medium | |
| 13 | +related: | |
| 14 | + - 60 | |
| 15 | +body: | | |
| 16 | + Add Ollama as a provider, OpenAI API only (/v1/chat/completions), and a demo | |
| 17 | + config in demo-ollama/ using qwen3.5:2b. Ollama, llmman, DMR and llama.cpp | |
| 18 | + look alike, but each keeps its own registry entry so that a difference found | |
| 19 | + later is one field to change. | |
| 20 | +tasks: | |
| 21 | + - id: 1 | |
| 22 | + title: "Measure Ollama's /v1 (reasoning field, progress, thinking switch, window, 404)" | |
| 23 | + state: closed | |
| 24 | + priority: none | |
| 25 | + author: | |
| 26 | + name: k33g | |
| 27 | + email: ph.charriere@gmail.com | |
| 28 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 29 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 30 | + - id: 2 | |
| 31 | + title: "Per-provider thinkingOff field; reasoning_effort none for Ollama" | |
| 32 | + state: closed | |
| 33 | + priority: none | |
| 34 | + author: | |
| 35 | + name: k33g | |
| 36 | + email: ph.charriere@gmail.com | |
| 37 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 38 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 39 | + - id: 3 | |
| 40 | + title: "Registry entry ollama + probeOllama (shared /api/ps reading with llmman)" | |
| 41 | + state: closed | |
| 42 | + priority: none | |
| 43 | + author: | |
| 44 | + name: k33g | |
| 45 | + email: ph.charriere@gmail.com | |
| 46 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 47 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 48 | + - id: 4 | |
| 49 | + title: "Tests: Resolve rows, TestProbeOllama, thinking-off middleware" | |
| 50 | + state: closed | |
| 51 | + priority: none | |
| 52 | + author: | |
| 53 | + name: k33g | |
| 54 | + email: ph.charriere@gmail.com | |
| 55 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 56 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 57 | + - id: 5 | |
| 58 | + title: "demo-ollama/agent.ollama.yaml with qwen3.5:2b" | |
| 59 | + state: closed | |
| 60 | + priority: none | |
| 61 | + author: | |
| 62 | + name: k33g | |
| 63 | + email: ph.charriere@gmail.com | |
| 64 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 65 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 66 | + - id: 6 | |
| 67 | + title: "Docs EN+FR (how-to, reference, explanation, indexes)" | |
| 68 | + state: closed | |
| 69 | + priority: none | |
| 70 | + author: | |
| 71 | + name: k33g | |
| 72 | + email: ph.charriere@gmail.com | |
| 73 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 74 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 75 | + - id: 7 | |
| 76 | + title: "Live check against Ollama (tool call, ctx, compaction)" | |
| 77 | + state: closed | |
| 78 | + priority: none | |
| 79 | + author: | |
| 80 | + name: k33g | |
| 81 | + email: ph.charriere@gmail.com | |
| 82 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 83 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 84 | + - id: 8 | |
| 85 | + title: "Check reasoning_effort none on a non-thinking model" | |
| 86 | + state: open | |
| 87 | + priority: none | |
| 88 | + author: | |
| 89 | + name: k33g | |
| 90 | + email: ph.charriere@gmail.com | |
| 91 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 92 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 93 | + - id: 9 | |
| 94 | + title: "Quality gate (qlty) where available" | |
| 95 | + state: open | |
| 96 | + priority: none | |
| 97 | + author: | |
| 98 | + name: k33g | |
| 99 | + email: ph.charriere@gmail.com | |
| 100 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 101 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 102 | + - id: 10 | |
| 103 | + title: "Review, commit, merge" | |
| 104 | + state: open | |
| 105 | + priority: none | |
| 106 | + author: | |
| 107 | + name: k33g | |
| 108 | + email: ph.charriere@gmail.com | |
| 109 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 110 | + updatedAt: 2026-10-01T07:00:00.000Z | |
| 111 | +comments: | |
| 112 | + - id: 1 | |
| 113 | + author: | |
| 114 | + name: claude | |
| 115 | + email: noreply@anthropic.com | |
| 116 | + createdAt: 2026-10-01T07:00:00.000Z | |
| 117 | + body: | | |
| 118 | + **Implemented** on branch `feature/llmman-provider` (same branch as #60, | |
| 119 | + uncommitted, 2026-10-01). | |
| 120 | + | |
| 121 | + Measured on Ollama 0.35.0 (qwen3.5:2b), three real differences: | |
| 122 | + 1. Reasoning streams as `delta.reasoning` (not `reasoning_content`): already | |
| 123 | + read by reasoning.go. | |
| 124 | + 2. No `prompt_progress` (`return_progress` ignored) -> `progress: false`; the | |
| 125 | + firstTokenTimeout has to cover the whole prompt and the model load. | |
| 126 | + 3. `chat_template_kwargs.enable_thinking:false` is IGNORED; | |
| 127 | + `reasoning_effort: "none"` is obeyed. Proved on a real compaction: with | |
| 128 | + chat_template_kwargs the summary reasoned until the 300-token limit and | |
| 129 | + came back empty; with reasoning_effort it succeeded. | |
| 130 | + -> new per-provider field `thinkingOff` (usage.go: thinkingOffTemplate for | |
| 131 | + dmr/llamacpp/llmman, thinkingOffEffort for ollama), replacing the old | |
| 132 | + "progress implies chat_template_kwargs" coupling. | |
| 133 | + | |
| 134 | + Also: /api/ps reports the served window like llmman (`context_length: 32768`), | |
| 135 | + shared helper apips.go; Ollama's OpenAI API does NOT pull a missing model | |
| 136 | + (404 `model '…' not found`) -> probe warning says `ollama pull`. | |
| 137 | + | |
| 138 | + - Registry entry `ollama`: base `http://localhost:11434/v1`, fallback | |
| 139 | + `http://host.docker.internal:11434/v1`, key `OLLAMA_API_KEY` (optional, | |
| 140 | + for ollama.com). | |
| 141 | + - Tests: Resolve rows, TestProbeOllama, TestStreamOptionsWithoutThinkingOllama; | |
| 142 | + the unknown-provider tests now use `vllm`. `go test ./...` green. | |
| 143 | + - Live: banner `ctx: 32768 (/api/ps)`, one-shot prompt with a bash tool call. | |
| 144 | + - Docs EN+FR: how-to/run-with-ollama.md, reference, explanation, READMEs. | |
| 145 | + - Not done: quality gate (qlty absent from the sandbox). Unverified: | |
| 146 | + `reasoning_effort: "none"` on a NON-thinking model (none installed). | |
| new file mode 100644 | |||
| @@ -0,0 +1,146 @@ | |||
| 1 | +# Managed by IssueSpec. Hand edits are welcome; keep the schema valid. | ||
| 2 | +id: 62 | ||
| 3 | +title: Add Ollama provider | ||
| 4 | +state: open | ||
| 5 | +author: | ||
| 6 | + name: k33g | ||
| 7 | + email: ph.charriere@gmail.com | ||
| 8 | +createdAt: 2026-10-01T06:10:07.295Z | ||
| 9 | +updatedAt: 2026-10-01T07:00:00.000Z | ||
| 10 | +labels: | ||
| 11 | + - kind::feature | ||
| 12 | + - priority::medium | ||
| 13 | +related: | ||
| 14 | + - 60 | ||
| 15 | +body: | | ||
| 16 | + Add Ollama as a provider, OpenAI API only (/v1/chat/completions), and a demo | ||
| 17 | + config in demo-ollama/ using qwen3.5:2b. Ollama, llmman, DMR and llama.cpp | ||
| 18 | + look alike, but each keeps its own registry entry so that a difference found | ||
| 19 | + later is one field to change. | ||
| 20 | +tasks: | ||
| 21 | + - id: 1 | ||
| 22 | + title: "Measure Ollama's /v1 (reasoning field, progress, thinking switch, window, 404)" | ||
| 23 | + state: closed | ||
| 24 | + priority: none | ||
| 25 | + author: | ||
| 26 | + name: k33g | ||
| 27 | + email: ph.charriere@gmail.com | ||
| 28 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 29 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 30 | + - id: 2 | ||
| 31 | + title: "Per-provider thinkingOff field; reasoning_effort none for Ollama" | ||
| 32 | + state: closed | ||
| 33 | + priority: none | ||
| 34 | + author: | ||
| 35 | + name: k33g | ||
| 36 | + email: ph.charriere@gmail.com | ||
| 37 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 38 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 39 | + - id: 3 | ||
| 40 | + title: "Registry entry ollama + probeOllama (shared /api/ps reading with llmman)" | ||
| 41 | + state: closed | ||
| 42 | + priority: none | ||
| 43 | + author: | ||
| 44 | + name: k33g | ||
| 45 | + email: ph.charriere@gmail.com | ||
| 46 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 47 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 48 | + - id: 4 | ||
| 49 | + title: "Tests: Resolve rows, TestProbeOllama, thinking-off middleware" | ||
| 50 | + state: closed | ||
| 51 | + priority: none | ||
| 52 | + author: | ||
| 53 | + name: k33g | ||
| 54 | + email: ph.charriere@gmail.com | ||
| 55 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 56 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 57 | + - id: 5 | ||
| 58 | + title: "demo-ollama/agent.ollama.yaml with qwen3.5:2b" | ||
| 59 | + state: closed | ||
| 60 | + priority: none | ||
| 61 | + author: | ||
| 62 | + name: k33g | ||
| 63 | + email: ph.charriere@gmail.com | ||
| 64 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 65 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 66 | + - id: 6 | ||
| 67 | + title: "Docs EN+FR (how-to, reference, explanation, indexes)" | ||
| 68 | + state: closed | ||
| 69 | + priority: none | ||
| 70 | + author: | ||
| 71 | + name: k33g | ||
| 72 | + email: ph.charriere@gmail.com | ||
| 73 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 74 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 75 | + - id: 7 | ||
| 76 | + title: "Live check against Ollama (tool call, ctx, compaction)" | ||
| 77 | + state: closed | ||
| 78 | + priority: none | ||
| 79 | + author: | ||
| 80 | + name: k33g | ||
| 81 | + email: ph.charriere@gmail.com | ||
| 82 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 83 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 84 | + - id: 8 | ||
| 85 | + title: "Check reasoning_effort none on a non-thinking model" | ||
| 86 | + state: open | ||
| 87 | + priority: none | ||
| 88 | + author: | ||
| 89 | + name: k33g | ||
| 90 | + email: ph.charriere@gmail.com | ||
| 91 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 92 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 93 | + - id: 9 | ||
| 94 | + title: "Quality gate (qlty) where available" | ||
| 95 | + state: open | ||
| 96 | + priority: none | ||
| 97 | + author: | ||
| 98 | + name: k33g | ||
| 99 | + email: ph.charriere@gmail.com | ||
| 100 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 101 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 102 | + - id: 10 | ||
| 103 | + title: "Review, commit, merge" | ||
| 104 | + state: open | ||
| 105 | + priority: none | ||
| 106 | + author: | ||
| 107 | + name: k33g | ||
| 108 | + email: ph.charriere@gmail.com | ||
| 109 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 110 | + updatedAt: 2026-10-01T07:00:00.000Z | ||
| 111 | +comments: | ||
| 112 | + - id: 1 | ||
| 113 | + author: | ||
| 114 | + name: claude | ||
| 115 | + email: noreply@anthropic.com | ||
| 116 | + createdAt: 2026-10-01T07:00:00.000Z | ||
| 117 | + body: | | ||
| 118 | + **Implemented** on branch `feature/llmman-provider` (same branch as #60, | ||
| 119 | + uncommitted, 2026-10-01). | ||
| 120 | + | ||
| 121 | + Measured on Ollama 0.35.0 (qwen3.5:2b), three real differences: | ||
| 122 | + 1. Reasoning streams as `delta.reasoning` (not `reasoning_content`): already | ||
| 123 | + read by reasoning.go. | ||
| 124 | + 2. No `prompt_progress` (`return_progress` ignored) -> `progress: false`; the | ||
| 125 | + firstTokenTimeout has to cover the whole prompt and the model load. | ||
| 126 | + 3. `chat_template_kwargs.enable_thinking:false` is IGNORED; | ||
| 127 | + `reasoning_effort: "none"` is obeyed. Proved on a real compaction: with | ||
| 128 | + chat_template_kwargs the summary reasoned until the 300-token limit and | ||
| 129 | + came back empty; with reasoning_effort it succeeded. | ||
| 130 | + -> new per-provider field `thinkingOff` (usage.go: thinkingOffTemplate for | ||
| 131 | + dmr/llamacpp/llmman, thinkingOffEffort for ollama), replacing the old | ||
| 132 | + "progress implies chat_template_kwargs" coupling. | ||
| 133 | + | ||
| 134 | + Also: /api/ps reports the served window like llmman (`context_length: 32768`), | ||
| 135 | + shared helper apips.go; Ollama's OpenAI API does NOT pull a missing model | ||
| 136 | + (404 `model '…' not found`) -> probe warning says `ollama pull`. | ||
| 137 | + | ||
| 138 | + - Registry entry `ollama`: base `http://localhost:11434/v1`, fallback | ||
| 139 | + `http://host.docker.internal:11434/v1`, key `OLLAMA_API_KEY` (optional, | ||
| 140 | + for ollama.com). | ||
| 141 | + - Tests: Resolve rows, TestProbeOllama, TestStreamOptionsWithoutThinkingOllama; | ||
| 142 | + the unknown-provider tests now use `vllm`. `go test ./...` green. | ||
| 143 | + - Live: banner `ctx: 32768 (/api/ps)`, one-shot prompt with a bash tool call. | ||
| 144 | + - Docs EN+FR: how-to/run-with-ollama.md, reference, explanation, READMEs. | ||
| 145 | + - Not done: quality gate (qlty absent from the sandbox). Unverified: | ||
| 146 | + `reasoning_effort: "none"` on a NON-thinking model (none installed). | ||
added
demo-llmman/agent.llmman.yaml +199 -0 | new file mode 100644 | ||
| @@ -0,0 +1,199 @@ | ||
| 1 | +# Agent settings — mini-me served by llmman (https://github.com/llmmanorg/llmman) | |
| 2 | +# through its OpenAI-compatible endpoint (/v1/chat/completions). | |
| 3 | +# | |
| 4 | +# llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M # optional: the first question pulls it otherwise | |
| 5 | +# llmman serve # 127.0.0.1:17434 | |
| 6 | +# | |
| 7 | +# llmman loads the model on the first request (in its own llama-server) and | |
| 8 | +# unloads it after five idle minutes. | |
| 9 | +provider: llmman | |
| 10 | + | |
| 11 | +# The model id exactly as `llmman list` / GET /v1/models spells it. | |
| 12 | +model: hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M | |
| 13 | + | |
| 14 | +# `llmman serve`'s default bind; the OpenAI routes sit under /v1. | |
| 15 | +baseUrl: http://127.0.0.1:17434/v1 | |
| 16 | + | |
| 17 | +# From a Docker sandbox, the daemon runs on the host. | |
| 18 | +fallback: http://host.docker.internal:17434/v1 | |
| 19 | + | |
| 20 | +# Only when the daemon was started with LLMMAN_API_KEYS (mandatory when it | |
| 21 | +# listens beyond loopback); unset otherwise. The key never goes in this file. | |
| 22 | +# apiKeyEnv: LLMMAN_API_KEY | |
| 23 | + | |
| 24 | +# The `bash` tool. | |
| 25 | +# `false` removes it: the model can still read and edit files (editTools) and load skills, | |
| 26 | +# but never runs a command. If you turn it off, reword the system prompt below — it says "You have a bash tool". | |
| 27 | +bashTool: true | |
| 28 | + | |
| 29 | +# The built-in file tools: read_file, write_file, edit_file. | |
| 30 | +editTools: true | |
| 31 | + | |
| 32 | +# The built-in plan tool for managing multi-step plans. | |
| 33 | +# Set to false to disable plan creation and tracking via the `plan` tool. | |
| 34 | +planTool: true | |
| 35 | + | |
| 36 | +# Language servers for the `lsp` tool (definition, references, hover, symbols, diagnostics, rename). | |
| 37 | +# The file extension picks the server. A server missing from the PATH is skipped with a warning. | |
| 38 | +# No server listed = no `lsp` tool. Examples (uncomment what you have installed): | |
| 39 | +# lsp: | |
| 40 | +# servers: | |
| 41 | +# - name: gopls | |
| 42 | +# command: gopls | |
| 43 | +# extensions: [".go"] | |
| 44 | +# - name: typescript | |
| 45 | +# command: typescript-language-server | |
| 46 | +# args: ["--stdio"] | |
| 47 | +# extensions: [".ts", ".tsx", ".js", ".jsx"] | |
| 48 | +# - name: python | |
| 49 | +# command: pyright-langserver | |
| 50 | +# args: ["--stdio"] | |
| 51 | +# extensions: [".py"] | |
| 52 | +# - name: rust | |
| 53 | +# command: rust-analyzer | |
| 54 | +# extensions: [".rs"] | |
| 55 | + | |
| 56 | +# How many tokens the server actually serves. | |
| 57 | +# 0 = ask llmman: /api/ps tells it once the model is loaded, so the window is | |
| 58 | +# learned before the second question. Set it here if you want it from the start | |
| 59 | +# — and to the same value as LLMMAN_CONTEXT_LENGTH if you set that variable. | |
| 60 | +contextWindow: 0 | |
| 61 | + | |
| 62 | +# Max number of characters a tool returns to the model. | |
| 63 | +# Context safeguard: beyond that, the output is truncated (beginning + end kept). | |
| 64 | +maxOutput: 16000 | |
| 65 | + | |
| 66 | +# Max number of model <-> tools round trips for a single question. | |
| 67 | +# Raise it for tasks that need long chains of commands. | |
| 68 | +maxTurns: 40 | |
| 69 | + | |
| 70 | +# Where the `read_skill` tool looks for its markdown procedures. | |
| 71 | +# The path is relative to the CURRENT DIRECTORY: run the agent from demo-llmman/. | |
| 72 | +skillsDir: ../skills | |
| 73 | + | |
| 74 | +# Persistent sessions, per project. | |
| 75 | +# When enabled, every conversation is saved as one JSON file under `dir`. | |
| 76 | +# `/sessions` lists them, `/resume` continues one; in an editor, "Reload Agent" replays the thread. | |
| 77 | +sessions: | |
| 78 | + enabled: true | |
| 79 | + dir: .mm/sessions | |
| 80 | + | |
| 81 | +# How many lines of a command's output are echoed to the terminal. | |
| 82 | +# 0 disables it. | |
| 83 | +previewLines: 20 | |
| 84 | + | |
| 85 | +displayCommands: true | |
| 86 | +# Show what a thinking model reasons about before it answers (dim italic in the | |
| 87 | +# terminal, its own style in the window, a thought chunk in an editor). | |
| 88 | +# The reasoning is always READ — it is what keeps the watchdog alive while the | |
| 89 | +# model thinks — this only says whether to show it. | |
| 90 | +showThinking: true | |
| 91 | + | |
| 92 | + | |
| 93 | +# The system prompt: what the agent is, and what it is allowed to do. | |
| 94 | +# This is the knob to play with — it is the shortest path to changing behaviour. | |
| 95 | +system: | | |
| 96 | + Your name is Bob. | |
| 97 | + You are a coding agent working in a terminal. | |
| 98 | + You have a "bash" tool to run shell commands. | |
| 99 | + Use it to explore files, run tests, inspect the repository, etc. | |
| 100 | + Chain several commands if needed, then answer clearly in English. | |
| 101 | + | |
| 102 | + A request often mixes things you answer from yourself ("say hello") with | |
| 103 | + things only a command can answer ("list the files"). Handle every part, in | |
| 104 | + the order asked, and run a command for each part that needs one. | |
| 105 | + Never state the contents of a file, the output of a command, or the state of | |
| 106 | + the repository unless a command in THIS answer returned it. What you did not | |
| 107 | + read, you do not know: run the command instead of recalling it. | |
| 108 | + | |
| 109 | + SKILLS | |
| 110 | + You have a second tool, `read_skill`. Its description lists the procedures | |
| 111 | + available for this project — one per kind of task. | |
| 112 | + | |
| 113 | + Any request to DO something to a Go project is a skill, not a shell command | |
| 114 | + you invent. Match the request against that list, call `read_skill` FIRST, | |
| 115 | + before any bash command, and then follow what it says step by step. | |
| 116 | + | |
| 117 | + FILE EDITING | |
| 118 | + You have three tools for files: `read_file`, `edit_file` and `write_file`. | |
| 119 | + They are how a file gets read and changed here: each change is exact, | |
| 120 | + checked before it is written, and comes back as a diff with line numbers. | |
| 121 | + bash is for running things — building, testing, listing, searching. | |
| 122 | + | |
| 123 | + - Read before you write: call `read_file` on the file (numbered=true when | |
| 124 | + you need line numbers). You cannot target text you have not seen; never | |
| 125 | + rely on what you think you remember about a file. | |
| 126 | + - To change an existing file, call `edit_file` with one or more {old, new} | |
| 127 | + pairs. `old` is copied from the file character for character — same | |
| 128 | + spaces, same indentation, same line breaks — and appears exactly once: | |
| 129 | + add the surrounding lines until it is unique. Several pairs are applied | |
| 130 | + together, against the original file. An empty `new` deletes the text. | |
| 131 | + - Call `write_file` only to create a file, or to rewrite one entirely and | |
| 132 | + on purpose. On an existing file it replaces everything, including what | |
| 133 | + you did not intend to touch. | |
| 134 | + - Read the diff the tool returns: it says exactly what changed and on which | |
| 135 | + line. If `edit_file` refuses — text not found, ambiguous, overlapping | |
| 136 | + edits — read the file again and fix `old`. Do not fall back to | |
| 137 | + `write_file` to force the change through. | |
| 138 | + - After editing code, run the narrowest check with bash: the formatter, the | |
| 139 | + compiler, or the test covering that file. | |
| 140 | + | |
| 141 | + RULES | |
| 142 | + - Keep everything the file already does, unless the user asked to remove it. | |
| 143 | + - Touch only the files the request is about. Do not add tests, files or | |
| 144 | + features that were not asked for. | |
| 145 | + - Never run a git command unless the user says git, commit or push. | |
| 146 | + - Never move, rename or delete a file unless the user asked for it. | |
| 147 | + - Then answer in English, in a few lines. | |
| 148 | + - If you don't know how to use a <cli>, run `<cli> --help` (or `<cli> help`) | |
| 149 | + to understand the options, then run the command. | |
| 150 | + | |
| 151 | + BACKGROUND JOBS | |
| 152 | + Never let a command block the answer. Anything that serves, watches or runs | |
| 153 | + long goes to the background, with BOTH streams redirected and its pid kept: | |
| 154 | + | |
| 155 | + nohup <command> > /tmp/<job>.log 2>&1 & echo $! > /tmp/<job>.pid | |
| 156 | + | |
| 157 | + Redirecting only stdout still blocks until the process exits. Read the | |
| 158 | + `bg-jobs` skill before you wait on, inspect or stop such a job — each has a | |
| 159 | + rule you cannot guess. Stop every job you started before you finish, and say | |
| 160 | + which ones you left running. | |
| 161 | + | |
| 162 | +# Generation settings (OpenAI API keys). Low temperature for a coding agent: | |
| 163 | +# we want precise and reproducible answers, not creativity. | |
| 164 | +sampling: | |
| 165 | + temperature: 0.0 | |
| 166 | + parallel_tool_calls: false | |
| 167 | + top_p: 0.9 | |
| 168 | + max_tokens: 4096 | |
| 169 | + | |
| 170 | +# Silence allowed between two tokens once the answer has started. | |
| 171 | +watchdogTimeout: 30s | |
| 172 | + | |
| 173 | +# Silence allowed while the model reads the prompt, before the first token: | |
| 174 | +# on a laptop a 25k-token prompt can take minutes. llama-server reports its | |
| 175 | +# progress meanwhile, through llmman ("Reading the prompt 12.0k/25.3k tokens"), and each report | |
| 176 | +# restarts this clock, so it only has to cover one batch of the prompt. | |
| 177 | +firstTokenTimeout: 3m | |
| 178 | + | |
| 179 | +# Context compression. | |
| 180 | +# ON here: the agent learns the window from /api/ps, so `threshold` has a number to apply to. | |
| 181 | +context: | |
| 182 | + enabled: true | |
| 183 | + | |
| 184 | + # Compress when the history reaches this share of the window. | |
| 185 | + threshold: 75 | |
| 186 | + | |
| 187 | + # Fallback on the message count, for when the window is unknown. | |
| 188 | + # One command costs 2 messages (call + response), so 80 is roughly 30 commands of history. | |
| 189 | + maxMessages: 80 | |
| 190 | + | |
| 191 | + # How many recent questions (with their commands and answers) are kept as-is instead of being summarised. | |
| 192 | + # The model works on the last few turns, so summarising them would make it redo what it just did. | |
| 193 | + keepLastTurns: 3 | |
| 194 | + | |
| 195 | + # max_tokens of the summary request. Reasoning is turned off for it on this provider | |
| 196 | + # (a thinking model would otherwise spend the budget thinking and write nothing). | |
| 197 | + summaryMaxTokens: 1200 | |
| 198 | + | |
| 199 | + showStats: true | |
| new file mode 100644 | |||
| @@ -0,0 +1,199 @@ | |||
| 1 | +# Agent settings — mini-me served by llmman (https://github.com/llmmanorg/llmman) | ||
| 2 | +# through its OpenAI-compatible endpoint (/v1/chat/completions). | ||
| 3 | +# | ||
| 4 | +# llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M # optional: the first question pulls it otherwise | ||
| 5 | +# llmman serve # 127.0.0.1:17434 | ||
| 6 | +# | ||
| 7 | +# llmman loads the model on the first request (in its own llama-server) and | ||
| 8 | +# unloads it after five idle minutes. | ||
| 9 | +provider: llmman | ||
| 10 | + | ||
| 11 | +# The model id exactly as `llmman list` / GET /v1/models spells it. | ||
| 12 | +model: hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M | ||
| 13 | + | ||
| 14 | +# `llmman serve`'s default bind; the OpenAI routes sit under /v1. | ||
| 15 | +baseUrl: http://127.0.0.1:17434/v1 | ||
| 16 | + | ||
| 17 | +# From a Docker sandbox, the daemon runs on the host. | ||
| 18 | +fallback: http://host.docker.internal:17434/v1 | ||
| 19 | + | ||
| 20 | +# Only when the daemon was started with LLMMAN_API_KEYS (mandatory when it | ||
| 21 | +# listens beyond loopback); unset otherwise. The key never goes in this file. | ||
| 22 | +# apiKeyEnv: LLMMAN_API_KEY | ||
| 23 | + | ||
| 24 | +# The `bash` tool. | ||
| 25 | +# `false` removes it: the model can still read and edit files (editTools) and load skills, | ||
| 26 | +# but never runs a command. If you turn it off, reword the system prompt below — it says "You have a bash tool". | ||
| 27 | +bashTool: true | ||
| 28 | + | ||
| 29 | +# The built-in file tools: read_file, write_file, edit_file. | ||
| 30 | +editTools: true | ||
| 31 | + | ||
| 32 | +# The built-in plan tool for managing multi-step plans. | ||
| 33 | +# Set to false to disable plan creation and tracking via the `plan` tool. | ||
| 34 | +planTool: true | ||
| 35 | + | ||
| 36 | +# Language servers for the `lsp` tool (definition, references, hover, symbols, diagnostics, rename). | ||
| 37 | +# The file extension picks the server. A server missing from the PATH is skipped with a warning. | ||
| 38 | +# No server listed = no `lsp` tool. Examples (uncomment what you have installed): | ||
| 39 | +# lsp: | ||
| 40 | +# servers: | ||
| 41 | +# - name: gopls | ||
| 42 | +# command: gopls | ||
| 43 | +# extensions: [".go"] | ||
| 44 | +# - name: typescript | ||
| 45 | +# command: typescript-language-server | ||
| 46 | +# args: ["--stdio"] | ||
| 47 | +# extensions: [".ts", ".tsx", ".js", ".jsx"] | ||
| 48 | +# - name: python | ||
| 49 | +# command: pyright-langserver | ||
| 50 | +# args: ["--stdio"] | ||
| 51 | +# extensions: [".py"] | ||
| 52 | +# - name: rust | ||
| 53 | +# command: rust-analyzer | ||
| 54 | +# extensions: [".rs"] | ||
| 55 | + | ||
| 56 | +# How many tokens the server actually serves. | ||
| 57 | +# 0 = ask llmman: /api/ps tells it once the model is loaded, so the window is | ||
| 58 | +# learned before the second question. Set it here if you want it from the start | ||
| 59 | +# — and to the same value as LLMMAN_CONTEXT_LENGTH if you set that variable. | ||
| 60 | +contextWindow: 0 | ||
| 61 | + | ||
| 62 | +# Max number of characters a tool returns to the model. | ||
| 63 | +# Context safeguard: beyond that, the output is truncated (beginning + end kept). | ||
| 64 | +maxOutput: 16000 | ||
| 65 | + | ||
| 66 | +# Max number of model <-> tools round trips for a single question. | ||
| 67 | +# Raise it for tasks that need long chains of commands. | ||
| 68 | +maxTurns: 40 | ||
| 69 | + | ||
| 70 | +# Where the `read_skill` tool looks for its markdown procedures. | ||
| 71 | +# The path is relative to the CURRENT DIRECTORY: run the agent from demo-llmman/. | ||
| 72 | +skillsDir: ../skills | ||
| 73 | + | ||
| 74 | +# Persistent sessions, per project. | ||
| 75 | +# When enabled, every conversation is saved as one JSON file under `dir`. | ||
| 76 | +# `/sessions` lists them, `/resume` continues one; in an editor, "Reload Agent" replays the thread. | ||
| 77 | +sessions: | ||
| 78 | + enabled: true | ||
| 79 | + dir: .mm/sessions | ||
| 80 | + | ||
| 81 | +# How many lines of a command's output are echoed to the terminal. | ||
| 82 | +# 0 disables it. | ||
| 83 | +previewLines: 20 | ||
| 84 | + | ||
| 85 | +displayCommands: true | ||
| 86 | +# Show what a thinking model reasons about before it answers (dim italic in the | ||
| 87 | +# terminal, its own style in the window, a thought chunk in an editor). | ||
| 88 | +# The reasoning is always READ — it is what keeps the watchdog alive while the | ||
| 89 | +# model thinks — this only says whether to show it. | ||
| 90 | +showThinking: true | ||
| 91 | + | ||
| 92 | + | ||
| 93 | +# The system prompt: what the agent is, and what it is allowed to do. | ||
| 94 | +# This is the knob to play with — it is the shortest path to changing behaviour. | ||
| 95 | +system: | | ||
| 96 | + Your name is Bob. | ||
| 97 | + You are a coding agent working in a terminal. | ||
| 98 | + You have a "bash" tool to run shell commands. | ||
| 99 | + Use it to explore files, run tests, inspect the repository, etc. | ||
| 100 | + Chain several commands if needed, then answer clearly in English. | ||
| 101 | + | ||
| 102 | + A request often mixes things you answer from yourself ("say hello") with | ||
| 103 | + things only a command can answer ("list the files"). Handle every part, in | ||
| 104 | + the order asked, and run a command for each part that needs one. | ||
| 105 | + Never state the contents of a file, the output of a command, or the state of | ||
| 106 | + the repository unless a command in THIS answer returned it. What you did not | ||
| 107 | + read, you do not know: run the command instead of recalling it. | ||
| 108 | + | ||
| 109 | + SKILLS | ||
| 110 | + You have a second tool, `read_skill`. Its description lists the procedures | ||
| 111 | + available for this project — one per kind of task. | ||
| 112 | + | ||
| 113 | + Any request to DO something to a Go project is a skill, not a shell command | ||
| 114 | + you invent. Match the request against that list, call `read_skill` FIRST, | ||
| 115 | + before any bash command, and then follow what it says step by step. | ||
| 116 | + | ||
| 117 | + FILE EDITING | ||
| 118 | + You have three tools for files: `read_file`, `edit_file` and `write_file`. | ||
| 119 | + They are how a file gets read and changed here: each change is exact, | ||
| 120 | + checked before it is written, and comes back as a diff with line numbers. | ||
| 121 | + bash is for running things — building, testing, listing, searching. | ||
| 122 | + | ||
| 123 | + - Read before you write: call `read_file` on the file (numbered=true when | ||
| 124 | + you need line numbers). You cannot target text you have not seen; never | ||
| 125 | + rely on what you think you remember about a file. | ||
| 126 | + - To change an existing file, call `edit_file` with one or more {old, new} | ||
| 127 | + pairs. `old` is copied from the file character for character — same | ||
| 128 | + spaces, same indentation, same line breaks — and appears exactly once: | ||
| 129 | + add the surrounding lines until it is unique. Several pairs are applied | ||
| 130 | + together, against the original file. An empty `new` deletes the text. | ||
| 131 | + - Call `write_file` only to create a file, or to rewrite one entirely and | ||
| 132 | + on purpose. On an existing file it replaces everything, including what | ||
| 133 | + you did not intend to touch. | ||
| 134 | + - Read the diff the tool returns: it says exactly what changed and on which | ||
| 135 | + line. If `edit_file` refuses — text not found, ambiguous, overlapping | ||
| 136 | + edits — read the file again and fix `old`. Do not fall back to | ||
| 137 | + `write_file` to force the change through. | ||
| 138 | + - After editing code, run the narrowest check with bash: the formatter, the | ||
| 139 | + compiler, or the test covering that file. | ||
| 140 | + | ||
| 141 | + RULES | ||
| 142 | + - Keep everything the file already does, unless the user asked to remove it. | ||
| 143 | + - Touch only the files the request is about. Do not add tests, files or | ||
| 144 | + features that were not asked for. | ||
| 145 | + - Never run a git command unless the user says git, commit or push. | ||
| 146 | + - Never move, rename or delete a file unless the user asked for it. | ||
| 147 | + - Then answer in English, in a few lines. | ||
| 148 | + - If you don't know how to use a <cli>, run `<cli> --help` (or `<cli> help`) | ||
| 149 | + to understand the options, then run the command. | ||
| 150 | + | ||
| 151 | + BACKGROUND JOBS | ||
| 152 | + Never let a command block the answer. Anything that serves, watches or runs | ||
| 153 | + long goes to the background, with BOTH streams redirected and its pid kept: | ||
| 154 | + | ||
| 155 | + nohup <command> > /tmp/<job>.log 2>&1 & echo $! > /tmp/<job>.pid | ||
| 156 | + | ||
| 157 | + Redirecting only stdout still blocks until the process exits. Read the | ||
| 158 | + `bg-jobs` skill before you wait on, inspect or stop such a job — each has a | ||
| 159 | + rule you cannot guess. Stop every job you started before you finish, and say | ||
| 160 | + which ones you left running. | ||
| 161 | + | ||
| 162 | +# Generation settings (OpenAI API keys). Low temperature for a coding agent: | ||
| 163 | +# we want precise and reproducible answers, not creativity. | ||
| 164 | +sampling: | ||
| 165 | + temperature: 0.0 | ||
| 166 | + parallel_tool_calls: false | ||
| 167 | + top_p: 0.9 | ||
| 168 | + max_tokens: 4096 | ||
| 169 | + | ||
| 170 | +# Silence allowed between two tokens once the answer has started. | ||
| 171 | +watchdogTimeout: 30s | ||
| 172 | + | ||
| 173 | +# Silence allowed while the model reads the prompt, before the first token: | ||
| 174 | +# on a laptop a 25k-token prompt can take minutes. llama-server reports its | ||
| 175 | +# progress meanwhile, through llmman ("Reading the prompt 12.0k/25.3k tokens"), and each report | ||
| 176 | +# restarts this clock, so it only has to cover one batch of the prompt. | ||
| 177 | +firstTokenTimeout: 3m | ||
| 178 | + | ||
| 179 | +# Context compression. | ||
| 180 | +# ON here: the agent learns the window from /api/ps, so `threshold` has a number to apply to. | ||
| 181 | +context: | ||
| 182 | + enabled: true | ||
| 183 | + | ||
| 184 | + # Compress when the history reaches this share of the window. | ||
| 185 | + threshold: 75 | ||
| 186 | + | ||
| 187 | + # Fallback on the message count, for when the window is unknown. | ||
| 188 | + # One command costs 2 messages (call + response), so 80 is roughly 30 commands of history. | ||
| 189 | + maxMessages: 80 | ||
| 190 | + | ||
| 191 | + # How many recent questions (with their commands and answers) are kept as-is instead of being summarised. | ||
| 192 | + # The model works on the last few turns, so summarising them would make it redo what it just did. | ||
| 193 | + keepLastTurns: 3 | ||
| 194 | + | ||
| 195 | + # max_tokens of the summary request. Reasoning is turned off for it on this provider | ||
| 196 | + # (a thinking model would otherwise spend the budget thinking and write nothing). | ||
| 197 | + summaryMaxTokens: 1200 | ||
| 198 | + | ||
| 199 | + showStats: true | ||
added
demo-ollama/agent.ollama.yaml +202 -0 Large diff collapsed to keep the page fast — load it when you need it.
modified
docs/en/README.md +3 -1 | @@ -1,6 +1,6 @@ | ||
| 1 | 1 | # mini-me — documentation (English) |
| 2 | 2 | |
| 3 | -`mm` is a coding agent that runs in your terminal — line by line, or in a single window with `mm -tui` — or inside an editor through the Agent Client Protocol. It talks to a local model server (Docker Model Runner or llama.cpp's `llama-server`) and works with a handful of built-in tools: `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — plus, when turned on, `lsp` and `fetch_content`. | |
| 3 | +`mm` is a coding agent that runs in your terminal — line by line, or in a single window with `mm -tui` — or inside an editor through the Agent Client Protocol. It talks to a local model server (Docker Model Runner, llama.cpp's `llama-server`, llmman or Ollama) and works with a handful of built-in tools: `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — plus, when turned on, `lsp` and `fetch_content`. | |
| 4 | 4 | |
| 5 | 5 | This documentation is organised along the four [Diátaxis](https://diataxis.fr) quadrants. Pick the one that matches what you need right now. |
| 6 | 6 | |
| @@ -11,6 +11,8 @@ This documentation is organised along the four [Diátaxis](https://diataxis.fr) | ||
| 11 | 11 | ## How-to guides — get a task done |
| 12 | 12 | |
| 13 | 13 | - [Run mini-me on llama.cpp instead of Docker Model Runner](how-to/run-with-llama-cpp.md) |
| 14 | +- [Run mini-me on llmman](how-to/run-with-llmman.md) — `provider: llmman`, the demo config in `demo-llmman/` | |
| 15 | +- [Run mini-me on Ollama](how-to/run-with-ollama.md) — `provider: ollama`, the demo config in `demo-ollama/` | |
| 14 | 16 | - [Use mini-me from an editor (ACP)](how-to/use-from-an-editor.md) |
| 15 | 17 | - [Run mini-me in a Docker Sandbox](how-to/run-in-a-docker-sandbox.md) — the template and the two kits under `docker-sbx/`, window mode and editor |
| 16 | 18 | - [Run mini-me with the v3 sandbox kits](how-to/run-with-sandbox-kits-v3.md) — the same two kits for the kit specification v3, under `docker-sbx-v3/` |
| @@ -1,6 +1,6 @@ | |||
| 1 | # mini-me — documentation (English) | 1 | # mini-me — documentation (English) |
| 2 | 2 | ||
| 3 | -`mm` is a coding agent that runs in your terminal — line by line, or in a single window with `mm -tui` — or inside an editor through the Agent Client Protocol. It talks to a local model server (Docker Model Runner or llama.cpp's `llama-server`) and works with a handful of built-in tools: `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — plus, when turned on, `lsp` and `fetch_content`. | 3 | +`mm` is a coding agent that runs in your terminal — line by line, or in a single window with `mm -tui` — or inside an editor through the Agent Client Protocol. It talks to a local model server (Docker Model Runner, llama.cpp's `llama-server`, llmman or Ollama) and works with a handful of built-in tools: `bash`, `read_skill`, `read_file`, `write_file`, `edit_file` — plus, when turned on, `lsp` and `fetch_content`. |
| 4 | 4 | ||
| 5 | This documentation is organised along the four [Diátaxis](https://diataxis.fr) quadrants. Pick the one that matches what you need right now. | 5 | This documentation is organised along the four [Diátaxis](https://diataxis.fr) quadrants. Pick the one that matches what you need right now. |
| 6 | 6 | ||
| @@ -11,6 +11,8 @@ This documentation is organised along the four [Diátaxis](https://diataxis.fr) | |||
| 11 | ## How-to guides — get a task done | 11 | ## How-to guides — get a task done |
| 12 | 12 | ||
| 13 | - [Run mini-me on llama.cpp instead of Docker Model Runner](how-to/run-with-llama-cpp.md) | 13 | - [Run mini-me on llama.cpp instead of Docker Model Runner](how-to/run-with-llama-cpp.md) |
| 14 | +- [Run mini-me on llmman](how-to/run-with-llmman.md) — `provider: llmman`, the demo config in `demo-llmman/` | ||
| 15 | +- [Run mini-me on Ollama](how-to/run-with-ollama.md) — `provider: ollama`, the demo config in `demo-ollama/` | ||
| 14 | - [Use mini-me from an editor (ACP)](how-to/use-from-an-editor.md) | 16 | - [Use mini-me from an editor (ACP)](how-to/use-from-an-editor.md) |
| 15 | - [Run mini-me in a Docker Sandbox](how-to/run-in-a-docker-sandbox.md) — the template and the two kits under `docker-sbx/`, window mode and editor | 17 | - [Run mini-me in a Docker Sandbox](how-to/run-in-a-docker-sandbox.md) — the template and the two kits under `docker-sbx/`, window mode and editor |
| 16 | - [Run mini-me with the v3 sandbox kits](how-to/run-with-sandbox-kits-v3.md) — the same two kits for the kit specification v3, under `docker-sbx-v3/` | 18 | - [Run mini-me with the v3 sandbox kits](how-to/run-with-sandbox-kits-v3.md) — the same two kits for the kit specification v3, under `docker-sbx-v3/` |
modified
docs/en/explanation/providers.md +6 -2 | @@ -2,7 +2,7 @@ | ||
| 2 | 2 | |
| 3 | 3 | ## What is this about? |
| 4 | 4 | |
| 5 | -`mm` talks to a local model server through the OpenAI chat-completions API. Two servers are supported out of the box, Docker Model Runner and llama.cpp's `llama-server`, under the names `dmr` and `llamacpp`. The engine package used to be tied to Docker Model Runner; the provider abstraction is what generalised it. | |
| 5 | +`mm` talks to a local model server through the OpenAI chat-completions API. Four servers are supported out of the box: Docker Model Runner, llama.cpp's `llama-server`, llmman and Ollama, under the names `dmr`, `llamacpp`, `llmman` and `ollama`. The engine package used to be tied to Docker Model Runner; the provider abstraction is what generalised it. | |
| 6 | 6 | |
| 7 | 7 | ## Why it is designed this way |
| 8 | 8 | |
| @@ -16,6 +16,10 @@ | ||
| 16 | 16 | |
| 17 | 17 | **The probe is advice, never a failure.** On a demo machine the server is often started after the agent. The probe checks reachability, warns when the configured model is not in DMR's list, and on llama.cpp reads the served context size from `/props`, one level above `/v1`, because that endpoint reports what the server serves, not what the model was trained with. When the window is still unknown and compression is on, the REPL probes again before the first question. |
| 18 | 18 | |
| 19 | +**llmman is a third registry entry, with its own probe.** [llmman](https://github.com/llmmanorg/llmman) (`llmman serve`, port 17434) is a daemon in front of one `llama-server` per loaded model, and speaks several APIs; `mm` uses only its OpenAI one. Everything llama.cpp-specific passes through it — prompt progress, reasoning, `enable_thinking: false` for summaries — so it gets the same treatment as `llamacpp`. Its `/props`, though, is a stub (`n_ctx` 4096, no chat template, measured on llmman 0.1.498), so the probe reads the served window from `/api/ps` instead, for the configured model, once it is loaded. The trained context from `/api/show` is not used: `LLMMAN_CONTEXT_LENGTH` can serve less, and a window too large means compression that never triggers before the server refuses the prompt. A model missing from `/v1/models` is only a warning: llmman pulls it on the first request. | |
| 20 | + | |
| 21 | +**Ollama is a fourth entry, and shows why each server keeps its own.** Ollama's `/v1` looks like the others, but measured on Ollama 0.35.0 it differs in three ways. Reasoning streams as `delta.reasoning`, which the tap already reads. No prompt progress comes back, so `return_progress` is not sent. And `chat_template_kwargs` is ignored: the summary of a compaction reasoned until its token limit and came back empty; `reasoning_effort: "none"` is what Ollama obeys, and with it the same compaction succeeded. That is why how to turn reasoning off is a per-provider field, not something implied by "is llama.cpp". Like llmman, Ollama reports the served window on `/api/ps` once the model is loaded (the two share that reading); unlike llmman, it does not pull a missing model, so the probe's warning says `ollama pull`. | |
| 22 | + | |
| 19 | 23 | **Errors are rewritten in the server's own words.** A stack of wrapped messages is noise; "tool calls need llama-server started with --jinja" is the whole diagnosis. The provider recognises connection refusals, the missing `--jinja` flag, 401/403, 404 with a pull or alias hint, 429 and 5xx. Unknown errors come back unchanged: a raw message beats a wrong hint. |
| 20 | 24 | |
| 21 | 25 | **A stream the server breaks is retried, and its words are kept.** llama.cpp reports a failure in the middle of a streamed answer as an SSE `error:` event (the 200 status is already sent). openai-go only reads `data:` lines, so the message used to vanish behind `unexpected end of JSON input`. The tap that reads the stream for reasoning also picks up those events, and the model call that broke is retried once — that call only: the tools that ran earlier in the turn are not replayed, and text the broken attempt already streamed is not taken back. If it breaks again, the error shows the server's own message (`the model server reported an error while generating: Failed to parse input at pos …`), or says the connection broke. Either way the history is kept: asking to continue picks the work up. |
| @@ -34,5 +38,5 @@ | ||
| 34 | 38 | |
| 35 | 39 | - The two configuration files that parametrise each provider are listed in the [configuration reference](../reference/configuration.md). |
| 36 | 40 | - The window the probe learns is what [context compression](context-compression.md) measures against. |
| 37 | -- Switching servers for one run is covered in [how to run on llama.cpp](../how-to/run-with-llama-cpp.md). | |
| 41 | +- Switching servers for one run is covered in [how to run on llama.cpp](../how-to/run-with-llama-cpp.md) [how to run on llmman](../how-to/run-with-llmman.md) and [how to run on Ollama](../how-to/run-with-ollama.md). | |
| 38 | 42 | - Checking the tool order and measuring the cache gain on your own server: [how to check that the tools go out in a stable order](../how-to/verify-tool-order.md). |
| @@ -2,7 +2,7 @@ | |||
| 2 | 2 | ||
| 3 | ## What is this about? | 3 | ## What is this about? |
| 4 | 4 | ||
| 5 | -`mm` talks to a local model server through the OpenAI chat-completions API. Two servers are supported out of the box, Docker Model Runner and llama.cpp's `llama-server`, under the names `dmr` and `llamacpp`. The engine package used to be tied to Docker Model Runner; the provider abstraction is what generalised it. | 5 | +`mm` talks to a local model server through the OpenAI chat-completions API. Four servers are supported out of the box: Docker Model Runner, llama.cpp's `llama-server`, llmman and Ollama, under the names `dmr`, `llamacpp`, `llmman` and `ollama`. The engine package used to be tied to Docker Model Runner; the provider abstraction is what generalised it. |
| 6 | 6 | ||
| 7 | ## Why it is designed this way | 7 | ## Why it is designed this way |
| 8 | 8 | ||
| @@ -16,6 +16,10 @@ | |||
| 16 | 16 | ||
| 17 | **The probe is advice, never a failure.** On a demo machine the server is often started after the agent. The probe checks reachability, warns when the configured model is not in DMR's list, and on llama.cpp reads the served context size from `/props`, one level above `/v1`, because that endpoint reports what the server serves, not what the model was trained with. When the window is still unknown and compression is on, the REPL probes again before the first question. | 17 | **The probe is advice, never a failure.** On a demo machine the server is often started after the agent. The probe checks reachability, warns when the configured model is not in DMR's list, and on llama.cpp reads the served context size from `/props`, one level above `/v1`, because that endpoint reports what the server serves, not what the model was trained with. When the window is still unknown and compression is on, the REPL probes again before the first question. |
| 18 | 18 | ||
| 19 | +**llmman is a third registry entry, with its own probe.** [llmman](https://github.com/llmmanorg/llmman) (`llmman serve`, port 17434) is a daemon in front of one `llama-server` per loaded model, and speaks several APIs; `mm` uses only its OpenAI one. Everything llama.cpp-specific passes through it — prompt progress, reasoning, `enable_thinking: false` for summaries — so it gets the same treatment as `llamacpp`. Its `/props`, though, is a stub (`n_ctx` 4096, no chat template, measured on llmman 0.1.498), so the probe reads the served window from `/api/ps` instead, for the configured model, once it is loaded. The trained context from `/api/show` is not used: `LLMMAN_CONTEXT_LENGTH` can serve less, and a window too large means compression that never triggers before the server refuses the prompt. A model missing from `/v1/models` is only a warning: llmman pulls it on the first request. | ||
| 20 | + | ||
| 21 | +**Ollama is a fourth entry, and shows why each server keeps its own.** Ollama's `/v1` looks like the others, but measured on Ollama 0.35.0 it differs in three ways. Reasoning streams as `delta.reasoning`, which the tap already reads. No prompt progress comes back, so `return_progress` is not sent. And `chat_template_kwargs` is ignored: the summary of a compaction reasoned until its token limit and came back empty; `reasoning_effort: "none"` is what Ollama obeys, and with it the same compaction succeeded. That is why how to turn reasoning off is a per-provider field, not something implied by "is llama.cpp". Like llmman, Ollama reports the served window on `/api/ps` once the model is loaded (the two share that reading); unlike llmman, it does not pull a missing model, so the probe's warning says `ollama pull`. | ||
| 22 | + | ||
| 19 | **Errors are rewritten in the server's own words.** A stack of wrapped messages is noise; "tool calls need llama-server started with --jinja" is the whole diagnosis. The provider recognises connection refusals, the missing `--jinja` flag, 401/403, 404 with a pull or alias hint, 429 and 5xx. Unknown errors come back unchanged: a raw message beats a wrong hint. | 23 | **Errors are rewritten in the server's own words.** A stack of wrapped messages is noise; "tool calls need llama-server started with --jinja" is the whole diagnosis. The provider recognises connection refusals, the missing `--jinja` flag, 401/403, 404 with a pull or alias hint, 429 and 5xx. Unknown errors come back unchanged: a raw message beats a wrong hint. |
| 20 | 24 | ||
| 21 | **A stream the server breaks is retried, and its words are kept.** llama.cpp reports a failure in the middle of a streamed answer as an SSE `error:` event (the 200 status is already sent). openai-go only reads `data:` lines, so the message used to vanish behind `unexpected end of JSON input`. The tap that reads the stream for reasoning also picks up those events, and the model call that broke is retried once — that call only: the tools that ran earlier in the turn are not replayed, and text the broken attempt already streamed is not taken back. If it breaks again, the error shows the server's own message (`the model server reported an error while generating: Failed to parse input at pos …`), or says the connection broke. Either way the history is kept: asking to continue picks the work up. | 25 | **A stream the server breaks is retried, and its words are kept.** llama.cpp reports a failure in the middle of a streamed answer as an SSE `error:` event (the 200 status is already sent). openai-go only reads `data:` lines, so the message used to vanish behind `unexpected end of JSON input`. The tap that reads the stream for reasoning also picks up those events, and the model call that broke is retried once — that call only: the tools that ran earlier in the turn are not replayed, and text the broken attempt already streamed is not taken back. If it breaks again, the error shows the server's own message (`the model server reported an error while generating: Failed to parse input at pos …`), or says the connection broke. Either way the history is kept: asking to continue picks the work up. |
| @@ -34,5 +38,5 @@ | |||
| 34 | 38 | ||
| 35 | - The two configuration files that parametrise each provider are listed in the [configuration reference](../reference/configuration.md). | 39 | - The two configuration files that parametrise each provider are listed in the [configuration reference](../reference/configuration.md). |
| 36 | - The window the probe learns is what [context compression](context-compression.md) measures against. | 40 | - The window the probe learns is what [context compression](context-compression.md) measures against. |
| 37 | -- Switching servers for one run is covered in [how to run on llama.cpp](../how-to/run-with-llama-cpp.md). | 41 | +- Switching servers for one run is covered in [how to run on llama.cpp](../how-to/run-with-llama-cpp.md) [how to run on llmman](../how-to/run-with-llmman.md) and [how to run on Ollama](../how-to/run-with-ollama.md). |
| 38 | - Checking the tool order and measuring the cache gain on your own server: [how to check that the tools go out in a stable order](../how-to/verify-tool-order.md). | 42 | - Checking the tool order and measuring the cache gain on your own server: [how to check that the tools go out in a stable order](../how-to/verify-tool-order.md). |
added
docs/en/how-to/run-with-llmman.md +39 -0 | new file mode 100644 | ||
| @@ -0,0 +1,39 @@ | ||
| 1 | +# How to run mini-me on llmman | |
| 2 | + | |
| 3 | +This guide shows how to point `mm` at [llmman](https://github.com/llmmanorg/llmman) through its OpenAI-compatible endpoint. It assumes llmman is installed and you know how to build `mm`. | |
| 4 | + | |
| 5 | +## Steps | |
| 6 | + | |
| 7 | +1. Start the daemon. It listens on `127.0.0.1:17434`: | |
| 8 | + | |
| 9 | + ```bash | |
| 10 | + llmman serve | |
| 11 | + ``` | |
| 12 | + | |
| 13 | +2. Optionally pull the model first. Otherwise llmman pulls it on the first question, and that question waits for the download: | |
| 14 | + | |
| 15 | + ```bash | |
| 16 | + llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M | |
| 17 | + ``` | |
| 18 | + | |
| 19 | +3. Run the demo configuration from its directory (its `skillsDir` is `../skills`): | |
| 20 | + | |
| 21 | + ```bash | |
| 22 | + cd demo-llmman | |
| 23 | + ../mm agent.llmman.yaml | |
| 24 | + ``` | |
| 25 | + | |
| 26 | +4. Check the banner. `provider: llmman` confirms the provider. `ctx: 262144 (/api/ps)` means the model was already loaded and `mm` read the window it is served with; `ctx: unknown` means it was not loaded yet — `mm` asks again before the next question and prints `ctx: … (/api/ps)` once it knows. | |
| 27 | + | |
| 28 | +## Variants | |
| 29 | + | |
| 30 | +- **Another model.** Set `model:` to the id exactly as `llmman list` (or `GET /v1/models`) spells it. | |
| 31 | +- **From a Docker sandbox.** Nothing to change: when `127.0.0.1:17434` does not answer, `mm` falls back to `http://host.docker.internal:17434/v1`, the daemon on the host. | |
| 32 | +- **Daemon started with `LLMMAN_API_KEYS`** (mandatory when it listens beyond loopback). Export one of the keys in `LLMMAN_API_KEY`, the provider's default variable, or name another variable under `apiKeyEnv`. | |
| 33 | +- **`LLMMAN_CONTEXT_LENGTH` set on the daemon.** `/api/ps` already reports it once the model is loaded; set `contextWindow` to the same value if you want compression to know it from the first question. | |
| 34 | +- **Keep your own `agent.yaml`.** `AGENT_PROVIDER=llmman ./mm` works as long as the file leaves `baseUrl` empty. | |
| 35 | + | |
| 36 | +## See also | |
| 37 | + | |
| 38 | +- [Providers: one implementation per protocol](../explanation/providers.md) — why llmman's window comes from `/api/ps`, not `/props`. | |
| 39 | +- [Configuration reference](../reference/configuration.md) — provider defaults. | |
| new file mode 100644 | |||
| @@ -0,0 +1,39 @@ | |||
| 1 | +# How to run mini-me on llmman | ||
| 2 | + | ||
| 3 | +This guide shows how to point `mm` at [llmman](https://github.com/llmmanorg/llmman) through its OpenAI-compatible endpoint. It assumes llmman is installed and you know how to build `mm`. | ||
| 4 | + | ||
| 5 | +## Steps | ||
| 6 | + | ||
| 7 | +1. Start the daemon. It listens on `127.0.0.1:17434`: | ||
| 8 | + | ||
| 9 | + ```bash | ||
| 10 | + llmman serve | ||
| 11 | + ``` | ||
| 12 | + | ||
| 13 | +2. Optionally pull the model first. Otherwise llmman pulls it on the first question, and that question waits for the download: | ||
| 14 | + | ||
| 15 | + ```bash | ||
| 16 | + llmman pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M | ||
| 17 | + ``` | ||
| 18 | + | ||
| 19 | +3. Run the demo configuration from its directory (its `skillsDir` is `../skills`): | ||
| 20 | + | ||
| 21 | + ```bash | ||
| 22 | + cd demo-llmman | ||
| 23 | + ../mm agent.llmman.yaml | ||
| 24 | + ``` | ||
| 25 | + | ||
| 26 | +4. Check the banner. `provider: llmman` confirms the provider. `ctx: 262144 (/api/ps)` means the model was already loaded and `mm` read the window it is served with; `ctx: unknown` means it was not loaded yet — `mm` asks again before the next question and prints `ctx: … (/api/ps)` once it knows. | ||
| 27 | + | ||
| 28 | +## Variants | ||
| 29 | + | ||
| 30 | +- **Another model.** Set `model:` to the id exactly as `llmman list` (or `GET /v1/models`) spells it. | ||
| 31 | +- **From a Docker sandbox.** Nothing to change: when `127.0.0.1:17434` does not answer, `mm` falls back to `http://host.docker.internal:17434/v1`, the daemon on the host. | ||
| 32 | +- **Daemon started with `LLMMAN_API_KEYS`** (mandatory when it listens beyond loopback). Export one of the keys in `LLMMAN_API_KEY`, the provider's default variable, or name another variable under `apiKeyEnv`. | ||
| 33 | +- **`LLMMAN_CONTEXT_LENGTH` set on the daemon.** `/api/ps` already reports it once the model is loaded; set `contextWindow` to the same value if you want compression to know it from the first question. | ||
| 34 | +- **Keep your own `agent.yaml`.** `AGENT_PROVIDER=llmman ./mm` works as long as the file leaves `baseUrl` empty. | ||
| 35 | + | ||
| 36 | +## See also | ||
| 37 | + | ||
| 38 | +- [Providers: one implementation per protocol](../explanation/providers.md) — why llmman's window comes from `/api/ps`, not `/props`. | ||
| 39 | +- [Configuration reference](../reference/configuration.md) — provider defaults. | ||
added
docs/en/how-to/run-with-ollama.md +40 -0 | new file mode 100644 | ||
| @@ -0,0 +1,40 @@ | ||
| 1 | +# How to run mini-me on Ollama | |
| 2 | + | |
| 3 | +This guide shows how to point `mm` at [Ollama](https://ollama.com) through its OpenAI-compatible endpoint. It assumes Ollama is installed and you know how to build `mm`. | |
| 4 | + | |
| 5 | +## Steps | |
| 6 | + | |
| 7 | +1. Pull the model. Ollama's OpenAI API does not pull a missing model: a question to it fails with `model '…' not found`: | |
| 8 | + | |
| 9 | + ```bash | |
| 10 | + ollama pull qwen3.5:2b | |
| 11 | + ``` | |
| 12 | + | |
| 13 | +2. Make sure the server runs on `localhost:11434` (the desktop app starts it; otherwise `ollama serve`). | |
| 14 | + | |
| 15 | +3. Run the demo configuration from its directory (its `skillsDir` is `../skills`): | |
| 16 | + | |
| 17 | + ```bash | |
| 18 | + cd demo-ollama | |
| 19 | + ../mm agent.ollama.yaml | |
| 20 | + ``` | |
| 21 | + | |
| 22 | +4. Check the banner. `provider: ollama` confirms the provider. `ctx: 32768 (/api/ps)` means the model was already loaded and `mm` read the window it is served with; `ctx: unknown` means it was not loaded yet — `mm` asks again before the next question. | |
| 23 | + | |
| 24 | +## Variants | |
| 25 | + | |
| 26 | +- **Another model.** Set `model:` to the name exactly as `ollama list` spells it (`qwen3.5:9b`, …). | |
| 27 | +- **From a Docker sandbox.** Nothing to change: when `localhost:11434` does not answer, `mm` falls back to `http://host.docker.internal:11434/v1`, the Ollama on the host. | |
| 28 | +- **A larger window.** The OpenAI API cannot set `num_ctx` per request: start Ollama with `OLLAMA_CONTEXT_LENGTH`, and put the same value in `contextWindow` if you want compression to know it from the first question. | |
| 29 | +- **ollama.com's API.** Set `baseUrl` to its OpenAI endpoint and export the key in `OLLAMA_API_KEY` (or name another variable under `apiKeyEnv`). | |
| 30 | +- **Keep your own `agent.yaml`.** `AGENT_PROVIDER=ollama ./mm` works as long as the file leaves `baseUrl` empty. | |
| 31 | + | |
| 32 | +## What differs from the llama.cpp servers | |
| 33 | + | |
| 34 | +- No `Reading the prompt …` line: Ollama sends no progress while it reads, so `firstTokenTimeout` has to cover the whole prompt and the model's loading. | |
| 35 | +- The summary of a compaction turns reasoning off with `reasoning_effort: "none"` — Ollama ignores `chat_template_kwargs`. | |
| 36 | + | |
| 37 | +## See also | |
| 38 | + | |
| 39 | +- [Providers: one implementation per protocol](../explanation/providers.md) — what was measured on Ollama, and why each server keeps its own entry. | |
| 40 | +- [Configuration reference](../reference/configuration.md) — provider defaults. | |
| new file mode 100644 | |||
| @@ -0,0 +1,40 @@ | |||
| 1 | +# How to run mini-me on Ollama | ||
| 2 | + | ||
| 3 | +This guide shows how to point `mm` at [Ollama](https://ollama.com) through its OpenAI-compatible endpoint. It assumes Ollama is installed and you know how to build `mm`. | ||
| 4 | + | ||
| 5 | +## Steps | ||
| 6 | + | ||
| 7 | +1. Pull the model. Ollama's OpenAI API does not pull a missing model: a question to it fails with `model '…' not found`: | ||
| 8 | + | ||
| 9 | + ```bash | ||
| 10 | + ollama pull qwen3.5:2b | ||
| 11 | + ``` | ||
| 12 | + | ||
| 13 | +2. Make sure the server runs on `localhost:11434` (the desktop app starts it; otherwise `ollama serve`). | ||
| 14 | + | ||
| 15 | +3. Run the demo configuration from its directory (its `skillsDir` is `../skills`): | ||
| 16 | + | ||
| 17 | + ```bash | ||
| 18 | + cd demo-ollama | ||
| 19 | + ../mm agent.ollama.yaml | ||
| 20 | + ``` | ||
| 21 | + | ||
| 22 | +4. Check the banner. `provider: ollama` confirms the provider. `ctx: 32768 (/api/ps)` means the model was already loaded and `mm` read the window it is served with; `ctx: unknown` means it was not loaded yet — `mm` asks again before the next question. | ||
| 23 | + | ||
| 24 | +## Variants | ||
| 25 | + | ||
| 26 | +- **Another model.** Set `model:` to the name exactly as `ollama list` spells it (`qwen3.5:9b`, …). | ||
| 27 | +- **From a Docker sandbox.** Nothing to change: when `localhost:11434` does not answer, `mm` falls back to `http://host.docker.internal:11434/v1`, the Ollama on the host. | ||
| 28 | +- **A larger window.** The OpenAI API cannot set `num_ctx` per request: start Ollama with `OLLAMA_CONTEXT_LENGTH`, and put the same value in `contextWindow` if you want compression to know it from the first question. | ||
| 29 | +- **ollama.com's API.** Set `baseUrl` to its OpenAI endpoint and export the key in `OLLAMA_API_KEY` (or name another variable under `apiKeyEnv`). | ||
| 30 | +- **Keep your own `agent.yaml`.** `AGENT_PROVIDER=ollama ./mm` works as long as the file leaves `baseUrl` empty. | ||
| 31 | + | ||
| 32 | +## What differs from the llama.cpp servers | ||
| 33 | + | ||
| 34 | +- No `Reading the prompt …` line: Ollama sends no progress while it reads, so `firstTokenTimeout` has to cover the whole prompt and the model's loading. | ||
| 35 | +- The summary of a compaction turns reasoning off with `reasoning_effort: "none"` — Ollama ignores `chat_template_kwargs`. | ||
| 36 | + | ||
| 37 | +## See also | ||
| 38 | + | ||
| 39 | +- [Providers: one implementation per protocol](../explanation/providers.md) — what was measured on Ollama, and why each server keeps its own entry. | ||
| 40 | +- [Configuration reference](../reference/configuration.md) — provider defaults. | ||
modified
docs/en/reference/cli.md +1 -1 | @@ -55,7 +55,7 @@ The first existing source wins. | ||
| 55 | 55 | | `AGENT_MODEL` | Overrides the `model` key. | |
| 56 | 56 | | `AGENT_BASE_URL` | Overrides `baseUrl` and disables the fallback probe. | |
| 57 | 57 | | `DMR_BASE_URL` | Same as `AGENT_BASE_URL`, for the `dmr` provider only; `AGENT_BASE_URL` wins when both are set. | |
| 58 | -| Variable named by `apiKeyEnv` | Holds the API key. Default name: none for `dmr`, `LLAMA_API_KEY` for `llamacpp`. | | |
| 58 | +| Variable named by `apiKeyEnv` | Holds the API key. Default name: none for `dmr`, `LLAMA_API_KEY` for `llamacpp`, `LLMMAN_API_KEY` for `llmman`, `OLLAMA_API_KEY` for `ollama`. | | |
| 59 | 59 | |
| 60 | 60 | ## Start-up output |
| 61 | 61 | |
| @@ -55,7 +55,7 @@ The first existing source wins. | |||
| 55 | | `AGENT_MODEL` | Overrides the `model` key. | | 55 | | `AGENT_MODEL` | Overrides the `model` key. | |
| 56 | | `AGENT_BASE_URL` | Overrides `baseUrl` and disables the fallback probe. | | 56 | | `AGENT_BASE_URL` | Overrides `baseUrl` and disables the fallback probe. | |
| 57 | | `DMR_BASE_URL` | Same as `AGENT_BASE_URL`, for the `dmr` provider only; `AGENT_BASE_URL` wins when both are set. | | 57 | | `DMR_BASE_URL` | Same as `AGENT_BASE_URL`, for the `dmr` provider only; `AGENT_BASE_URL` wins when both are set. | |
| 58 | -| Variable named by `apiKeyEnv` | Holds the API key. Default name: none for `dmr`, `LLAMA_API_KEY` for `llamacpp`. | | 58 | +| Variable named by `apiKeyEnv` | Holds the API key. Default name: none for `dmr`, `LLAMA_API_KEY` for `llamacpp`, `LLMMAN_API_KEY` for `llmman`, `OLLAMA_API_KEY` for `ollama`. | |
| 59 | 59 | ||
| 60 | ## Start-up output | 60 | ## Start-up output |
| 61 | 61 | ||
modified
docs/en/reference/configuration.md +6 -4 | @@ -6,7 +6,7 @@ | ||
| 6 | 6 | |
| 7 | 7 | | Key | Type | Default | Description | |
| 8 | 8 | |-----|------|---------|-------------| |
| 9 | -| `provider` | string | `dmr` | Registry key of the LLM server: `dmr` or `llamacpp`. Overridden by `AGENT_PROVIDER`. | | |
| 9 | +| `provider` | string | `dmr` | Registry key of the LLM server: `dmr`, `llamacpp`, `llmman` or `ollama`. Overridden by `AGENT_PROVIDER`. | | |
| 10 | 10 | | `model` | string | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Model identifier in the provider's naming. Overridden by `AGENT_MODEL`. | |
| 11 | 11 | | `baseUrl` | string | `""` (provider default) | OpenAI-compatible endpoint tried first. Overridden by `AGENT_BASE_URL`. | |
| 12 | 12 | | `fallback` | string or absent | absent (provider default) | Endpoint used when `baseUrl` does not answer `/models` within 2 s. Absent = provider default; `""` = no fallback. | |
| @@ -24,7 +24,7 @@ | ||
| 24 | 24 | | `fetchContentTool` | bool | `false` | Declare `fetch_content`, which fetches web pages as Markdown. Off by default: it is the only built-in tool that reaches the network. | |
| 25 | 25 | | `sampling` | map | `temperature: 0.0`, `top_p: 0.9`, `max_tokens: 4096` | Generation settings, OpenAI API key names, passed to the server. The shipped `agent.yaml` adds `parallel_tool_calls: false`. `max_tokens` also bounds one tool call: a `write_file` of a long document (or a thinking model's reasoning plus the call) can hit it, and the call is then cut off. mm hands it back to the model twice, asking for smaller pieces, then stops with `the model hit max_tokens (N) while writing a tool call`; raise the value if that happens. With streaming, mm also asks the server for token counts (`stream_options.include_usage`). | |
| 26 | 26 | | `watchdogTimeout` | duration | `20s` | Silence tolerated between two streamed tokens before the connection is declared hung. Doubled for summary requests. | |
| 27 | -| `firstTokenTimeout` | duration | `3m` | Silence tolerated before a model call's first token, while the server reads the prompt (every round trip of a turn has one). Never shorter than `watchdogTimeout`; must be >= 0. With a llama.cpp server (`dmr`, `llamacpp`) mm asks for `return_progress`: each progress report restarts this clock, and the spinner shows `Reading the prompt 12.0k/25.3k tokens`. On a timeout the watchdog says `no first token from the server after …` (before the first token) or `connection hang detected after …` (after). | | |
| 27 | +| `firstTokenTimeout` | duration | `3m` | Silence tolerated before a model call's first token, while the server reads the prompt (every round trip of a turn has one). Never shorter than `watchdogTimeout`; must be >= 0. With a llama.cpp server (`dmr`, `llamacpp`, `llmman` — whose local GGUF models run in llama-server) mm asks for `return_progress`: each progress report restarts this clock, and the spinner shows `Reading the prompt 12.0k/25.3k tokens`. Ollama sends no progress: this one wait covers the whole prompt (and the model's loading). On a timeout the watchdog says `no first token from the server after …` (before the first token) or `connection hang detected after …` (after). | | |
| 28 | 28 | | `context` | map | see below | Automatic history compression. | |
| 29 | 29 | | `sessions` | map | see below | Persistent, per-project sessions. | |
| 30 | 30 | | `acp` | map | see below | How `mm -acp` introduces itself to the editor. | |
| @@ -40,7 +40,7 @@ | ||
| 40 | 40 | | `threshold` | int | `75` | Percentage of the context window beyond which the history is compressed before the next question. Between 1 and 100. | |
| 41 | 41 | | `maxMessages` | int | `80` | Message count that triggers a compression when the window is unknown or the estimate is low. `0` disables. Must be ≥ 0. | |
| 42 | 42 | | `keepLastTurns` | int | `3` | Question turns kept raw at the end of the history. Must be ≥ 1. | |
| 43 | -| `summaryMaxTokens` | int | `1200` | `max_tokens` of the summary request. Must be > 0. A thinking model's reasoning counts against it: on `dmr` and `llamacpp` the summary request turns reasoning off (`chat_template_kwargs.enable_thinking: false`); elsewhere, if the error says the model "reasoned until the limit", raise it. | | |
| 43 | +| `summaryMaxTokens` | int | `1200` | `max_tokens` of the summary request. Must be > 0. A thinking model's reasoning counts against it: on `dmr`, `llamacpp` and `llmman` the summary request turns reasoning off (`chat_template_kwargs.enable_thinking: false`), on `ollama` too (`reasoning_effort: "none"`, the one switch Ollama obeys); elsewhere, if the error says the model "reasoned until the limit", raise it. | | |
| 44 | 44 | | `prompt` | string | `""` | Replaces the built-in summary prompt when not empty. | |
| 45 | 45 | | `showStats` | bool | `true` | Print the one-line `🗜️` report after each compression. | |
| 46 | 46 | |
| @@ -90,6 +90,8 @@ The shipped `agent.lsp.yaml` is `agent.dmr.yaml` with this block on (gopls activ | ||
| 90 | 90 | |----------|-----------|------------|-------------|--------------| |
| 91 | 91 | | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | none | no | |
| 92 | 92 | | `llamacpp` | `http://127.0.0.1:8080/v1` | none | `LLAMA_API_KEY` | no | |
| 93 | +| `llmman` | `http://127.0.0.1:17434/v1` | `http://host.docker.internal:17434/v1` | `LLMMAN_API_KEY` | no | | |
| 94 | +| `ollama` | `http://localhost:11434/v1` | `http://host.docker.internal:11434/v1` | `OLLAMA_API_KEY` | no | | |
| 93 | 95 | |
| 94 | 96 | ## Validation errors |
| 95 | 97 | |
| @@ -115,7 +117,7 @@ Reported as `[config error: <path>: <message>]`, exit code 1. | ||
| 115 | 117 | | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | an extension without its dot | |
| 116 | 118 | | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | two servers on one extension | |
| 117 | 119 | | `permissions must be "auto" or "ask"` | any other value | |
| 118 | -| `unknown provider "x" (known: dmr, llamacpp)` | Reported by the engine, as `[engine error: …]`. | | |
| 120 | +| `unknown provider "x" (known: dmr, llamacpp, llmman, ollama)` | Reported by the engine, as `[engine error: …]`. | | |
| 119 | 121 | |
| 120 | 122 | A start-up warning, not an error, is printed when `context.enabled` is `true`, the window is unknown and `context.maxMessages` is `0`. |
| 121 | 123 | |
| @@ -6,7 +6,7 @@ | |||
| 6 | 6 | ||
| 7 | | Key | Type | Default | Description | | 7 | | Key | Type | Default | Description | |
| 8 | |-----|------|---------|-------------| | 8 | |-----|------|---------|-------------| |
| 9 | -| `provider` | string | `dmr` | Registry key of the LLM server: `dmr` or `llamacpp`. Overridden by `AGENT_PROVIDER`. | | 9 | +| `provider` | string | `dmr` | Registry key of the LLM server: `dmr`, `llamacpp`, `llmman` or `ollama`. Overridden by `AGENT_PROVIDER`. | |
| 10 | | `model` | string | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Model identifier in the provider's naming. Overridden by `AGENT_MODEL`. | | 10 | | `model` | string | `huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M` | Model identifier in the provider's naming. Overridden by `AGENT_MODEL`. | |
| 11 | | `baseUrl` | string | `""` (provider default) | OpenAI-compatible endpoint tried first. Overridden by `AGENT_BASE_URL`. | | 11 | | `baseUrl` | string | `""` (provider default) | OpenAI-compatible endpoint tried first. Overridden by `AGENT_BASE_URL`. | |
| 12 | | `fallback` | string or absent | absent (provider default) | Endpoint used when `baseUrl` does not answer `/models` within 2 s. Absent = provider default; `""` = no fallback. | | 12 | | `fallback` | string or absent | absent (provider default) | Endpoint used when `baseUrl` does not answer `/models` within 2 s. Absent = provider default; `""` = no fallback. | |
| @@ -24,7 +24,7 @@ | |||
| 24 | | `fetchContentTool` | bool | `false` | Declare `fetch_content`, which fetches web pages as Markdown. Off by default: it is the only built-in tool that reaches the network. | | 24 | | `fetchContentTool` | bool | `false` | Declare `fetch_content`, which fetches web pages as Markdown. Off by default: it is the only built-in tool that reaches the network. | |
| 25 | | `sampling` | map | `temperature: 0.0`, `top_p: 0.9`, `max_tokens: 4096` | Generation settings, OpenAI API key names, passed to the server. The shipped `agent.yaml` adds `parallel_tool_calls: false`. `max_tokens` also bounds one tool call: a `write_file` of a long document (or a thinking model's reasoning plus the call) can hit it, and the call is then cut off. mm hands it back to the model twice, asking for smaller pieces, then stops with `the model hit max_tokens (N) while writing a tool call`; raise the value if that happens. With streaming, mm also asks the server for token counts (`stream_options.include_usage`). | | 25 | | `sampling` | map | `temperature: 0.0`, `top_p: 0.9`, `max_tokens: 4096` | Generation settings, OpenAI API key names, passed to the server. The shipped `agent.yaml` adds `parallel_tool_calls: false`. `max_tokens` also bounds one tool call: a `write_file` of a long document (or a thinking model's reasoning plus the call) can hit it, and the call is then cut off. mm hands it back to the model twice, asking for smaller pieces, then stops with `the model hit max_tokens (N) while writing a tool call`; raise the value if that happens. With streaming, mm also asks the server for token counts (`stream_options.include_usage`). | |
| 26 | | `watchdogTimeout` | duration | `20s` | Silence tolerated between two streamed tokens before the connection is declared hung. Doubled for summary requests. | | 26 | | `watchdogTimeout` | duration | `20s` | Silence tolerated between two streamed tokens before the connection is declared hung. Doubled for summary requests. | |
| 27 | -| `firstTokenTimeout` | duration | `3m` | Silence tolerated before a model call's first token, while the server reads the prompt (every round trip of a turn has one). Never shorter than `watchdogTimeout`; must be >= 0. With a llama.cpp server (`dmr`, `llamacpp`) mm asks for `return_progress`: each progress report restarts this clock, and the spinner shows `Reading the prompt 12.0k/25.3k tokens`. On a timeout the watchdog says `no first token from the server after …` (before the first token) or `connection hang detected after …` (after). | | 27 | +| `firstTokenTimeout` | duration | `3m` | Silence tolerated before a model call's first token, while the server reads the prompt (every round trip of a turn has one). Never shorter than `watchdogTimeout`; must be >= 0. With a llama.cpp server (`dmr`, `llamacpp`, `llmman` — whose local GGUF models run in llama-server) mm asks for `return_progress`: each progress report restarts this clock, and the spinner shows `Reading the prompt 12.0k/25.3k tokens`. Ollama sends no progress: this one wait covers the whole prompt (and the model's loading). On a timeout the watchdog says `no first token from the server after …` (before the first token) or `connection hang detected after …` (after). | |
| 28 | | `context` | map | see below | Automatic history compression. | | 28 | | `context` | map | see below | Automatic history compression. | |
| 29 | | `sessions` | map | see below | Persistent, per-project sessions. | | 29 | | `sessions` | map | see below | Persistent, per-project sessions. | |
| 30 | | `acp` | map | see below | How `mm -acp` introduces itself to the editor. | | 30 | | `acp` | map | see below | How `mm -acp` introduces itself to the editor. | |
| @@ -40,7 +40,7 @@ | |||
| 40 | | `threshold` | int | `75` | Percentage of the context window beyond which the history is compressed before the next question. Between 1 and 100. | | 40 | | `threshold` | int | `75` | Percentage of the context window beyond which the history is compressed before the next question. Between 1 and 100. | |
| 41 | | `maxMessages` | int | `80` | Message count that triggers a compression when the window is unknown or the estimate is low. `0` disables. Must be ≥ 0. | | 41 | | `maxMessages` | int | `80` | Message count that triggers a compression when the window is unknown or the estimate is low. `0` disables. Must be ≥ 0. | |
| 42 | | `keepLastTurns` | int | `3` | Question turns kept raw at the end of the history. Must be ≥ 1. | | 42 | | `keepLastTurns` | int | `3` | Question turns kept raw at the end of the history. Must be ≥ 1. | |
| 43 | -| `summaryMaxTokens` | int | `1200` | `max_tokens` of the summary request. Must be > 0. A thinking model's reasoning counts against it: on `dmr` and `llamacpp` the summary request turns reasoning off (`chat_template_kwargs.enable_thinking: false`); elsewhere, if the error says the model "reasoned until the limit", raise it. | | 43 | +| `summaryMaxTokens` | int | `1200` | `max_tokens` of the summary request. Must be > 0. A thinking model's reasoning counts against it: on `dmr`, `llamacpp` and `llmman` the summary request turns reasoning off (`chat_template_kwargs.enable_thinking: false`), on `ollama` too (`reasoning_effort: "none"`, the one switch Ollama obeys); elsewhere, if the error says the model "reasoned until the limit", raise it. | |
| 44 | | `prompt` | string | `""` | Replaces the built-in summary prompt when not empty. | | 44 | | `prompt` | string | `""` | Replaces the built-in summary prompt when not empty. | |
| 45 | | `showStats` | bool | `true` | Print the one-line `🗜️` report after each compression. | | 45 | | `showStats` | bool | `true` | Print the one-line `🗜️` report after each compression. | |
| 46 | 46 | ||
| @@ -90,6 +90,8 @@ The shipped `agent.lsp.yaml` is `agent.dmr.yaml` with this block on (gopls activ | |||
| 90 | |----------|-----------|------------|-------------|--------------| | 90 | |----------|-----------|------------|-------------|--------------| |
| 91 | | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | none | no | | 91 | | `dmr` | `http://localhost:12434/engines/v1` | `http://host.docker.internal:12434/engines/v1` | none | no | |
| 92 | | `llamacpp` | `http://127.0.0.1:8080/v1` | none | `LLAMA_API_KEY` | no | | 92 | | `llamacpp` | `http://127.0.0.1:8080/v1` | none | `LLAMA_API_KEY` | no | |
| 93 | +| `llmman` | `http://127.0.0.1:17434/v1` | `http://host.docker.internal:17434/v1` | `LLMMAN_API_KEY` | no | | ||
| 94 | +| `ollama` | `http://localhost:11434/v1` | `http://host.docker.internal:11434/v1` | `OLLAMA_API_KEY` | no | | ||
| 93 | 95 | ||
| 94 | ## Validation errors | 96 | ## Validation errors |
| 95 | 97 | ||
| @@ -115,7 +117,7 @@ Reported as `[config error: <path>: <message>]`, exit code 1. | |||
| 115 | | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | an extension without its dot | | 117 | | `lsp.servers[N].extensions: "go" must start with a dot, like ".go"` | an extension without its dot | |
| 116 | | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | two servers on one extension | | 118 | | `lsp.servers[N]: extension ".ts" is already served by "typescript"` | two servers on one extension | |
| 117 | | `permissions must be "auto" or "ask"` | any other value | | 119 | | `permissions must be "auto" or "ask"` | any other value | |
| 118 | -| `unknown provider "x" (known: dmr, llamacpp)` | Reported by the engine, as `[engine error: …]`. | | 120 | +| `unknown provider "x" (known: dmr, llamacpp, llmman, ollama)` | Reported by the engine, as `[engine error: …]`. | |
| 119 | 121 | ||
| 120 | A start-up warning, not an error, is printed when `context.enabled` is `true`, the window is unknown and `context.maxMessages` is `0`. | 122 | A start-up warning, not an error, is printed when `context.enabled` is `true`, the window is unknown and `context.maxMessages` is `0`. |
| 121 | 123 | ||
modified
docs/en/reference/mcp-configuration.md +4 -2 Large diff collapsed to keep the page fast — load it when you need it.
modified
docs/fr/README.md +3 -1 Large diff collapsed to keep the page fast — load it when you need it.
modified
docs/fr/explanation/providers.md +6 -2 Large diff collapsed to keep the page fast — load it when you need it.
added
docs/fr/how-to/run-with-llmman.md +39 -0 Large diff collapsed to keep the page fast — load it when you need it.
added
docs/fr/how-to/run-with-ollama.md +40 -0 Large diff collapsed to keep the page fast — load it when you need it.
modified
docs/fr/reference/cli.md +1 -1 | @@ -55,7 +55,7 @@ La première source existante gagne. | ||
| 55 | 55 | | `AGENT_MODEL` | Surcharge la clé `model`. | |
| 56 | 56 | | `AGENT_BASE_URL` | Surcharge `baseUrl` et désactive la sonde de repli. | |
| 57 | 57 | | `DMR_BASE_URL` | Comme `AGENT_BASE_URL`, pour le fournisseur `dmr` seulement ; `AGENT_BASE_URL` gagne quand les deux sont définies. | |
| 58 | -| Variable nommée par `apiKeyEnv` | Contient la clé d'API. Nom par défaut : aucun pour `dmr`, `LLAMA_API_KEY` pour `llamacpp`. | | |
| 58 | +| Variable nommée par `apiKeyEnv` | Contient la clé d'API. Nom par défaut : aucun pour `dmr`, `LLAMA_API_KEY` pour `llamacpp`, `LLMMAN_API_KEY` pour `llmman`, `OLLAMA_API_KEY` pour `ollama`. | | |
| 59 | 59 | |
| 60 | 60 | ## Sortie de démarrage |
| 61 | 61 | |
| @@ -55,7 +55,7 @@ La première source existante gagne. | |||
| 55 | | `AGENT_MODEL` | Surcharge la clé `model`. | | 55 | | `AGENT_MODEL` | Surcharge la clé `model`. | |
| 56 | | `AGENT_BASE_URL` | Surcharge `baseUrl` et désactive la sonde de repli. | | 56 | | `AGENT_BASE_URL` | Surcharge `baseUrl` et désactive la sonde de repli. | |
| 57 | | `DMR_BASE_URL` | Comme `AGENT_BASE_URL`, pour le fournisseur `dmr` seulement ; `AGENT_BASE_URL` gagne quand les deux sont définies. | | 57 | | `DMR_BASE_URL` | Comme `AGENT_BASE_URL`, pour le fournisseur `dmr` seulement ; `AGENT_BASE_URL` gagne quand les deux sont définies. | |
| 58 | -| Variable nommée par `apiKeyEnv` | Contient la clé d'API. Nom par défaut : aucun pour `dmr`, `LLAMA_API_KEY` pour `llamacpp`. | | 58 | +| Variable nommée par `apiKeyEnv` | Contient la clé d'API. Nom par défaut : aucun pour `dmr`, `LLAMA_API_KEY` pour `llamacpp`, `LLMMAN_API_KEY` pour `llmman`, `OLLAMA_API_KEY` pour `ollama`. | |
| 59 | 59 | ||
| 60 | ## Sortie de démarrage | 60 | ## Sortie de démarrage |
| 61 | 61 | ||
modified
docs/fr/reference/configuration.md +6 -4 Large diff collapsed to keep the page fast — load it when you need it.
modified
internal/config/config.go +1 -1 Large diff collapsed to keep the page fast — load it when you need it.
added
internal/engine/apips.go +51 -0 Large diff collapsed to keep the page fast — load it when you need it.
added
internal/engine/llmman.go +33 -0 Large diff collapsed to keep the page fast — load it when you need it.
added
internal/engine/ollama.go +27 -0 Large diff collapsed to keep the page fast — load it when you need it.
modified
internal/engine/openai_compat.go +48 -6 Large diff collapsed to keep the page fast — load it when you need it.
modified
internal/engine/provider_test.go +85 -6 Large diff collapsed to keep the page fast — load it when you need it.
modified
internal/engine/summary_test.go +36 -4 Large diff collapsed to keep the page fast — load it when you need it.
modified
internal/engine/usage.go +41 -15 Large diff collapsed to keep the page fast — load it when you need it.
modified
internal/engine/usage_test.go +1 -1 Large diff collapsed to keep the page fast — load it when you need it.
modified
main.go +2 -1 Large diff collapsed to keep the page fast — load it when you need it.