rickub
bots-garden/mini-mepublic Fork
⑂ 5f33b3c
Branches
feature/agents-file feature/ask-agent-hooks feature/ask-cli feature/declared-tools feature/display-called-hooks feature/fetch-build-in feature/genkit-v1.13.1 feature/github-mcp-in-sbx feature/go-1.27.3 feature/hooks feature/improve-init feature/interactive-prompt feature/last-question feature/llmman-provider feature/log-errors feature/lsp-builtin feature/mcp-gateway-session feature/mcp-support feature/memory-search feature/mini-mes feature/mm-init feature/open-terminal feature/prompt-flag feature/response-request feature/resume-acp feature/sbx-template-kit feature/search-in-doc feature/selectors feature/skill-discovery feature/structured-output feature/system-one feature/talk-to-another-mm feature/tasks-builtin feature/try-again feature/unload-model feature/wasm-custom-tools feature/yolo feature/zed-skill-selector fix/generation-timeout fix/plan-mode fix/too-long-completion fix/tool-order fix/word-wrap improve-mem-usage main thinking
Tags
v0.0.0 v0.0.1 v0.0.2 v0.0.3 v0.0.4 v0.1.0 v0.1.1 v0.1.2 v0.1.3 v0.1.4 v0.1.5 v0.1.6 v0.1.7 v0.1.8 v0.1.9 v0.2.0 v0.2.1 v0.2.2 v0.2.3 v0.2.4 v0.2.5 v0.2.6 v0.3.0 v0.4.0 v0.5.0 v0.6.0 v0.7.0 v0.8.0 v0.9.0 v1.0.0 View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/bots-garden/mini-me.git
git clone ssh://git@rickub.com/bots-garden/mini-me.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Download ZIP

🛟 Updated.

k33g committed 2026-10-18T17:39:19+02:00 Browse files
5f33b3c parent: 282c4a5
modified .tickets/issues/0092-how-to-unload-a-model.yaml +49 -3
@@ -6,10 +6,56 @@ author:
66 name: k33g
77 email: ph.charriere@gmail.com
88 createdAt: 2026-10-17T14:24:18.694Z
9-updatedAt: 2026-10-17T14:24:34.591Z
9+updatedAt: 2026-10-18T16:00:00.000Z
1010 labels:
1111 - priority::urgent
1212 milestone: v0.7.0
13-body: ""
13+body: |
14+ Is there an API to unload a model from memory, for llmman, Ollama and
15+ llama.cpp (llama-server)?
1416 tasks: []
15-comments: []
17+comments:
18+ - id: 1
19+ author:
20+ name: k33g
21+ email: ph.charriere@gmail.com
22+ createdAt: 2026-10-18T16:00:00.000Z
23+ body: |
24+ Findings (2026-10-18). Yes for all three, three different ways.
25+
26+ llmman (docs/api.md, "Model lifecycle"; VERIFIED live on 127.0.0.1:17434)
27+ - API: POST /api/generate {"model": "<model>", "keep_alive": 0}
28+ → {"done": true, "done_reason": "unload"}; the model leaves /api/ps.
29+ Checked with hf.co/unsloth/Qwen3.5-0.8B-GGUF:latest (loaded by a
30+ generate, listed in /api/ps, gone after the call).
31+ - CLI: `llmman stop <model>` (with aggregation, reaches the model
32+ wherever it was loaded).
33+ - Automatic: idle models unload after keep_alive (default 5 min;
34+ per request, or daemon-wide with LLMMAN_KEEP_ALIVE);
35+ LLMMAN_MAX_LOADED_MODELS evicts the least-recently-used idle one.
36+ /api/ps shows each model's expires_at.
37+ - keep_alive is NOT documented on /v1/chat/completions (only on the
38+ Ollama routes and /v1/systemone).
39+
40+ Ollama (docs/api.md "Unload a model", twice; cmd/cmd.go)
41+ - API: POST /api/generate {"model": "<model>", "keep_alive": 0} (empty
42+ prompt), or POST /api/chat {"model": "<model>", "messages": [],
43+ "keep_alive": 0} → "done_reason": "unload".
44+ - CLI: `ollama stop MODEL` ("Stop a running model").
45+ - Automatic: keep_alive (default 5 min), as llmman.
46+ Not run here (no Ollama server in reach).
47+
48+ llama.cpp, llama-server (tools/server/README.md)
49+ - Router mode only (llama-server started WITHOUT a model, models from
50+ the cache, --models-dir or --models-preset): POST /models/unload
51+ {"model": "<model>"} → {"success": true}; POST /models/load to load;
52+ GET /models shows status loaded/unloaded; --models-max N caps the
53+ loaded ones (default 4). Preset option stop-timeout (default 10 s)
54+ before forcing termination.
55+ - Single-model mode: no unload route; --sleep-idle-seconds N unloads
56+ the model and its KV cache after N idle seconds and reloads it on the
57+ next task (sleep state on GET /props). Otherwise, stop the process.
58+ Not run here (no llama-server in reach).
59+
60+ For mm: engine/apips.go already reads /api/ps (llmman, Ollama). A
61+ `/unload` command or an unload on exit would be a follow-up, not done.
@@ -6,10 +6,56 @@ author:
6 name: k33g6 name: k33g
7 email: ph.charriere@gmail.com7 email: ph.charriere@gmail.com
8 createdAt: 2026-10-17T14:24:18.694Z8 createdAt: 2026-10-17T14:24:18.694Z
9-updatedAt: 2026-10-17T14:24:34.591Z9+updatedAt: 2026-10-18T16:00:00.000Z
10 labels:10 labels:
11 - priority::urgent11 - priority::urgent
12 milestone: v0.7.012 milestone: v0.7.0
13-body: ""13+body: |
14+ Is there an API to unload a model from memory, for llmman, Ollama and
15+ llama.cpp (llama-server)?
14 tasks: []16 tasks: []
15-comments: []17+comments:
18+ - id: 1
19+ author:
20+ name: k33g
21+ email: ph.charriere@gmail.com
22+ createdAt: 2026-10-18T16:00:00.000Z
23+ body: |
24+ Findings (2026-10-18). Yes for all three, three different ways.
25+
26+ llmman (docs/api.md, "Model lifecycle"; VERIFIED live on 127.0.0.1:17434)
27+ - API: POST /api/generate {"model": "<model>", "keep_alive": 0}
28+ → {"done": true, "done_reason": "unload"}; the model leaves /api/ps.
29+ Checked with hf.co/unsloth/Qwen3.5-0.8B-GGUF:latest (loaded by a
30+ generate, listed in /api/ps, gone after the call).
31+ - CLI: `llmman stop <model>` (with aggregation, reaches the model
32+ wherever it was loaded).
33+ - Automatic: idle models unload after keep_alive (default 5 min;
34+ per request, or daemon-wide with LLMMAN_KEEP_ALIVE);
35+ LLMMAN_MAX_LOADED_MODELS evicts the least-recently-used idle one.
36+ /api/ps shows each model's expires_at.
37+ - keep_alive is NOT documented on /v1/chat/completions (only on the
38+ Ollama routes and /v1/systemone).
39+
40+ Ollama (docs/api.md "Unload a model", twice; cmd/cmd.go)
41+ - API: POST /api/generate {"model": "<model>", "keep_alive": 0} (empty
42+ prompt), or POST /api/chat {"model": "<model>", "messages": [],
43+ "keep_alive": 0} → "done_reason": "unload".
44+ - CLI: `ollama stop MODEL` ("Stop a running model").
45+ - Automatic: keep_alive (default 5 min), as llmman.
46+ Not run here (no Ollama server in reach).
47+
48+ llama.cpp, llama-server (tools/server/README.md)
49+ - Router mode only (llama-server started WITHOUT a model, models from
50+ the cache, --models-dir or --models-preset): POST /models/unload
51+ {"model": "<model>"} → {"success": true}; POST /models/load to load;
52+ GET /models shows status loaded/unloaded; --models-max N caps the
53+ loaded ones (default 4). Preset option stop-timeout (default 10 s)
54+ before forcing termination.
55+ - Single-model mode: no unload route; --sleep-idle-seconds N unloads
56+ the model and its KV cache after N idle seconds and reloads it on the
57+ next task (sleep state on GET /props). Otherwise, stop the process.
58+ Not run here (no llama-server in reach).
59+
60+ For mm: engine/apips.go already reads /api/ps (llmman, Ollama). A
61+ `/unload` command or an unload on exit would be a follow-up, not done.
added .tickets/issues/0101-unload-a-model-command-and-tool.yaml +116 -0
new file mode 100644
@@ -0,0 +1,116 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 101
3+title: Unload a model from memory — /unload command and unload_model tool
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-18T16:30:00.000Z
9+updatedAt: 2026-10-18T16:30:00.000Z
10+labels:
11+ - kind::feature
12+milestone: v0.7.0
13+body: |
14+ Follows ticket 92 (research: how each server unloads a model). Give mm the
15+ capacity to unload a model from the server's memory, for the llmman, ollama
16+ and llamacpp providers, in three forms:
17+
18+ 1. a slash command `/unload [model]`, in the line mode, the window (-tui)
19+ AND over ACP (an editor's slash-command menu);
20+ 2. a built-in tool `unload_model`, like `ask_agent`, that the model calls
21+ itself when its instructions say so — e.g. "unload your model before you
22+ hand the task to another agent", or before handing control back;
23+ 3. a demo on llmman in `demos/unload-models/`.
24+
25+ The capacity can be turned off in the configuration file.
26+
27+ Proposal (to confirm before coding):
28+ - Key `unloadModels` (bool, default true). `false`: no `/unload` command
29+ (not listed, not advertised over ACP) and no `unload_model` tool.
30+ - `/unload` alone unloads THIS agent's model; `/unload <model>` another one
31+ served by the same server. The next question reloads the model, as usual.
32+ - Tool `unload_model {model?: string}`: empty = this agent's model. The
33+ output says what happened, as text (unloaded / was not loaded / not
34+ supported by this server, and why), never a Go error.
35+ - Per provider (ticket 92 comment 1):
36+ - llmman, ollama: POST <root>/api/generate {"model": m, "keep_alive": 0}
37+ (root = baseUrl without /v1); success = done_reason "unload".
38+ - llamacpp: POST <root>/models/unload {"model": m} — router mode only; a
39+ single-model llama-server has no such route: say so, and point to
40+ --sleep-idle-seconds.
41+ - dmr: out of scope for now — "not supported by this provider".
42+ - Permission: `unload_model` changes nothing on disk; it is gated like
43+ `ask_agent` (asked when `yolo: false`)? To decide.
44+ - Hooks: the tool goes through preToolCall/postToolCall like every tool.
45+
46+ Open questions:
47+ - Default on or off? (proposal: on, since unloading is harmless — the next
48+ request reloads.)
49+ - Should the tool be allowed to unload a model other than its own, or only
50+ its own (safer for a small model that might unload the wrong one)?
51+ - Permission with `yolo: false`: ask, or never ask?
52+tasks:
53+ - id: 1
54+ title: "Agree on the key name and default, the tool's scope (own model only or any), and its permission"
55+ state: open
56+ priority: none
57+ author:
58+ name: k33g
59+ email: ph.charriere@gmail.com
60+ createdAt: 2026-10-18T16:30:00.000Z
61+ updatedAt: 2026-10-18T16:30:00.000Z
62+ - id: 2
63+ title: "engine: Unload(ctx, model) per provider — llmman/ollama /api/generate keep_alive 0, llamacpp /models/unload, others not supported — with tests on a fake server"
64+ state: open
65+ priority: none
66+ author:
67+ name: k33g
68+ email: ph.charriere@gmail.com
69+ createdAt: 2026-10-18T16:30:00.000Z
70+ updatedAt: 2026-10-18T16:30:00.000Z
71+ - id: 3
72+ title: "config key (unloadModels), validation and banner"
73+ state: open
74+ priority: none
75+ author:
76+ name: k33g
77+ email: ph.charriere@gmail.com
78+ createdAt: 2026-10-18T16:30:00.000Z
79+ updatedAt: 2026-10-18T16:30:00.000Z
80+ - id: 4
81+ title: "/unload [model] in the command table for Terminal | Window | Editor, handled by the three front ends"
82+ state: open
83+ priority: none
84+ author:
85+ name: k33g
86+ email: ph.charriere@gmail.com
87+ createdAt: 2026-10-18T16:30:00.000Z
88+ updatedAt: 2026-10-18T16:30:00.000Z
89+ - id: 5
90+ title: "built-in tool unload_model (reserved name, declared unless unloadModels is false)"
91+ state: open
92+ priority: none
93+ author:
94+ name: k33g
95+ email: ph.charriere@gmail.com
96+ createdAt: 2026-10-18T16:30:00.000Z
97+ updatedAt: 2026-10-18T16:30:00.000Z
98+ - id: 6
99+ title: "demos/unload-models/ on llmman: a coordinator that unloads its model before handing a task to another agent (different model), READMEs EN+FR, measured"
100+ state: open
101+ priority: none
102+ author:
103+ name: k33g
104+ email: ph.charriere@gmail.com
105+ createdAt: 2026-10-18T16:30:00.000Z
106+ updatedAt: 2026-10-18T16:30:00.000Z
107+ - id: 7
108+ title: "Docs EN+FR (reference: configuration, tools, slash-commands, acp; how-to), quality gate, .memory"
109+ state: open
110+ priority: none
111+ author:
112+ name: k33g
113+ email: ph.charriere@gmail.com
114+ createdAt: 2026-10-18T16:30:00.000Z
115+ updatedAt: 2026-10-18T16:30:00.000Z
116+comments: []
new file mode 100644
@@ -0,0 +1,116 @@
1+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2+id: 101
3+title: Unload a model from memory — /unload command and unload_model tool
4+state: open
5+author:
6+ name: k33g
7+ email: ph.charriere@gmail.com
8+createdAt: 2026-10-18T16:30:00.000Z
9+updatedAt: 2026-10-18T16:30:00.000Z
10+labels:
11+ - kind::feature
12+milestone: v0.7.0
13+body: |
14+ Follows ticket 92 (research: how each server unloads a model). Give mm the
15+ capacity to unload a model from the server's memory, for the llmman, ollama
16+ and llamacpp providers, in three forms:
17+
18+ 1. a slash command `/unload [model]`, in the line mode, the window (-tui)
19+ AND over ACP (an editor's slash-command menu);
20+ 2. a built-in tool `unload_model`, like `ask_agent`, that the model calls
21+ itself when its instructions say so — e.g. "unload your model before you
22+ hand the task to another agent", or before handing control back;
23+ 3. a demo on llmman in `demos/unload-models/`.
24+
25+ The capacity can be turned off in the configuration file.
26+
27+ Proposal (to confirm before coding):
28+ - Key `unloadModels` (bool, default true). `false`: no `/unload` command
29+ (not listed, not advertised over ACP) and no `unload_model` tool.
30+ - `/unload` alone unloads THIS agent's model; `/unload <model>` another one
31+ served by the same server. The next question reloads the model, as usual.
32+ - Tool `unload_model {model?: string}`: empty = this agent's model. The
33+ output says what happened, as text (unloaded / was not loaded / not
34+ supported by this server, and why), never a Go error.
35+ - Per provider (ticket 92 comment 1):
36+ - llmman, ollama: POST <root>/api/generate {"model": m, "keep_alive": 0}
37+ (root = baseUrl without /v1); success = done_reason "unload".
38+ - llamacpp: POST <root>/models/unload {"model": m} — router mode only; a
39+ single-model llama-server has no such route: say so, and point to
40+ --sleep-idle-seconds.
41+ - dmr: out of scope for now — "not supported by this provider".
42+ - Permission: `unload_model` changes nothing on disk; it is gated like
43+ `ask_agent` (asked when `yolo: false`)? To decide.
44+ - Hooks: the tool goes through preToolCall/postToolCall like every tool.
45+
46+ Open questions:
47+ - Default on or off? (proposal: on, since unloading is harmless — the next
48+ request reloads.)
49+ - Should the tool be allowed to unload a model other than its own, or only
50+ its own (safer for a small model that might unload the wrong one)?
51+ - Permission with `yolo: false`: ask, or never ask?
52+tasks:
53+ - id: 1
54+ title: "Agree on the key name and default, the tool's scope (own model only or any), and its permission"
55+ state: open
56+ priority: none
57+ author:
58+ name: k33g
59+ email: ph.charriere@gmail.com
60+ createdAt: 2026-10-18T16:30:00.000Z
61+ updatedAt: 2026-10-18T16:30:00.000Z
62+ - id: 2
63+ title: "engine: Unload(ctx, model) per provider — llmman/ollama /api/generate keep_alive 0, llamacpp /models/unload, others not supported — with tests on a fake server"
64+ state: open
65+ priority: none
66+ author:
67+ name: k33g
68+ email: ph.charriere@gmail.com
69+ createdAt: 2026-10-18T16:30:00.000Z
70+ updatedAt: 2026-10-18T16:30:00.000Z
71+ - id: 3
72+ title: "config key (unloadModels), validation and banner"
73+ state: open
74+ priority: none
75+ author:
76+ name: k33g
77+ email: ph.charriere@gmail.com
78+ createdAt: 2026-10-18T16:30:00.000Z
79+ updatedAt: 2026-10-18T16:30:00.000Z
80+ - id: 4
81+ title: "/unload [model] in the command table for Terminal | Window | Editor, handled by the three front ends"
82+ state: open
83+ priority: none
84+ author:
85+ name: k33g
86+ email: ph.charriere@gmail.com
87+ createdAt: 2026-10-18T16:30:00.000Z
88+ updatedAt: 2026-10-18T16:30:00.000Z
89+ - id: 5
90+ title: "built-in tool unload_model (reserved name, declared unless unloadModels is false)"
91+ state: open
92+ priority: none
93+ author:
94+ name: k33g
95+ email: ph.charriere@gmail.com
96+ createdAt: 2026-10-18T16:30:00.000Z
97+ updatedAt: 2026-10-18T16:30:00.000Z
98+ - id: 6
99+ title: "demos/unload-models/ on llmman: a coordinator that unloads its model before handing a task to another agent (different model), READMEs EN+FR, measured"
100+ state: open
101+ priority: none
102+ author:
103+ name: k33g
104+ email: ph.charriere@gmail.com
105+ createdAt: 2026-10-18T16:30:00.000Z
106+ updatedAt: 2026-10-18T16:30:00.000Z
107+ - id: 7
108+ title: "Docs EN+FR (reference: configuration, tools, slash-commands, acp; how-to), quality gate, .memory"
109+ state: open
110+ priority: none
111+ author:
112+ name: k33g
113+ email: ph.charriere@gmail.com
114+ createdAt: 2026-10-18T16:30:00.000Z
115+ updatedAt: 2026-10-18T16:30:00.000Z
116+comments: []