bots-garden/mini-me · aef6e17

⑂ aef6e17

🛟 Updated.

k33g committed 2026-10-20T13:20:16+02:00 Browse files

aef6e17 parent: 303fa04

modified .tickets/issues/0046-add-builtin-tools-as-plugins-with-wasm.yaml +78 -3 Viewed

@@ -6,7 +6,7 @@ author:
6	6	name: k33g
7	7	email: ph.charriere@gmail.com
8	8	createdAt: 2026-09-27T02:30:34.975Z
9		-updatedAt: 2026-10-20T07:42:43.656Z
	9	+updatedAt: 2026-10-21T00:00:00.000Z
10	10	labels: []
11	11	milestone: v0.9.0
12	12	epic: Builtin tools
@@ -17,10 +17,85 @@ body: \|
17	17	tasks:
18	18	- id: 1
19	19	title: Find useful use cases before
20		- state: open
	20	+ state: closed
21	21	priority: urgent
22	22	author:
23	23	name: k33g
24	24	email: ph.charriere@gmail.com
25	25	createdAt: 2026-10-20T07:42:43.652Z
26		-comments: []
	26	+ updatedAt: 2026-10-21T00:00:00.000Z
	27	+ - id: 2
	28	+ title: "Experiment (not started): bash-guard as a WASM hook vs the Node one — time per call, size, capability set, author experience"
	29	+ state: open
	30	+ priority: none
	31	+ author:
	32	+ name: k33g
	33	+ email: ph.charriere@gmail.com
	34	+ createdAt: 2026-10-21T00:00:00.000Z
	35	+ updatedAt: 2026-10-21T00:00:00.000Z
	36	+comments:
	37	+ - id: 1
	38	+ author:
	39	+ name: k33g
	40	+ email: ph.charriere@gmail.com
	41	+ createdAt: 2026-10-21T00:00:00.000Z
	42	+ body: \|
	43	+ Task 1 — use cases (analysis only, nothing implemented; 2026-10-20).
	44	+
	45	+ ## What mm has today to extend the agent
	46	+ - hooks: `sh -c <command>`, JSON on stdin/stdout (internal/hooks);
	47	+ - declared tools (ticket 88): a program and its argv;
	48	+ - MCP servers: stdio or HTTP;
	49	+ - skills: Markdown the model reads.
	50	+ The first three start a process with ALL the user's rights.
	51	+
	52	+ ## What WASM plugins would bring
	53	+ 1. A sandbox by capability — the strongest argument. A module sees no
	54	+ file, no network, no environment unless mm grants it (e.g. one
	55	+ folder, read-only). Today a guard hook like
	56	+ demos/hooks/hooks/bash-guard.js can do anything on the machine. It
	57	+ extends the yolo / permission logic already in place.
	58	+ 2. No runtime to install. The hooks, guards and unload-models demos
	59	+ need Node; one .wasm (from Rust, TinyGo, JS…) runs as-is on macOS,
	60	+ Linux, Windows and in sbx sandboxes.
	61	+ 3. Speed, as a side effect. Measured here: a Node hook costs ~62 ms
	62	+ per call (median of 20, node v22, both a trivial hook and
	63	+ demos/hooks/hooks/bash-guard.js) against 0.2 ms for `sh -c true`;
	64	+ with preToolCall + postToolCall that is 125–190 ms per tool call —
	65	+ small next to a 10–30 s model turn. WASM instantiation NOT
	66	+ measured.
	67	+ 4. The binary stays simple: wazero is pure Go, so mm would stay a
	68	+ static, CGO-free binary.
	69	+
	70	+ ## What it would cost
	71	+ - An ABI to design and then freeze: exported functions, input/output
	72	+ format (likely the hooks' JSON), the capabilities a plugin is given.
	73	+ That is the real work — a contract we cannot break later.
	74	+ - WASI limits: networking is still immature; a plugin that calls an
	75	+ API is hampered. This matters for this ticket's own example: a
	76	+ safe/unsafe check with LlamaGuard (or Granite Guardian, as in
	77	+ demos/guard and demos/guards) means calling a model server, i.e.
	78	+ network access — either granted as a capability, or the WASM plugin
	79	+ does the decision logic and mm does the HTTP call for it. wazero's
	80	+ current WASI support NOT checked.
	81	+ - Overlap with MCP for tools: MCP already has an ecosystem; for tools
	82	+ WASM only adds the sandbox and "nothing to install".
	83	+ - One more dependency, and less comfortable debugging than a script.
	84	+
	85	+ ## Where it would pay, most useful first
	86	+ 1. Guard hooks (preGeneration, preToolCall, preAsk): small,
	87	+ security-sensitive, called often, mostly no network — the ideal
	88	+ case.
	89	+ 2. "Safe" declared tools that read one given folder only — this
	90	+ ticket's "like the skills, in a tools directory": a tools/ folder
	91	+ of .wasm files, each with a manifest naming its capabilities.
	92	+ 3. Document formats for search_docs (ticket 106): an Org-mode or HTML
	93	+ passage splitter as a plugin.
	94	+ Not useful for model providers or front ends.
	95	+
	96	+ ## Proposal (not started)
	97	+ A measured experiment on hooks only (task 2): rewrite bash-guard.js as
	98	+ a WASM hook with a minimal capability set, compare time per call,
	99	+ size and the hook author's experience with the Node version. If it
	100	+ pays, extend to declared tools, then to the LlamaGuard example.
	101	+

    name: k33g
    email: ph.charriere@gmail.com
  createdAt: 2026-09-27T02:30:34.975Z
--updatedAt: 2026-10-20T07:42:43.656Z
++updatedAt: 2026-10-21T00:00:00.000Z
  labels: []
  milestone: v0.9.0
  epic: Builtin tools
  tasks:
    - id: 1
      title: Find useful use cases before
--    state: open
++    state: closed
      priority: urgent
      author:
        name: k33g
        email: ph.charriere@gmail.com
      createdAt: 2026-10-20T07:42:43.652Z
--comments: []
++    updatedAt: 2026-10-21T00:00:00.000Z
++  - id: 2
++    title: "Experiment (not started): bash-guard as a WASM hook vs the Node one — time per call, size, capability set, author experience"
++    state: open
++    priority: none
++    author:
++      name: k33g
++      email: ph.charriere@gmail.com
++    createdAt: 2026-10-21T00:00:00.000Z
++    updatedAt: 2026-10-21T00:00:00.000Z
++comments:
++  - id: 1
++    author:
++      name: k33g
++      email: ph.charriere@gmail.com
++    createdAt: 2026-10-21T00:00:00.000Z
++    body: |
++      Task 1 — use cases (analysis only, nothing implemented; 2026-10-20).
++
++      ## What mm has today to extend the agent
++      - hooks: `sh -c <command>`, JSON on stdin/stdout (internal/hooks);
++      - declared tools (ticket 88): a program and its argv;
++      - MCP servers: stdio or HTTP;
++      - skills: Markdown the model reads.
++      The first three start a process with ALL the user's rights.
++
++      ## What WASM plugins would bring
++      1. A sandbox by capability — the strongest argument. A module sees no
++         file, no network, no environment unless mm grants it (e.g. one
++         folder, read-only). Today a guard hook like
++         demos/hooks/hooks/bash-guard.js can do anything on the machine. It
++         extends the yolo / permission logic already in place.
++      2. No runtime to install. The hooks, guards and unload-models demos
++         need Node; one .wasm (from Rust, TinyGo, JS…) runs as-is on macOS,
++         Linux, Windows and in sbx sandboxes.
++      3. Speed, as a side effect. Measured here: a Node hook costs ~62 ms
++         per call (median of 20, node v22, both a trivial hook and
++         demos/hooks/hooks/bash-guard.js) against 0.2 ms for `sh -c true`;
++         with preToolCall + postToolCall that is 125–190 ms per tool call —
++         small next to a 10–30 s model turn. WASM instantiation NOT
++         measured.
++      4. The binary stays simple: wazero is pure Go, so mm would stay a
++         static, CGO-free binary.
++
++      ## What it would cost
++      - An ABI to design and then freeze: exported functions, input/output
++        format (likely the hooks' JSON), the capabilities a plugin is given.
++        That is the real work — a contract we cannot break later.
++      - WASI limits: networking is still immature; a plugin that calls an
++        API is hampered. This matters for this ticket's own example: a
++        safe/unsafe check with LlamaGuard (or Granite Guardian, as in
++        demos/guard and demos/guards) means calling a model server, i.e.
++        network access — either granted as a capability, or the WASM plugin
++        does the decision logic and mm does the HTTP call for it. wazero's
++        current WASI support NOT checked.
++      - Overlap with MCP for tools: MCP already has an ecosystem; for tools
++        WASM only adds the sandbox and "nothing to install".
++      - One more dependency, and less comfortable debugging than a script.
++
++      ## Where it would pay, most useful first
++      1. Guard hooks (preGeneration, preToolCall, preAsk): small,
++         security-sensitive, called often, mostly no network — the ideal
++         case.
++      2. "Safe" declared tools that read one given folder only — this
++         ticket's "like the skills, in a tools directory": a tools/ folder
++         of .wasm files, each with a manifest naming its capabilities.
++      3. Document formats for search_docs (ticket 106): an Org-mode or HTML
++         passage splitter as a plugin.
++      Not useful for model providers or front ends.
++
++      ## Proposal (not started)
++      A measured experiment on hooks only (task 2): rewrite bash-guard.js as
++      a WASM hook with a minimal capability set, compare time per call,
++      size and the hook author's experience with the Node version. If it
++      pays, extend to declared tools, then to the LlamaGuard example.
++

modified .tickets/issues/0104-how-to-search-in-a-documentation-markdown-files.yaml +2 -3 Viewed

@@ -1,12 +1,12 @@
1	1	# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
2	2	id: 104
3	3	title: How to search in a documentation (markdown files)
4		-state: open
	4	+state: closed
5	5	author:
6	6	name: k33g
7	7	email: ph.charriere@gmail.com
8	8	createdAt: 2026-10-20T07:40:25.739Z
9		-updatedAt: 2026-10-20T19:00:00.000Z
	9	+updatedAt: 2026-10-20T09:52:55.720Z
10	10	labels: []
11	11	milestone: v0.8.0
12	12	body: ""
@@ -90,4 +90,3 @@ comments:
90	90	Decided by the user: build it into mm — a new built-in tool search_doc,
91	91	BM25, passages with the link to their file; search_memory is kept as
92	92	it is. Ticket 106.
93		-

  # Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
  id: 104
  title: How to search in a documentation (markdown files)
--state: open
++state: closed
  author:
    name: k33g
    email: ph.charriere@gmail.com
  createdAt: 2026-10-20T07:40:25.739Z
--updatedAt: 2026-10-20T19:00:00.000Z
++updatedAt: 2026-10-20T09:52:55.720Z
  labels: []
  milestone: v0.8.0
  body: ""
        Decided by the user: build it into mm — a new built-in tool search_doc,
        BM25, passages with the link to their file; search_memory is kept as
        it is. Ticket 106.
--

added .tickets/issues/0107-study-yzma-in-process-llama-cpp.yaml +116 -0 Viewed

new file mode 100644
@@ -0,0 +1,116 @@
	1	+# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
	2	+id: 107
	3	+title: Study what integrating yzma (in-process llama.cpp for Go) would imply
	4	+state: open
	5	+author:
	6	+ name: k33g
	7	+ email: ph.charriere@gmail.com
	8	+createdAt: 2026-10-21T00:30:00.000Z
	9	+updatedAt: 2026-10-21T00:30:00.000Z
	10	+labels: []
	11	+epic: Models
	12	+related:
	13	+ - 60
	14	+ - 101
	15	+body: \|
	16	+ Study what integrating yzma (https://yzma.ai/, github.com/hybridgroup/yzma)
	17	+ into mm would imply. Study only: nothing is implemented in this ticket.
	18	+
	19	+ What yzma is (from yzma.ai and its README, read 2026-10-20): a Go library
	20	+ that runs llama.cpp IN THE SAME PROCESS as the Go program, with no model
	21	+ server; no CGo (purego + ffi), so a normal `go build`; GGUF models;
	22	+ Linux, macOS, Windows and the browser (WebAssembly); CUDA, Metal, Vulkan,
	23	+ ROCm, WebGPU; vision models (images, audio, video). The llama.cpp
	24	+ prebuilt libraries are DOWNLOADED (by the `yzma` command or by the
	25	+ application, CUDA/ROCm auto-detected, SHA-256 checked), not bundled. Its
	26	+ README pairs yzma v1.30.0 with llama.cpp b11480+. Its API is llama.cpp's
	27	+ own, low level: llama.Load, ModelLoadFromFile, InitFromModel, Tokenize,
	28	+ BatchGetOne, SamplerChainInit, SamplerSample, TokenToPiece… Not found on
	29	+ those two pages: the license name, chat templates, tool calling, a
	30	+ streaming API. A downstream project, Kronk, offers an OpenAI-compatible
	31	+ API on top of it.
	32	+
	33	+ Why it is worth a look: mm needs a model SERVER today (DMR, llmman,
	34	+ Ollama, llama-server). With yzma mm could run a GGUF model by itself — one
	35	+ binary, nothing else to install or start, CGO-free like mm now.
	36	+
	37	+ What mm takes from the server today, and would have to do itself
	38	+ in-process (the core of task 2):
	39	+ - the chat template (Jinja, per model), and its kwargs
	40	+ (chat_template_kwargs.enable_thinking…);
	41	+ - tool calling: turning the model's own tool-call syntax (Qwen, Gemma's
	42	+ `<\|tool_call>`, Mellum, gpt-oss harmony…) into OpenAI tool_calls;
	43	+ - the reasoning split (`reasoning_content`) for thinking models;
	44	+ - structured output: a JSON schema turned into a grammar;
	45	+ - streaming, usage counts (prompt / completion tokens), the context
	46	+ window probe;
	47	+ - loading / unloading models (/unload, unload_model), keep-alive;
	48	+ - serving several agents at once (mmConfigs: each asked agent is a
	49	+ child `mm -acp` process).
	50	+ mm talks to all of this through Genkit's OpenAI-compatible plugin
	51	+ (compat_oai): an in-process engine is either a new Genkit model plugin or
	52	+ an OpenAI-compatible shim in front of yzma.
	53	+
	54	+ Open questions:
	55	+ - does yzma (or llama.cpp's C API it exposes) apply Jinja chat templates
	56	+ and parse tool calls, or only the built-in templates and no tools?
	57	+ - is Kronk the better first step (no mm code), with in-process left for
	58	+ later?
	59	+ - one model per mm process vs the asked agents: memory doubles if an
	60	+ asker and its expert run the same model in two processes.
	61	+tasks:
	62	+ - id: 1
	63	+ title: "Read yzma itself: license, current version and its llama.cpp pairing, Go version, how the libraries are fetched and verified, what the API covers (chat templates, tool calls, grammars / JSON schema, embeddings, streaming, token counts, unloading)"
	64	+ state: open
	65	+ priority: none
	66	+ author:
	67	+ name: k33g
	68	+ email: ph.charriere@gmail.com
	69	+ createdAt: 2026-10-21T00:30:00.000Z
	70	+ updatedAt: 2026-10-21T00:30:00.000Z
	71	+ - id: 2
	72	+ title: "Inventory what mm takes from a model server today and say, item by item, what yzma gives, what mm would have to write, and what it would lose (see the list in the body)"
	73	+ state: open
	74	+ priority: none
	75	+ author:
	76	+ name: k33g
	77	+ email: ph.charriere@gmail.com
	78	+ createdAt: 2026-10-21T00:30:00.000Z
	79	+ updatedAt: 2026-10-21T00:30:00.000Z
	80	+ - id: 3
	81	+ title: "Look at Kronk (built on yzma, OpenAI-compatible API): would it give the same benefit with no mm code, as one more provider entry?"
	82	+ state: open
	83	+ priority: none
	84	+ author:
	85	+ name: k33g
	86	+ email: ph.charriere@gmail.com
	87	+ createdAt: 2026-10-21T00:30:00.000Z
	88	+ updatedAt: 2026-10-21T00:30:00.000Z
	89	+ - id: 4
	90	+ title: "Weigh the process model: the model inside mm's process vs mm's agents asking agents (one mm -acp child per asked agent, each loading its own model), ACP sessions sharing one model, /unload and unload_model, a crash in llama.cpp taking mm down"
	91	+ state: open
	92	+ priority: none
	93	+ author:
	94	+ name: k33g
	95	+ email: ph.charriere@gmail.com
	96	+ createdAt: 2026-10-21T00:30:00.000Z
	97	+ updatedAt: 2026-10-21T00:30:00.000Z
	98	+ - id: 5
	99	+ title: "Weigh distribution: mm stays CGO-free, but needs the llama.cpp shared libraries per platform and accelerator (CUDA, ROCm, Metal, Vulkan) — downloaded at first run? by the release? in the sbx template?"
	100	+ state: open
	101	+ priority: none
	102	+ author:
	103	+ name: k33g
	104	+ email: ph.charriere@gmail.com
	105	+ createdAt: 2026-10-21T00:30:00.000Z
	106	+ updatedAt: 2026-10-21T00:30:00.000Z
	107	+ - id: 6
	108	+ title: "Conclude with a recommendation (no code, or a provider, or an in-process engine) and, if any, a measured prototype plan on one small model"
	109	+ state: open
	110	+ priority: none
	111	+ author:
	112	+ name: k33g
	113	+ email: ph.charriere@gmail.com
	114	+ createdAt: 2026-10-21T00:30:00.000Z
	115	+ updatedAt: 2026-10-21T00:30:00.000Z
	116	+comments: []

++# Managed by IssueSpec. Hand edits are welcome; keep the schema valid.
++id: 107
++title: Study what integrating yzma (in-process llama.cpp for Go) would imply
++state: open
++author:
++  name: k33g
++  email: ph.charriere@gmail.com
++createdAt: 2026-10-21T00:30:00.000Z
++updatedAt: 2026-10-21T00:30:00.000Z
++labels: []
++epic: Models
++related:
++  - 60
++  - 101
++body: |
++  Study what integrating yzma (https://yzma.ai/, github.com/hybridgroup/yzma)
++  into mm would imply. Study only: nothing is implemented in this ticket.
++
++  What yzma is (from yzma.ai and its README, read 2026-10-20): a Go library
++  that runs llama.cpp IN THE SAME PROCESS as the Go program, with no model
++  server; no CGo (purego + ffi), so a normal `go build`; GGUF models;
++  Linux, macOS, Windows and the browser (WebAssembly); CUDA, Metal, Vulkan,
++  ROCm, WebGPU; vision models (images, audio, video). The llama.cpp
++  prebuilt libraries are DOWNLOADED (by the `yzma` command or by the
++  application, CUDA/ROCm auto-detected, SHA-256 checked), not bundled. Its
++  README pairs yzma v1.30.0 with llama.cpp b11480+. Its API is llama.cpp's
++  own, low level: llama.Load, ModelLoadFromFile, InitFromModel, Tokenize,
++  BatchGetOne, SamplerChainInit, SamplerSample, TokenToPiece… Not found on
++  those two pages: the license name, chat templates, tool calling, a
++  streaming API. A downstream project, Kronk, offers an OpenAI-compatible
++  API on top of it.
++
++  Why it is worth a look: mm needs a model SERVER today (DMR, llmman,
++  Ollama, llama-server). With yzma mm could run a GGUF model by itself — one
++  binary, nothing else to install or start, CGO-free like mm now.
++
++  What mm takes from the server today, and would have to do itself
++  in-process (the core of task 2):
++  - the chat template (Jinja, per model), and its kwargs
++    (chat_template_kwargs.enable_thinking…);
++  - tool calling: turning the model's own tool-call syntax (Qwen, Gemma's
++    `<|tool_call>`, Mellum, gpt-oss harmony…) into OpenAI tool_calls;
++  - the reasoning split (`reasoning_content`) for thinking models;
++  - structured output: a JSON schema turned into a grammar;
++  - streaming, usage counts (prompt / completion tokens), the context
++    window probe;
++  - loading / unloading models (/unload, unload_model), keep-alive;
++  - serving several agents at once (mmConfigs: each asked agent is a
++    child `mm -acp` process).
++  mm talks to all of this through Genkit's OpenAI-compatible plugin
++  (compat_oai): an in-process engine is either a new Genkit model plugin or
++  an OpenAI-compatible shim in front of yzma.
++
++  Open questions:
++  - does yzma (or llama.cpp's C API it exposes) apply Jinja chat templates
++    and parse tool calls, or only the built-in templates and no tools?
++  - is Kronk the better first step (no mm code), with in-process left for
++    later?
++  - one model per mm process vs the asked agents: memory doubles if an
++    asker and its expert run the same model in two processes.
++tasks:
++  - id: 1
++    title: "Read yzma itself: license, current version and its llama.cpp pairing, Go version, how the libraries are fetched and verified, what the API covers (chat templates, tool calls, grammars / JSON schema, embeddings, streaming, token counts, unloading)"
++    state: open
++    priority: none
++    author:
++      name: k33g
++      email: ph.charriere@gmail.com
++    createdAt: 2026-10-21T00:30:00.000Z
++    updatedAt: 2026-10-21T00:30:00.000Z
++  - id: 2
++    title: "Inventory what mm takes from a model server today and say, item by item, what yzma gives, what mm would have to write, and what it would lose (see the list in the body)"
++    state: open
++    priority: none
++    author:
++      name: k33g
++      email: ph.charriere@gmail.com
++    createdAt: 2026-10-21T00:30:00.000Z
++    updatedAt: 2026-10-21T00:30:00.000Z
++  - id: 3
++    title: "Look at Kronk (built on yzma, OpenAI-compatible API): would it give the same benefit with no mm code, as one more provider entry?"
++    state: open
++    priority: none
++    author:
++      name: k33g
++      email: ph.charriere@gmail.com
++    createdAt: 2026-10-21T00:30:00.000Z
++    updatedAt: 2026-10-21T00:30:00.000Z
++  - id: 4
++    title: "Weigh the process model: the model inside mm's process vs mm's agents asking agents (one mm -acp child per asked agent, each loading its own model), ACP sessions sharing one model, /unload and unload_model, a crash in llama.cpp taking mm down"
++    state: open
++    priority: none
++    author:
++      name: k33g
++      email: ph.charriere@gmail.com
++    createdAt: 2026-10-21T00:30:00.000Z
++    updatedAt: 2026-10-21T00:30:00.000Z
++  - id: 5
++    title: "Weigh distribution: mm stays CGO-free, but needs the llama.cpp shared libraries per platform and accelerator (CUDA, ROCm, Metal, Vulkan) — downloaded at first run? by the release? in the sbx template?"
++    state: open
++    priority: none
++    author:
++      name: k33g
++      email: ph.charriere@gmail.com
++    createdAt: 2026-10-21T00:30:00.000Z
++    updatedAt: 2026-10-21T00:30:00.000Z
++  - id: 6
++    title: "Conclude with a recommendation (no code, or a provider, or an in-process engine) and, if any, a measured prototype plan on one small model"
++    state: open
++    priority: none
++    author:
++      name: k33g
++      email: ph.charriere@gmail.com
++    createdAt: 2026-10-21T00:30:00.000Z
++    updatedAt: 2026-10-21T00:30:00.000Z
++comments: []