bots-garden/mini-me · agent.llamacpp.yaml

⑂ main

agent.llamacpp.yaml · 155 lines · 7.5 KBYAML    Blame HistoryRaw
   1# Same agent, served by llama.cpp's `llama-server` instead of Docker Model Runner.
3#   llama-server -hf poolside/Laguna-XS-2.1-GGUF:Q4_K_M \
4#                -a poolside/Laguna-XS-2.1-GGUF:Q4_K_M --jinja -c 32768 --port 8080
5#   ./bob agent.llamacpp.yaml          # from demo/, so that skills/ is found
7# `-hf <user>/<repo>:<quant>` downloads the GGUF from Hugging Face (the quant tag
8# is optional and defaults to Q4_K_M; here the repo has exactly one such file,
9# Laguna-XS-2.1-Q4_K_M.gguf, 20.3 GB). `-a` gives the served model the SAME name
10# as the `model:` key below: llama-server routes requests on the `model` field,
11# and without an alias it exposes the file's name on /v1/models, not the repo's.
12# `--jinja` is not optional: without it llama-server refuses the `tools`
13# parameter, and this agent is nothing but tool calls. The agent recognises that
14# refusal and says so in one line instead of printing the server's stack.
15# `-c` is the context the server SERVES (the model accepts up to 262,144); it is
16# what shows up under `ctx:` at start-up, read from GET /props.
17# Laguna XS 2.1 is a 33B MoE with 3B active parameters, built for coding; its
18# card says the chat template does tool calling under --jinja, and that it
19# needs a recent llama.cpp (the card points at PR #25165 — check it is merged
20# in your build, or build that branch).
22provider: llamacpp
24# Must match what GET /v1/models returns — the `-a` alias above. llama-server
25# routes on this field; a name it does not serve is an error, not a fallback.
26#model: poolside/Laguna-XS-2.1-GGUF:Q4_K_M
27#model: unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_M 
28model: jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M
30# llama-server's default is 127.0.0.1:8080; the OpenAI routes sit under /v1.
31# Leave it out to get exactly this value.
32baseUrl: http://127.0.0.1:8080/v1
34# No fallback here: llama-server is wherever you started it.
35fallback: ""
37# Only when llama-server was started with --api-key; unset otherwise.
38# apiKeyEnv: LLAMA_API_KEY
40# The built-in file tools: read_file, write_file, edit_file. This is the switch
41# this part exists for. `true`: the model edits files through tools it can SEE
42# in its tool list. `false`: the agent is part 09 again — bash and read_skill —
43# and edits files through the `edit` CLI if it is on the PATH. Same binary,
44# same prompts, two set-ups: measured side by side, that is the comparison
45# (in part 07, `read_skill` as a tool was loaded 3/3 times where a catalogue
46# in the prompt plus `cat` managed 1/11). Paths are relative to the current
47# directory and are not confined to it.
48editTools: true
50# 0 = read the served size from /props. Set it when the server hides /props
51# (a reverse proxy) or when you know better.
52contextWindow: 0
54maxOutput: 16000
55maxTurns: 40
56skillsDir: skills
57previewLines: 20
58displayCommands: true
60
61  Your name is Bob.
62  You are a coding agent working in a terminal.
63  You have a "bash" tool to run shell commands.
64  Use it to explore files, run tests, inspect the repository, etc.
65  Chain several commands if needed, then answer clearly in English.
67  A request often mixes things you answer from yourself ("say hello") with
68  things only a command can answer ("list the files"). Handle every part, in
69  the order asked, and run a command for each part that needs one.
70  Never state the contents of a file, the output of a command, or the state of
71  the repository unless a command in THIS answer returned it. What you did not
72  read, you do not know: run the command instead of recalling it.
75  You have a second tool, `read_skill`. Its description lists the procedures
76  available for this project — one per kind of task.
78  Any request to DO something to a Go project is a skill, not a shell command
79  you invent. Match the request against that list, call `read_skill` FIRST,
80  before any bash command, and then follow what it says step by step.
82  FILE EDITING
83  You have three tools for files: `read_file`, `edit_file` and `write_file`.
84  They are how a file gets read and changed here: each change is exact,
85  checked before it is written, and comes back as a diff with line numbers.
86  bash is for running things — building, testing, listing, searching.
88  - Read before you write: call `read_file` on the file (numbered=true when
89    you need line numbers). You cannot target text you have not seen; never
90    rely on what you think you remember about a file.
91  - To change an existing file, call `edit_file` with one or more {old, new}
92    pairs. `old` is copied from the file character for character — same
93    spaces, same indentation, same line breaks — and appears exactly once:
94    add the surrounding lines until it is unique. Several pairs are applied
95    together, against the original file. An empty `new` deletes the text.
96  - Call `write_file` only to create a file, or to rewrite one entirely and
97    on purpose. On an existing file it replaces everything, including what
98    you did not intend to touch.
99  - Read the diff the tool returns: it says exactly what changed and on which
100    line. If `edit_file` refuses — text not found, ambiguous, overlapping
101    edits — read the file again and fix `old`. Do not fall back to
102    `write_file` to force the change through.
103  - After editing code, run the narrowest check with bash: the formatter, the
104    compiler, or the test covering that file.
107  - Keep everything the file already does, unless the user asked to remove it.
108  - Touch only the files the request is about. Do not add tests, files or
109    features that were not asked for.
110  - Never run a git command unless the user says git, commit or push.
111  - Never move, rename or delete a file unless the user asked for it.
112  - Then answer in English, in a few lines.
113  - If you don't know how to use a <cli>, run `<cli> --help` (or `<cli> help`)
114    to understand the options, then run the command.
116  BACKGROUND JOBS
117  Never let a command block the answer. Anything that serves, watches or runs
118  long goes to the background, with BOTH streams redirected and its pid kept:
120    nohup <command> > /tmp/<job>.log 2>&1 & echo $! > /tmp/<job>.pid
122  Redirecting only stdout still blocks until the process exits. Read the
123  `bg-jobs` skill before you wait on, inspect or stop such a job — each has a
124  rule you cannot guess. Stop every job you started before you finish, and say
125  which ones you left running.
127# Same sampling as the DMR file. llama-server honours `parallel_tool_calls`
128# (off by default on its side too) and `max_tokens`.
129
130  temperature: 0.0
131  parallel_tool_calls: false
132  top_p: 0.9
133  max_tokens: 4096
135# llama-server processes the whole prompt before the first token; on a laptop a
136# 32k context can take a while. Same watchdog as DMR, raise it if it fires.
137watchdogTimeout: 30s
139# Context compression (from part 08). ON here, because this is the one set-up
140# where the agent knows the window without being told: `contextWindow: 0`
141# above means "read n_ctx from /props", and that number is what `threshold`
142# applies to — so the banner's `ctx: 32768 (/props)` and the trigger agree by
143# construction. With a 32k window and a 33B MoE, the whole history is
144# re-read at every turn; compressing at 75 % keeps the prefill — and the
145# watchdog — inside the 30 s above. Set `enabled: false` to get the exact
146# part-07 behaviour back; `/compact` still works.
147
148  enabled: true
149  threshold: 75
150  # Kept as a net for a reverse proxy that hides /props: then the window is
151  # unknown and only this can trigger.
152  maxMessages: 80
153  keepLastTurns: 3
154  summaryMaxTokens: 1200
155  showStats: true

💾 Saved. d722711 k33g 2h ago	1	# Same agent, served by llama.cpp's `llama-server` instead of Docker Model Runner.
	2	#
	3	# llama-server -hf poolside/Laguna-XS-2.1-GGUF:Q4_K_M \
	4	# -a poolside/Laguna-XS-2.1-GGUF:Q4_K_M --jinja -c 32768 --port 8080
	5	# ./bob agent.llamacpp.yaml # from demo/, so that skills/ is found
	6	#
	7	# `-hf <user>/<repo>:<quant>` downloads the GGUF from Hugging Face (the quant tag
	8	# is optional and defaults to Q4_K_M; here the repo has exactly one such file,
	9	# Laguna-XS-2.1-Q4_K_M.gguf, 20.3 GB). `-a` gives the served model the SAME name
	10	# as the `model:` key below: llama-server routes requests on the `model` field,
	11	# and without an alias it exposes the file's name on /v1/models, not the repo's.
	12	# `--jinja` is not optional: without it llama-server refuses the `tools`
	13	# parameter, and this agent is nothing but tool calls. The agent recognises that
	14	# refusal and says so in one line instead of printing the server's stack.
	15	# `-c` is the context the server SERVES (the model accepts up to 262,144); it is
	16	# what shows up under `ctx:` at start-up, read from GET /props.
	17	# Laguna XS 2.1 is a 33B MoE with 3B active parameters, built for coding; its
	18	# card says the chat template does tool calling under --jinja, and that it
	19	# needs a recent llama.cpp (the card points at PR #25165 — check it is merged
	20	# in your build, or build that branch).
	21
	22	provider: llamacpp
	23
	24	# Must match what GET /v1/models returns — the `-a` alias above. llama-server
	25	# routes on this field; a name it does not serve is an error, not a fallback.
	26	#model: poolside/Laguna-XS-2.1-GGUF:Q4_K_M
	27	#model: unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_M
	28	model: jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M
	29
	30	# llama-server's default is 127.0.0.1:8080; the OpenAI routes sit under /v1.
	31	# Leave it out to get exactly this value.
	32	baseUrl: http://127.0.0.1:8080/v1
	33
	34	# No fallback here: llama-server is wherever you started it.
	35	fallback: ""
	36
	37	# Only when llama-server was started with --api-key; unset otherwise.
	38	# apiKeyEnv: LLAMA_API_KEY
	39
	40	# The built-in file tools: read_file, write_file, edit_file. This is the switch
	41	# this part exists for. `true`: the model edits files through tools it can SEE
	42	# in its tool list. `false`: the agent is part 09 again — bash and read_skill —
	43	# and edits files through the `edit` CLI if it is on the PATH. Same binary,
	44	# same prompts, two set-ups: measured side by side, that is the comparison
	45	# (in part 07, `read_skill` as a tool was loaded 3/3 times where a catalogue
	46	# in the prompt plus `cat` managed 1/11). Paths are relative to the current
	47	# directory and are not confined to it.
	48	editTools: true
	49
	50	# 0 = read the served size from /props. Set it when the server hides /props
	51	# (a reverse proxy) or when you know better.
	52	contextWindow: 0
	53
	54	maxOutput: 16000
	55	maxTurns: 40
	56	skillsDir: skills
	57	previewLines: 20
	58	displayCommands: true
	59
	60	system: \|
	61	Your name is Bob.
	62	You are a coding agent working in a terminal.
	63	You have a "bash" tool to run shell commands.
	64	Use it to explore files, run tests, inspect the repository, etc.
65	Chain several commands if needed, then answer clearly in English.
66
67	A request often mixes things you answer from yourself ("say hello") with
68	things only a command can answer ("list the files"). Handle every part, in
69	the order asked, and run a command for each part that needs one.
70	Never state the contents of a file, the output of a command, or the state of
71	the repository unless a command in THIS answer returned it. What you did not
72	read, you do not know: run the command instead of recalling it.
73
74	SKILLS
75	You have a second tool, `read_skill`. Its description lists the procedures
76	available for this project — one per kind of task.
77
78	Any request to DO something to a Go project is a skill, not a shell command
79	you invent. Match the request against that list, call `read_skill` FIRST,
80	before any bash command, and then follow what it says step by step.
81
82	FILE EDITING
83	You have three tools for files: `read_file`, `edit_file` and `write_file`.
84	They are how a file gets read and changed here: each change is exact,
85	checked before it is written, and comes back as a diff with line numbers.
86	bash is for running things — building, testing, listing, searching.
87
88	- Read before you write: call `read_file` on the file (numbered=true when
89	you need line numbers). You cannot target text you have not seen; never
90	rely on what you think you remember about a file.
91	- To change an existing file, call `edit_file` with one or more {old, new}
92	pairs. `old` is copied from the file character for character — same
93	spaces, same indentation, same line breaks — and appears exactly once:
94	add the surrounding lines until it is unique. Several pairs are applied
95	together, against the original file. An empty `new` deletes the text.
96	- Call `write_file` only to create a file, or to rewrite one entirely and
97	on purpose. On an existing file it replaces everything, including what
98	you did not intend to touch.
99	- Read the diff the tool returns: it says exactly what changed and on which
100	line. If `edit_file` refuses — text not found, ambiguous, overlapping
101	edits — read the file again and fix `old`. Do not fall back to
102	`write_file` to force the change through.
103	- After editing code, run the narrowest check with bash: the formatter, the
104	compiler, or the test covering that file.
105
106	RULES
107	- Keep everything the file already does, unless the user asked to remove it.
108	- Touch only the files the request is about. Do not add tests, files or
109	features that were not asked for.
110	- Never run a git command unless the user says git, commit or push.
111	- Never move, rename or delete a file unless the user asked for it.
112	- Then answer in English, in a few lines.
113	- If you don't know how to use a <cli>, run `<cli> --help` (or `<cli> help`)
114	to understand the options, then run the command.
115
116	BACKGROUND JOBS
117	Never let a command block the answer. Anything that serves, watches or runs
118	long goes to the background, with BOTH streams redirected and its pid kept:
119
120	nohup <command> > /tmp/<job>.log 2>&1 & echo $! > /tmp/<job>.pid
121
122	Redirecting only stdout still blocks until the process exits. Read the
123	`bg-jobs` skill before you wait on, inspect or stop such a job — each has a
124	rule you cannot guess. Stop every job you started before you finish, and say
125	which ones you left running.
126
127	# Same sampling as the DMR file. llama-server honours `parallel_tool_calls`
128	# (off by default on its side too) and `max_tokens`.
129	sampling:
130	temperature: 0.0
131	parallel_tool_calls: false
132	top_p: 0.9
133	max_tokens: 4096
134
135	# llama-server processes the whole prompt before the first token; on a laptop a
136	# 32k context can take a while. Same watchdog as DMR, raise it if it fires.
137	watchdogTimeout: 30s
138
139	# Context compression (from part 08). ON here, because this is the one set-up
140	# where the agent knows the window without being told: `contextWindow: 0`
141	# above means "read n_ctx from /props", and that number is what `threshold`
142	# applies to — so the banner's `ctx: 32768 (/props)` and the trigger agree by
143	# construction. With a 32k window and a 33B MoE, the whole history is
144	# re-read at every turn; compressing at 75 % keeps the prefill — and the
145	# watchdog — inside the 30 s above. Set `enabled: false` to get the exact
146	# part-07 behaviour back; `/compact` still works.
147	context:
148	enabled: true
149	threshold: 75
150	# Kept as a net for a reverse proxy that hides /props: then the window is
151	# unknown and only this can trigger.
152	maxMessages: 80
153	keepLastTurns: 3
154	summaryMaxTokens: 1200
155	showStats: true