rickub
docker-agent-cookbook/work-with-llama-cpppublic Fork
⑂ c3a1846
Branches
main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/docker-agent-cookbook/work-with-llama-cpp.git
git clone ssh://git@rickub.com/docker-agent-cookbook/work-with-llama-cpp.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Download ZIP

🛟 Updated.

k33g committed 2026-09-14T13:45:09+02:00 Browse files
c3a1846 parent: f225c7a
modified README.md +30 -3
@@ -17,9 +17,9 @@ Run this:
1717 llama serve -hf JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_M:Q4_K_M
1818 ```
1919
20-![01-llamacpp](assets/01-llamacpp.png)
20+![01-llamacpp](./assets/01-llamacpp.png)
2121
22-![02-llamacpp](assets/02-llamacpp.png)
22+![02-llamacpp](./assets/02-llamacpp.png)
2323
2424
2525 ### Check the `llama.cpp server`
@@ -48,5 +48,32 @@ curl http://localhost:8080/v1/chat/completions \
4848
4949 **`agent.yaml`**:
5050 ```yaml
51-
51+providers:
52+ llamacpp:
53+ api_type: openai_chatcompletions
54+ base_url: http://localhost:8080/v1
55+
56+models:
57+ mellum2:
58+ provider: llamacpp
59+ model: JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_M:Q4_K_M
60+ #max_tokens: 8192
61+ temperature: 0.7
62+ provider_opts:
63+ context_size: 262144
64+
65+agents:
66+ root:
67+ model: mellum2
68+ description: A helpful AI assistant running on a local llama.cpp server
69+ instruction: |
70+ You name is Bob 🤓, you are a knowledgeable code assistant that helps users with various tasks.
71+ Be helpful, accurate, and concise in your responses.
72+ You have access to the local filesystem and shell: use these tools
73+ welcome_message: |
74+ 🤖 Local Assistant propulsed by **llama.cpp** 🦙
75+
76+ toolsets:
77+ - type: filesystem
78+ - type: shell
5279 ```
\ No newline at end of file
@@ -17,9 +17,9 @@ Run this:
17 llama serve -hf JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_M:Q4_K_M17 llama serve -hf JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_M:Q4_K_M
18 ```18 ```
19 19
20-![01-llamacpp](assets/01-llamacpp.png)20+![01-llamacpp](./assets/01-llamacpp.png)
21 21
22-![02-llamacpp](assets/02-llamacpp.png)22+![02-llamacpp](./assets/02-llamacpp.png)
23 23
24 24
25 ### Check the `llama.cpp server`25 ### Check the `llama.cpp server`
@@ -48,5 +48,32 @@ curl http://localhost:8080/v1/chat/completions \
48 48
49 **`agent.yaml`**:49 **`agent.yaml`**:
50 ```yaml50 ```yaml
51-51+providers:
52+ llamacpp:
53+ api_type: openai_chatcompletions
54+ base_url: http://localhost:8080/v1
55+
56+models:
57+ mellum2:
58+ provider: llamacpp
59+ model: JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_M:Q4_K_M
60+ #max_tokens: 8192
61+ temperature: 0.7
62+ provider_opts:
63+ context_size: 262144
64+
65+agents:
66+ root:
67+ model: mellum2
68+ description: A helpful AI assistant running on a local llama.cpp server
69+ instruction: |
70+ You name is Bob 🤓, you are a knowledgeable code assistant that helps users with various tasks.
71+ Be helpful, accurate, and concise in your responses.
72+ You have access to the local filesystem and shell: use these tools
73+ welcome_message: |
74+ 🤖 Local Assistant propulsed by **llama.cpp** 🦙
75+
76+ toolsets:
77+ - type: filesystem
78+ - type: shell
52 ```79 ```
\ No newline at end of file\ No newline at end of file
modified agent.yaml +0 -1
@@ -21,7 +21,6 @@ agents:
2121 Be helpful, accurate, and concise in your responses.
2222 You have access to the local filesystem and shell: use these tools
2323 welcome_message: |
24-
2524 🤖 Local Assistant propulsed by **llama.cpp** 🦙
2625
2726 toolsets:
@@ -21,7 +21,6 @@ agents:
21 Be helpful, accurate, and concise in your responses.21 Be helpful, accurate, and concise in your responses.
22 You have access to the local filesystem and shell: use these tools22 You have access to the local filesystem and shell: use these tools
23 welcome_message: |23 welcome_message: |
24-
25 🤖 Local Assistant propulsed by **llama.cpp** 🦙24 🤖 Local Assistant propulsed by **llama.cpp** 🦙
26 25
27 toolsets:26 toolsets: