Houla ... Houla
1. What is this project?
Houla runs the Olala-v1.0-mini-7A1B language model (by OVHai) on your machine, inside a Docker sandbox (sbx), and exposes it as an OpenAI-compatible API on http://localhost:8080.
- The model is served by
llama-server, from OVHai's llama.cpp fork (CPU only, no GPU needed). - The model file (GGUF, ~7 GB) is downloaded once into the
./modelsfolder and reused afterwards. - Everything is described in
sbxenv.yaml: one command creates the sandbox, downloads the model and starts the server.
2. Create the Hugging Face secret
Why? The model is downloaded from Hugging Face. With a token, downloads are authenticated (fewer rate limits, and access to gated or private models).
The real token never enters the sandbox: the sandbox only sees a placeholder in HF_TOKEN, and the sbx proxy swaps in the real token when a request goes to huggingface.co.
- Create a Read token at https://huggingface.co/settings/tokens.
- On the host, run once (replace
hf_xxxwith your token):
printf '%s' "hf_xxx" | sbx secret set-custom \
--host huggingface.co \
--host '*.huggingface.co' \
--env HF_TOKEN \
--value-stdin
The secret is global (it applies to every sandbox). Do this before creating the sandbox. More details: docs/huggingface-token-sbx.en.md.
Never put the token in
sbxenv.yaml: that file is committed.
3. First launch
On the host, at the project root:
sbx env create sbxenv.yaml
This creates the houla-olala sandbox, downloads the model (first time only, a few minutes) and starts the server on port 8080.
Test it from the host:
# Is the server up?
curl -s http://localhost:8080/health
# Which model is served?
curl -s http://localhost:8080/v1/models
# Ask a question
curl -s http://localhost:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "Olala-v1.0-mini-7A1B-q8_0-safe2",
"messages": [{"role": "user", "content": "Explain how a heat pump works in two sentences."}],
"reasoning_effort": "low"
}' | jq -r '.choices[0].message.content'
# Streaming answer
curl -sN http://localhost:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "Olala-v1.0-mini-7A1B-q8_0-safe2",
"stream": true,
"reasoning_effort": "disabled",
"messages": [{"role": "user", "content": "Say hello"}]
}'
reasoning_effort can be disabled, low, medium (default) or high.
4. Start again after deleting the sandbox
sbx env rm sbxenv.yaml # delete the sandbox (the model stays in ./models)
sbx env create sbxenv.yaml # recreate it: no new download, the server starts directly
If the sandbox was only stopped (sbx stop, reboot), no need to recreate it:
sbx run -d --name houla-olala
sbx env exec --no-input sbxenv.yaml -- bash -lc "$PWD/scripts/olala.sh serve"
5. Settings: port and model
Pass them with --env-arg when creating the sandbox:
# Another host port
sbx env create sbxenv.yaml --env-arg host_port=9090
# Another model file (from the same Hugging Face repository)
sbx env create sbxenv.yaml --env-arg model_file=Olala-v1.0-mini-7A1B-q5_k_m.gguf
| Argument | Default | Purpose |
|---|---|---|
host_port |
8080 |
Port on the host |
model_repo |
OVHaiLLM/Olala-v1.0-mini-7A1B-GGUF |
Hugging Face repository |
model_file |
Olala-v1.0-mini-7A1B-q8_0-safe2.gguf |
GGUF file to download and serve |
Use the same
--env-argvalues with everysbx envcommand of that sandbox (rm,exec…).
With another port, replace8080in the curl URLs. With another model, the"model"name in the requests is the file name without.gguf.
Other settings (models_dir, threads, ctx) and details: docs/olala-sbx.en.md.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 |
|