bots-garden/houlapublic Fork
⑂ v0.0.1
Commits
⬇ Clone ▾
git clone https://git.rickub.com/bots-garden/houla.git
git clone ssh://git@rickub.com/bots-garden/houla.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Download ZIP
🛟 Updated. 1dfc968Unverified · on v0.0.1 · k33g · 4h ago
README.md · 110 lines · 4.0 KBmarkdown
Blame HistoryOpen raw

Houla ... Houla

🇫🇷 Version française

1. What is this project?

Houla runs the Olala-v1.0-mini-7A1B language model (by OVHai) on your machine, inside a Docker sandbox (sbx), and exposes it as an OpenAI-compatible API on http://localhost:8080.

  • The model is served by llama-server, from OVHai's llama.cpp fork (CPU only, no GPU needed).
  • The model file (GGUF, ~7 GB) is downloaded once into the ./models folder and reused afterwards.
  • Everything is described in sbxenv.yaml: one command creates the sandbox, downloads the model and starts the server.

2. Create the Hugging Face secret

Why? The model is downloaded from Hugging Face. With a token, downloads are authenticated (fewer rate limits, and access to gated or private models).

The real token never enters the sandbox: the sandbox only sees a placeholder in HF_TOKEN, and the sbx proxy swaps in the real token when a request goes to huggingface.co.

  1. Create a Read token at https://huggingface.co/settings/tokens.
  2. On the host, run once (replace hf_xxx with your token):
printf '%s' "hf_xxx" | sbx secret set-custom \
    --host huggingface.co \
    --host '*.huggingface.co' \
    --env HF_TOKEN \
    --value-stdin

The secret is global (it applies to every sandbox). Do this before creating the sandbox. More details: docs/huggingface-token-sbx.en.md.

Never put the token in sbxenv.yaml: that file is committed.

3. First launch

On the host, at the project root:

sbx env create sbxenv.yaml

This creates the houla-olala sandbox, downloads the model (first time only, a few minutes) and starts the server on port 8080.

Test it from the host:

# Is the server up?
curl -s http://localhost:8080/health

# Which model is served?
curl -s http://localhost:8080/v1/models

# Ask a question
curl -s http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Olala-v1.0-mini-7A1B-q8_0-safe2",
    "messages": [{"role": "user", "content": "Explain how a heat pump works in two sentences."}],
    "reasoning_effort": "low"
  }' | jq -r '.choices[0].message.content'

# Streaming answer
curl -sN http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Olala-v1.0-mini-7A1B-q8_0-safe2",
    "stream": true,
    "reasoning_effort": "disabled",
    "messages": [{"role": "user", "content": "Say hello"}]
  }'

reasoning_effort can be disabled, low, medium (default) or high.

4. Start again after deleting the sandbox

sbx env rm sbxenv.yaml       # delete the sandbox (the model stays in ./models)
sbx env create sbxenv.yaml   # recreate it: no new download, the server starts directly

If the sandbox was only stopped (sbx stop, reboot), no need to recreate it:

sbx run -d --name houla-olala
sbx env exec --no-input sbxenv.yaml -- bash -lc "$PWD/scripts/olala.sh serve"

5. Settings: port and model

Pass them with --env-arg when creating the sandbox:

# Another host port
sbx env create sbxenv.yaml --env-arg host_port=9090

# Another model file (from the same Hugging Face repository)
sbx env create sbxenv.yaml --env-arg model_file=Olala-v1.0-mini-7A1B-q5_k_m.gguf
Argument Default Purpose
host_port 8080 Port on the host
model_repo OVHaiLLM/Olala-v1.0-mini-7A1B-GGUF Hugging Face repository
model_file Olala-v1.0-mini-7A1B-q8_0-safe2.gguf GGUF file to download and serve

Use the same --env-arg values with every sbx env command of that sandbox (rm, exec…).
With another port, replace 8080 in the curl URLs. With another model, the "model" name in the requests is the file name without .gguf.

Other settings (models_dir, threads, ctx) and details: docs/olala-sbx.en.md.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
# Houla ... Houla

🇫🇷 [Version française](README.fr.md)

## 1. What is this project?

Houla runs the **Olala-v1.0-mini-7A1B** language model (by OVHai) on your machine, inside a Docker sandbox (`sbx`), and exposes it as an **OpenAI-compatible API** on `http://localhost:8080`.

- The model is served by `llama-server`, from OVHai's [llama.cpp fork](https://github.com/OVHai-LLM/llama.cpp) (CPU only, no GPU needed).
- The model file (GGUF, ~7 GB) is downloaded once into the `./models` folder and reused afterwards.
- Everything is described in [`sbxenv.yaml`](sbxenv.yaml): one command creates the sandbox, downloads the model and starts the server.

## 2. Create the Hugging Face secret

**Why?** The model is downloaded from [Hugging Face](https://huggingface.co/OVHaiLLM/Olala-v1.0-mini-7A1B-GGUF). With a token, downloads are authenticated (fewer rate limits, and access to gated or private models).

The real token never enters the sandbox: the sandbox only sees a placeholder in `HF_TOKEN`, and the sbx proxy swaps in the real token when a request goes to `huggingface.co`.

1. Create a **Read** token at <https://huggingface.co/settings/tokens>.
2. On the host, run once (replace `hf_xxx` with your token):

```bash
printf '%s' "hf_xxx" | sbx secret set-custom \
    --host huggingface.co \
    --host '*.huggingface.co' \
    --env HF_TOKEN \
    --value-stdin
```

The secret is global (it applies to every sandbox). Do this **before** creating the sandbox. More details: [docs/huggingface-token-sbx.en.md](docs/huggingface-token-sbx.en.md).

> Never put the token in `sbxenv.yaml`: that file is committed.

## 3. First launch

On the host, at the project root:

```bash
sbx env create sbxenv.yaml
```

This creates the `houla-olala` sandbox, downloads the model (first time only, a few minutes) and starts the server on port 8080.

Test it from the host:

```bash
# Is the server up?
curl -s http://localhost:8080/health

# Which model is served?
curl -s http://localhost:8080/v1/models

# Ask a question
curl -s http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Olala-v1.0-mini-7A1B-q8_0-safe2",
    "messages": [{"role": "user", "content": "Explain how a heat pump works in two sentences."}],
    "reasoning_effort": "low"
  }' | jq -r '.choices[0].message.content'

# Streaming answer
curl -sN http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Olala-v1.0-mini-7A1B-q8_0-safe2",
    "stream": true,
    "reasoning_effort": "disabled",
    "messages": [{"role": "user", "content": "Say hello"}]
  }'
```

`reasoning_effort` can be `disabled`, `low`, `medium` (default) or `high`.

## 4. Start again after deleting the sandbox

```bash
sbx env rm sbxenv.yaml       # delete the sandbox (the model stays in ./models)
sbx env create sbxenv.yaml   # recreate it: no new download, the server starts directly
```

If the sandbox was only **stopped** (`sbx stop`, reboot), no need to recreate it:

```bash
sbx run -d --name houla-olala
sbx env exec --no-input sbxenv.yaml -- bash -lc "$PWD/scripts/olala.sh serve"
```

## 5. Settings: port and model

Pass them with `--env-arg` when creating the sandbox:

```bash
# Another host port
sbx env create sbxenv.yaml --env-arg host_port=9090

# Another model file (from the same Hugging Face repository)
sbx env create sbxenv.yaml --env-arg model_file=Olala-v1.0-mini-7A1B-q5_k_m.gguf
```

| Argument | Default | Purpose |
| --- | --- | --- |
| `host_port` | `8080` | Port on the host |
| `model_repo` | `OVHaiLLM/Olala-v1.0-mini-7A1B-GGUF` | Hugging Face repository |
| `model_file` | `Olala-v1.0-mini-7A1B-q8_0-safe2.gguf` | GGUF file to download and serve |

> Use the **same** `--env-arg` values with every `sbx env` command of that sandbox (`rm`, `exec`…).
> With another port, replace `8080` in the curl URLs. With another model, the `"model"` name in the requests is the file name without `.gguf`.

Other settings (`models_dir`, `threads`, `ctx`) and details: [docs/olala-sbx.en.md](docs/olala-sbx.en.md).