🛟 Updated.
6dcdaeb parent: 2129f04 modified
docs/olala-sbx.en.md +1 -1 | @@ -33,7 +33,7 @@ Options (pass them again to every `sbx env` command of this sandbox): | ||
| 33 | 33 | | `models_dir` | `./models` | Models folder (relative to the repository root, or absolute) | |
| 34 | 34 | | `host_port` | `8080` | Host port | |
| 35 | 35 | | `threads` | `16` | CPU threads (keep 2 CPUs free: using all of them is ~30% slower) | |
| 36 | -| `ctx` | `32768` | Context size (shared by the 4 slots) | | |
| 36 | +| `ctx` | `16384` | Context size (shared by the 4 slots) | | |
| 37 | 37 | |
| 38 | 38 | ```bash |
| 39 | 39 | sbx env create sbxenv.yaml --env-arg model_file=Olala-v1.0-mini-7A1B-q5_k_m.gguf --env-arg host_port=9090 |
| @@ -33,7 +33,7 @@ Options (pass them again to every `sbx env` command of this sandbox): | |||
| 33 | | `models_dir` | `./models` | Models folder (relative to the repository root, or absolute) | | 33 | | `models_dir` | `./models` | Models folder (relative to the repository root, or absolute) | |
| 34 | | `host_port` | `8080` | Host port | | 34 | | `host_port` | `8080` | Host port | |
| 35 | | `threads` | `16` | CPU threads (keep 2 CPUs free: using all of them is ~30% slower) | | 35 | | `threads` | `16` | CPU threads (keep 2 CPUs free: using all of them is ~30% slower) | |
| 36 | -| `ctx` | `32768` | Context size (shared by the 4 slots) | | 36 | +| `ctx` | `16384` | Context size (shared by the 4 slots) | |
| 37 | 37 | ||
| 38 | ```bash | 38 | ```bash |
| 39 | sbx env create sbxenv.yaml --env-arg model_file=Olala-v1.0-mini-7A1B-q5_k_m.gguf --env-arg host_port=9090 | 39 | sbx env create sbxenv.yaml --env-arg model_file=Olala-v1.0-mini-7A1B-q5_k_m.gguf --env-arg host_port=9090 |
modified
docs/olala-sbx.fr.md +1 -1 | @@ -33,7 +33,7 @@ Options (à repasser à chaque commande `sbx env` de ce sandbox) : | ||
| 33 | 33 | | `models_dir` | `./models` | Dossier des modèles (relatif à la racine du dépôt, ou absolu) | |
| 34 | 34 | | `host_port` | `8080` | Port de l'hôte | |
| 35 | 35 | | `threads` | `16` | Threads CPU (garde 2 CPU libres : tous les utiliser est ~30 % plus lent) | |
| 36 | -| `ctx` | `32768` | Taille du contexte (partagée par les 4 slots) | | |
| 36 | +| `ctx` | `16384` | Taille du contexte (partagée par les 4 slots) | | |
| 37 | 37 | |
| 38 | 38 | ```bash |
| 39 | 39 | sbx env create sbxenv.yaml --env-arg model_file=Olala-v1.0-mini-7A1B-q5_k_m.gguf --env-arg host_port=9090 |
| @@ -33,7 +33,7 @@ Options (à repasser à chaque commande `sbx env` de ce sandbox) : | |||
| 33 | | `models_dir` | `./models` | Dossier des modèles (relatif à la racine du dépôt, ou absolu) | | 33 | | `models_dir` | `./models` | Dossier des modèles (relatif à la racine du dépôt, ou absolu) | |
| 34 | | `host_port` | `8080` | Port de l'hôte | | 34 | | `host_port` | `8080` | Port de l'hôte | |
| 35 | | `threads` | `16` | Threads CPU (garde 2 CPU libres : tous les utiliser est ~30 % plus lent) | | 35 | | `threads` | `16` | Threads CPU (garde 2 CPU libres : tous les utiliser est ~30 % plus lent) | |
| 36 | -| `ctx` | `32768` | Taille du contexte (partagée par les 4 slots) | | 36 | +| `ctx` | `16384` | Taille du contexte (partagée par les 4 slots) | |
| 37 | 37 | ||
| 38 | ```bash | 38 | ```bash |
| 39 | sbx env create sbxenv.yaml --env-arg model_file=Olala-v1.0-mini-7A1B-q5_k_m.gguf --env-arg host_port=9090 | 39 | sbx env create sbxenv.yaml --env-arg model_file=Olala-v1.0-mini-7A1B-q5_k_m.gguf --env-arg host_port=9090 |
modified
sbxenv.yaml +1 -1 | @@ -44,7 +44,7 @@ args: | ||
| 44 | 44 | pattern: "[1-9][0-9]*" |
| 45 | 45 | description: CPU threads used by llama-server (leave a couple of CPUs free) |
| 46 | 46 | ctx: |
| 47 | - default: "32768" | |
| 47 | + default: "16384" | |
| 48 | 48 | pattern: "[1-9][0-9]*" |
| 49 | 49 | description: Context size (shared by the parallel slots) |
| 50 | 50 | |
| @@ -44,7 +44,7 @@ args: | |||
| 44 | pattern: "[1-9][0-9]*" | 44 | pattern: "[1-9][0-9]*" |
| 45 | description: CPU threads used by llama-server (leave a couple of CPUs free) | 45 | description: CPU threads used by llama-server (leave a couple of CPUs free) |
| 46 | ctx: | 46 | ctx: |
| 47 | - default: "32768" | 47 | + default: "16384" |
| 48 | pattern: "[1-9][0-9]*" | 48 | pattern: "[1-9][0-9]*" |
| 49 | description: Context size (shared by the parallel slots) | 49 | description: Context size (shared by the parallel slots) |
| 50 | 50 | ||