Six verbs, and the last of the nix
The justfile was thirteen recipes, each carrying its own copy of the same re-entry dance: test an environment variable, `nix develop` back into the same recipe if it is unset, do the work if it is set. Now it is six verbs — build, run, test, modal, serve, tools — and the three near-identical Flutter recipes are one private `_flutter` that differs only in which entry point it paints. The toolchain is `tools/toolchain.sh` for everything, not just the web build: Nim joins Flutter, the JDK and the Clojure CLI as a pinned tarball, and `toolchain.sh android` fetches Google's command-line tools on demand so Gradle has an SDK it is allowed to write to. That is what the APK needed a writable copy of the store for, and it is the last thing the flake was doing that a tarball could not. So flake.nix and flake.lock are gone, and `.modal/dev` — which baked `nix print-dev-env` into its image and entered `nix develop` at run time — is rebuilt on the same debian image and the same script `.modal/web` already used. `_loader.py` loses its `[nix]` support with them: the ptyshim, the devShell warming layer, the substituter plumbing, the `nix develop` wrapper. 139 lines out, 25 in. The containers lose the `flutter-` prefix too, this being a Flutter-only project: `just modal dev`, `just modal web`. Their directories on the devshell volume are renamed to match, so the first run after this starts from a cold toolchain once. The deployed app in `.modal/web/serve.py` keeps its name — it is the live URL, and freeq's auth broker allowlists that origin. Verified from a cold .toolchain: test common/nim/dart pass, build desktop and build ui both compile, and both container specs construct their images through the pruned loader. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2e24e64 parent: e505ded modified
.gitignore +3 -2 | @@ -11,7 +11,8 @@ | ||
| 11 | 11 | /nim/tests/t* |
| 12 | 12 | !/nim/tests/t*.nim |
| 13 | 13 | |
| 14 | -# The pinned Flutter/JDK/Clojure tarballs `tools/toolchain.sh` fetches, plus | |
| 14 | +# The pinned Flutter/JDK/Clojure/Nim tarballs and the Android SDK that | |
| 15 | +# `tools/toolchain.sh` fetches, plus | |
| 15 | 16 | # the pub cache, gitlibs and maven repo it keeps beside them. A gigabyte of |
| 16 | 17 | # SDK, reproducible from the hashes in that script. |
| 17 | 18 | /.toolchain/ |
| @@ -31,5 +32,5 @@ | ||
| 31 | 32 | # Python bytecode, from the loader the .modal/ containers import. |
| 32 | 33 | __pycache__/ |
| 33 | 34 | |
| 34 | -# The mirror `just web-local` keeps of the Modal-built web bundle. | |
| 35 | +# The mirror `just serve` keeps of the Modal-built web bundle. | |
| 35 | 36 | .web-local/ |
| @@ -11,7 +11,8 @@ | |||
| 11 | /nim/tests/t* | 11 | /nim/tests/t* |
| 12 | !/nim/tests/t*.nim | 12 | !/nim/tests/t*.nim |
| 13 | 13 | ||
| 14 | -# The pinned Flutter/JDK/Clojure tarballs `tools/toolchain.sh` fetches, plus | 14 | +# The pinned Flutter/JDK/Clojure/Nim tarballs and the Android SDK that |
| 15 | +# `tools/toolchain.sh` fetches, plus | ||
| 15 | # the pub cache, gitlibs and maven repo it keeps beside them. A gigabyte of | 16 | # the pub cache, gitlibs and maven repo it keeps beside them. A gigabyte of |
| 16 | # SDK, reproducible from the hashes in that script. | 17 | # SDK, reproducible from the hashes in that script. |
| 17 | /.toolchain/ | 18 | /.toolchain/ |
| @@ -31,5 +32,5 @@ | |||
| 31 | # Python bytecode, from the loader the .modal/ containers import. | 32 | # Python bytecode, from the loader the .modal/ containers import. |
| 32 | __pycache__/ | 33 | __pycache__/ |
| 33 | 34 | ||
| 34 | -# The mirror `just web-local` keeps of the Modal-built web bundle. | 35 | +# The mirror `just serve` keeps of the Modal-built web bundle. |
| 35 | .web-local/ | 36 | .web-local/ |
modified
.modal/_loader.py +25 -139 | @@ -2,23 +2,19 @@ | ||
| 2 | 2 | |
| 3 | 3 | Every container -- under `containers/` here, `.modal/` in a repo that merely |
| 4 | 4 | builds itself on Modal -- is a directory with a `container.toml` and |
| 5 | -a stub `container.py`. See `spec.md` for the keys; this file is what reads | |
| 6 | -them. Nothing here is Modal-specific configuration in its own right -- each | |
| 5 | +a stub `container.py`. The keys are documented by the comments below, which | |
| 6 | +is the whole of the spec. Nothing here is Modal-specific configuration in its | |
| 7 | +own right -- each | |
| 7 | 8 | spec key maps onto a documented Modal argument, and the mapping is meant to |
| 8 | 9 | stay boring enough to read straight through. |
| 9 | 10 | """ |
| 10 | 11 | |
| 11 | 12 | import os |
| 12 | -import shlex | |
| 13 | 13 | import subprocess |
| 14 | 14 | import tomllib |
| 15 | 15 | |
| 16 | 16 | import modal |
| 17 | 17 | |
| 18 | -REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) | |
| 19 | -PTYSHIM_C = os.path.join(REPO, "ptyshim.c") | |
| 20 | -SHIM_SO = "/opt/ptyshim.so" | |
| 21 | - | |
| 22 | 18 | |
| 23 | 19 | class SpecError(Exception): |
| 24 | 20 | """The container.toml says something that cannot be built.""" |
| @@ -44,34 +40,16 @@ class Container: | ||
| 44 | 40 | self.command = run.get("command", "") |
| 45 | 41 | self.env = dict(run.get("env", {})) |
| 46 | 42 | |
| 47 | - # Substituters reach nix through NIX_CONFIG rather than nix.conf, and | |
| 48 | - # at run time rather than build time. Written into the image's nix.conf | |
| 49 | - # instead, nix touches the path while the image is being built and | |
| 50 | - # creates it -- and Modal will not mount a volume over a non-empty | |
| 51 | - # directory, so the container that wanted the cache cannot start. As | |
| 52 | - # env it covers every nix command that runs in the container, the ones | |
| 53 | - # typed by hand in a shell included, and leaves the mount point empty. | |
| 54 | - if subs := list(spec.get("nix", {}).get("substituters", [])): | |
| 55 | - line = "extra-substituters = " + " ".join(subs) | |
| 56 | - self.env["NIX_CONFIG"] = ( | |
| 57 | - f"{self.env['NIX_CONFIG']}\n{line}" if "NIX_CONFIG" in self.env | |
| 58 | - else line | |
| 59 | - ) | |
| 60 | - | |
| 61 | 43 | # "function" (the default) or "sandbox". Sandboxes can run on a real |
| 62 | 44 | # VM, which Functions cannot -- see ../README.md. |
| 63 | 45 | self.runtime = spec.get("container", {}).get("runtime", "function") |
| 64 | 46 | |
| 65 | - nix = spec.get("nix", {}) | |
| 66 | - self.use_flake = bool(nix.get("flake", False)) | |
| 67 | - self.use_shim = bool(nix.get("shim", False)) | |
| 68 | - | |
| 69 | 47 | # name -> mount path. Modal Volumes, mounted while the container runs |
| 70 | 48 | # and NOT while its image is built: a volume mount is not part of the |
| 71 | 49 | # resulting image, so anything written to one during a build step is |
| 72 | 50 | # gone by the time the container starts. Persist across runs is the |
| 73 | - # whole point -- a nix store to substitute from, a cargo target | |
| 74 | - # directory, a dataset too big to bake in. | |
| 51 | + # whole point -- a toolchain too big to fetch every run, a build | |
| 52 | + # tree an incremental compile reuses, a dataset too big to bake in. | |
| 75 | 53 | self.volume_spec = dict(spec.get("volumes", {})) |
| 76 | 54 | |
| 77 | 55 | # Ports to tunnel out of a Sandbox, from [network] ports. Encrypted, |
| @@ -86,7 +64,7 @@ class Container: | ||
| 86 | 64 | |
| 87 | 65 | # Modal re-imports this module inside the container, so everything |
| 88 | 66 | # below runs twice: once here, once out there. Out there the local |
| 89 | - # tree does not exist -- no flake.nix, no ptyshim.c, no repo -- and | |
| 67 | + # tree does not exist -- no repo to copy from -- and | |
| 90 | 68 | # the image is already built, so validating and rebuilding it would |
| 91 | 69 | # only fail. The App still has to exist for the decorators to bind. |
| 92 | 70 | if self.is_remote: |
| @@ -124,17 +102,6 @@ class Container: | ||
| 124 | 102 | ) |
| 125 | 103 | if bool(c.get("base")) == bool(c.get("registry")): |
| 126 | 104 | raise SpecError("set exactly one of [container] base or registry") |
| 127 | - if self.use_flake: | |
| 128 | - if not os.path.exists(os.path.join(self.dir, "flake.nix")): | |
| 129 | - raise SpecError("[nix] flake = true but there is no flake.nix") | |
| 130 | - if not self.use_shim: | |
| 131 | - # Warming the shell builds nix-shell-env, which is the gVisor | |
| 132 | - # pty bug. Failing here beats failing ten minutes into a build. | |
| 133 | - raise SpecError( | |
| 134 | - "[nix] flake = true needs shim = true -- see ../README.md" | |
| 135 | - ) | |
| 136 | - if self.use_shim and not os.path.exists(PTYSHIM_C): | |
| 137 | - raise SpecError(f"[nix] shim = true but {PTYSHIM_C} is missing") | |
| 138 | 105 | if self.ports and self.runtime != "sandbox": |
| 139 | 106 | raise SpecError( |
| 140 | 107 | "[network] ports needs [container] runtime = \"sandbox\" --" |
| @@ -146,9 +113,9 @@ class Container: | ||
| 146 | 113 | f"[volumes] {name} must be an absolute path, not {mount!r}" |
| 147 | 114 | ) |
| 148 | 115 | # Mounting over one of these hides what the image already has |
| 149 | - # there -- /nix in particular, where the empty volume would shadow | |
| 150 | - # the store the base image spent its build populating. | |
| 151 | - if mount.rstrip("/") in ("", "/nix", "/nix/store", "/usr", "/etc"): | |
| 116 | + # there: an empty volume would shadow what the image's own build | |
| 117 | + # put in its place. | |
| 118 | + if mount.rstrip("/") in ("", "/usr", "/etc", "/bin", "/lib"): | |
| 152 | 119 | raise SpecError( |
| 153 | 120 | f"[volumes] {name} may not mount over {mount} --" |
| 154 | 121 | " it would hide what the image has there" |
| @@ -161,33 +128,6 @@ class Container: | ||
| 161 | 128 | |
| 162 | 129 | # -- image ----------------------------------------------------------- |
| 163 | 130 | |
| 164 | - @property | |
| 165 | - def nix(self) -> str: | |
| 166 | - """The `nix` command as the container runs it. | |
| 167 | - | |
| 168 | - A sandbox on a real VM has a working pty, so the shim buys nothing | |
| 169 | - there and is left off even when the image was built with it. | |
| 170 | - """ | |
| 171 | - if self.use_shim and not self.vm_at_runtime: | |
| 172 | - return f"LD_PRELOAD={SHIM_SO} nix" | |
| 173 | - return "nix" | |
| 174 | - | |
| 175 | - @property | |
| 176 | - def vm_at_runtime(self) -> bool: | |
| 177 | - """True when this container actually runs on a VM rather than gVisor.""" | |
| 178 | - return (self.runtime == "sandbox" | |
| 179 | - and bool(self.experimental_options.get("vm_runtime"))) | |
| 180 | - | |
| 181 | - @property | |
| 182 | - def build_nix(self) -> str: | |
| 183 | - """The `nix` command for BUILD steps, which always run under gVisor. | |
| 184 | - | |
| 185 | - Image builds are Functions underneath, so a sandbox container still | |
| 186 | - needs the shim while its image is being built -- only its run time | |
| 187 | - gets the VM. | |
| 188 | - """ | |
| 189 | - return f"LD_PRELOAD={SHIM_SO} nix" if self.use_shim else "nix" | |
| 190 | - | |
| 191 | 131 | def _build_image(self) -> modal.Image: |
| 192 | 132 | c = self.spec["container"] |
| 193 | 133 | build = self.spec.get("build", {}) |
| @@ -197,45 +137,6 @@ class Container: | ||
| 197 | 137 | else: |
| 198 | 138 | image = modal.Image.from_registry(c["registry"]) |
| 199 | 139 | |
| 200 | - # `nix profile install` puts things in ~/.nix-profile/bin, which is on | |
| 201 | - # nobody's PATH here -- so the install succeeds and the very next line | |
| 202 | - # says `command not found`. Set on the image rather than in a .bashrc, | |
| 203 | - # because a container's command runs under `sh -c` and reads neither. | |
| 204 | - # Spelled out rather than prefixed onto $PATH: an image env is a value, | |
| 205 | - # not a shell expression, so "$PATH" here would be four literal | |
| 206 | - # characters. | |
| 207 | - image = image.env({ | |
| 208 | - "PATH": "/root/.nix-profile/bin:/usr/local/sbin:/usr/local/bin" | |
| 209 | - ":/usr/sbin:/usr/bin:/sbin:/bin", | |
| 210 | - }) | |
| 211 | - | |
| 212 | - if self.use_shim: | |
| 213 | - image = image.add_local_file( | |
| 214 | - PTYSHIM_C, "/opt/ptyshim.c", copy=True | |
| 215 | - ).run_commands( | |
| 216 | - f"gcc -shared -fPIC -O2 -o {SHIM_SO} /opt/ptyshim.c -ldl" | |
| 217 | - ) | |
| 218 | - | |
| 219 | - # Warm the devShell BEFORE the source is copied in. Everything the | |
| 220 | - # shell needs is binary-cached, so the download happens once -- but | |
| 221 | - # only if this layer survives. Copy the source first and any edit to | |
| 222 | - # any file invalidates the warm, and the whole closure is fetched | |
| 223 | - # again on every build. Only flake.nix and flake.lock go in here, so | |
| 224 | - # the layer is invalidated by a dependency change and nothing else. | |
| 225 | - if self.use_flake: | |
| 226 | - for f in ("flake.nix", "flake.lock"): | |
| 227 | - path = os.path.join(self.dir, f) | |
| 228 | - if os.path.exists(path): | |
| 229 | - image = image.add_local_file( | |
| 230 | - path, f"{self.workdir}/{f}", copy=True | |
| 231 | - ) | |
| 232 | - image = image.run_commands( | |
| 233 | - f"cd {self.workdir} && {self.build_nix} develop" | |
| 234 | - " --accept-flake-config --command true", | |
| 235 | - f'echo "store paths after warming:' | |
| 236 | - f' $({self.build_nix} path-info --all | wc -l)"', | |
| 237 | - ) | |
| 238 | - | |
| 239 | 140 | # copy=True throughout: later run_commands need these files present. |
| 240 | 141 | # `context` is what include paths are relative to, and it may sit above |
| 241 | 142 | # the container directory -- a container that builds the repo it lives |
| @@ -244,19 +145,13 @@ class Container: | ||
| 244 | 145 | |
| 245 | 146 | # Image building and program building, kept apart. |
| 246 | 147 | # |
| 247 | - # `[build] commands` run AFTER the source copy, so any edit anywhere in | |
| 248 | - # the tree invalidates them -- and for a container whose commands warm | |
| 249 | - # a devShell, that means minutes of nix on every iteration of a | |
| 250 | - # one-line change. `warm` + `setup` are the same two steps moved in | |
| 251 | - # front of the source: `warm` names only the files the flake actually | |
| 252 | - # evaluates (its own, and whatever the devShell's derivations read), | |
| 253 | - # and `setup` runs against those alone. Editing `flutter/src` then | |
| 254 | - # invalidates nothing above the final copy, and the toolchain layer is | |
| 255 | - # reused until a dependency moves. | |
| 256 | - # | |
| 257 | - # Volumes are mounted here for `commands`' reason: a step that can | |
| 258 | - # reach the binary cache never has to build, which is the one thing | |
| 259 | - # gVisor will not do. | |
| 148 | + # `[build] commands` run AFTER the source copy, so any edit anywhere | |
| 149 | + # in the tree invalidates them. `warm` + `setup` are the same step | |
| 150 | + # moved in front of the source: `warm` names only the files the step | |
| 151 | + # actually reads, and `setup` runs against those alone, so editing | |
| 152 | + # `flutter/src` invalidates nothing above the final copy. A container | |
| 153 | + # whose setup is an `apt-get` needs no `warm` at all -- it reads | |
| 154 | + # nothing out of the tree, so nothing in the tree can invalidate it. | |
| 260 | 155 | for rel in build.get("warm", []): |
| 261 | 156 | src = os.path.normpath(os.path.join(context, rel)) |
| 262 | 157 | dest = f"{self.workdir}/{rel}" |
| @@ -281,21 +176,17 @@ class Container: | ||
| 281 | 176 | else: |
| 282 | 177 | image = image.add_local_file(src, dest, copy=True) |
| 283 | 178 | |
| 284 | - # A repo copied in brings its `.git` along, and in a worktree that is a | |
| 285 | - # *file* holding `gitdir: <path on the machine that copied it>`. Nix | |
| 286 | - # believes it and goes looking for a checkout that is not there -- | |
| 287 | - # `nix develop` and `nix build .#x` both die before evaluating | |
| 288 | - # anything. Nothing in a container wants the git metadata, so it goes. | |
| 179 | + # A repo copied in brings its `.git` along, and in a worktree that is | |
| 180 | + # a *file* holding `gitdir: <path on the machine that copied it>` -- | |
| 181 | + # which points at nothing out here, so any tool that follows it fails | |
| 182 | + # in a way that has nothing to do with what it was asked to do. | |
| 183 | + # Nothing in a container wants the git metadata, so it goes. | |
| 289 | 184 | image = image.run_commands(f"rm -rf {self.workdir}/.git") |
| 290 | 185 | |
| 291 | 186 | if commands := build.get("commands", []): |
| 292 | - # Volumes mounted for the build too, not just the run. A build step | |
| 293 | - # that wants nix to *build* something is the one thing gVisor will | |
| 294 | - # not do -- image builds are Functions underneath, and a derivation | |
| 295 | - # there dies on `unexpected EOF reading a line`. Substituting is | |
| 296 | - # fine, so a step that can reach the cache never has to build, and | |
| 297 | - # the shim stays retired. The mount is not part of the resulting | |
| 298 | - # image; only what the step writes outside it is. | |
| 187 | + # Volumes mounted for the build too, not just the run, so a step | |
| 188 | + # can read a cache the last run filled. The mount is not part of | |
| 189 | + # the resulting image; only what the step writes outside it is. | |
| 299 | 190 | image = image.run_commands(*commands, volumes=self.volumes) |
| 300 | 191 | |
| 301 | 192 | # container.py does `from _loader import Container`, and Modal mounts |
| @@ -386,15 +277,10 @@ class Container: | ||
| 386 | 277 | return kwargs |
| 387 | 278 | |
| 388 | 279 | def shell_command(self, override: str = "") -> str: |
| 389 | - """The full shell line the container runs, devShell wrapper included.""" | |
| 280 | + """The full shell line the container runs.""" | |
| 390 | 281 | command = override or self.command |
| 391 | 282 | if not command: |
| 392 | 283 | raise SpecError("container.toml has no [run] command") |
| 393 | - if self.use_flake: | |
| 394 | - return ( | |
| 395 | - f"cd {self.workdir} && {self.nix} develop --accept-flake-config" | |
| 396 | - f" --command sh -c {shlex.quote(command)}" | |
| 397 | - ) | |
| 398 | 284 | return f"cd {self.workdir} && {command}" |
| 399 | 285 | |
| 400 | 286 | def run_sandbox(self, override: str = "") -> str: |
| @@ -2,23 +2,19 @@ | |||
| 2 | 2 | ||
| 3 | Every container -- under `containers/` here, `.modal/` in a repo that merely | 3 | Every container -- under `containers/` here, `.modal/` in a repo that merely |
| 4 | builds itself on Modal -- is a directory with a `container.toml` and | 4 | builds itself on Modal -- is a directory with a `container.toml` and |
| 5 | -a stub `container.py`. See `spec.md` for the keys; this file is what reads | 5 | +a stub `container.py`. The keys are documented by the comments below, which |
| 6 | -them. Nothing here is Modal-specific configuration in its own right -- each | 6 | +is the whole of the spec. Nothing here is Modal-specific configuration in its |
| 7 | +own right -- each | ||
| 7 | spec key maps onto a documented Modal argument, and the mapping is meant to | 8 | spec key maps onto a documented Modal argument, and the mapping is meant to |
| 8 | stay boring enough to read straight through. | 9 | stay boring enough to read straight through. |
| 9 | """ | 10 | """ |
| 10 | 11 | ||
| 11 | import os | 12 | import os |
| 12 | -import shlex | ||
| 13 | import subprocess | 13 | import subprocess |
| 14 | import tomllib | 14 | import tomllib |
| 15 | 15 | ||
| 16 | import modal | 16 | import modal |
| 17 | 17 | ||
| 18 | -REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) | ||
| 19 | -PTYSHIM_C = os.path.join(REPO, "ptyshim.c") | ||
| 20 | -SHIM_SO = "/opt/ptyshim.so" | ||
| 21 | - | ||
| 22 | 18 | ||
| 23 | class SpecError(Exception): | 19 | class SpecError(Exception): |
| 24 | """The container.toml says something that cannot be built.""" | 20 | """The container.toml says something that cannot be built.""" |
| @@ -44,34 +40,16 @@ class Container: | |||
| 44 | self.command = run.get("command", "") | 40 | self.command = run.get("command", "") |
| 45 | self.env = dict(run.get("env", {})) | 41 | self.env = dict(run.get("env", {})) |
| 46 | 42 | ||
| 47 | - # Substituters reach nix through NIX_CONFIG rather than nix.conf, and | ||
| 48 | - # at run time rather than build time. Written into the image's nix.conf | ||
| 49 | - # instead, nix touches the path while the image is being built and | ||
| 50 | - # creates it -- and Modal will not mount a volume over a non-empty | ||
| 51 | - # directory, so the container that wanted the cache cannot start. As | ||
| 52 | - # env it covers every nix command that runs in the container, the ones | ||
| 53 | - # typed by hand in a shell included, and leaves the mount point empty. | ||
| 54 | - if subs := list(spec.get("nix", {}).get("substituters", [])): | ||
| 55 | - line = "extra-substituters = " + " ".join(subs) | ||
| 56 | - self.env["NIX_CONFIG"] = ( | ||
| 57 | - f"{self.env['NIX_CONFIG']}\n{line}" if "NIX_CONFIG" in self.env | ||
| 58 | - else line | ||
| 59 | - ) | ||
| 60 | - | ||
| 61 | # "function" (the default) or "sandbox". Sandboxes can run on a real | 43 | # "function" (the default) or "sandbox". Sandboxes can run on a real |
| 62 | # VM, which Functions cannot -- see ../README.md. | 44 | # VM, which Functions cannot -- see ../README.md. |
| 63 | self.runtime = spec.get("container", {}).get("runtime", "function") | 45 | self.runtime = spec.get("container", {}).get("runtime", "function") |
| 64 | 46 | ||
| 65 | - nix = spec.get("nix", {}) | ||
| 66 | - self.use_flake = bool(nix.get("flake", False)) | ||
| 67 | - self.use_shim = bool(nix.get("shim", False)) | ||
| 68 | - | ||
| 69 | # name -> mount path. Modal Volumes, mounted while the container runs | 47 | # name -> mount path. Modal Volumes, mounted while the container runs |
| 70 | # and NOT while its image is built: a volume mount is not part of the | 48 | # and NOT while its image is built: a volume mount is not part of the |
| 71 | # resulting image, so anything written to one during a build step is | 49 | # resulting image, so anything written to one during a build step is |
| 72 | # gone by the time the container starts. Persist across runs is the | 50 | # gone by the time the container starts. Persist across runs is the |
| 73 | - # whole point -- a nix store to substitute from, a cargo target | 51 | + # whole point -- a toolchain too big to fetch every run, a build |
| 74 | - # directory, a dataset too big to bake in. | 52 | + # tree an incremental compile reuses, a dataset too big to bake in. |
| 75 | self.volume_spec = dict(spec.get("volumes", {})) | 53 | self.volume_spec = dict(spec.get("volumes", {})) |
| 76 | 54 | ||
| 77 | # Ports to tunnel out of a Sandbox, from [network] ports. Encrypted, | 55 | # Ports to tunnel out of a Sandbox, from [network] ports. Encrypted, |
| @@ -86,7 +64,7 @@ class Container: | |||
| 86 | 64 | ||
| 87 | # Modal re-imports this module inside the container, so everything | 65 | # Modal re-imports this module inside the container, so everything |
| 88 | # below runs twice: once here, once out there. Out there the local | 66 | # below runs twice: once here, once out there. Out there the local |
| 89 | - # tree does not exist -- no flake.nix, no ptyshim.c, no repo -- and | 67 | + # tree does not exist -- no repo to copy from -- and |
| 90 | # the image is already built, so validating and rebuilding it would | 68 | # the image is already built, so validating and rebuilding it would |
| 91 | # only fail. The App still has to exist for the decorators to bind. | 69 | # only fail. The App still has to exist for the decorators to bind. |
| 92 | if self.is_remote: | 70 | if self.is_remote: |
| @@ -124,17 +102,6 @@ class Container: | |||
| 124 | ) | 102 | ) |
| 125 | if bool(c.get("base")) == bool(c.get("registry")): | 103 | if bool(c.get("base")) == bool(c.get("registry")): |
| 126 | raise SpecError("set exactly one of [container] base or registry") | 104 | raise SpecError("set exactly one of [container] base or registry") |
| 127 | - if self.use_flake: | ||
| 128 | - if not os.path.exists(os.path.join(self.dir, "flake.nix")): | ||
| 129 | - raise SpecError("[nix] flake = true but there is no flake.nix") | ||
| 130 | - if not self.use_shim: | ||
| 131 | - # Warming the shell builds nix-shell-env, which is the gVisor | ||
| 132 | - # pty bug. Failing here beats failing ten minutes into a build. | ||
| 133 | - raise SpecError( | ||
| 134 | - "[nix] flake = true needs shim = true -- see ../README.md" | ||
| 135 | - ) | ||
| 136 | - if self.use_shim and not os.path.exists(PTYSHIM_C): | ||
| 137 | - raise SpecError(f"[nix] shim = true but {PTYSHIM_C} is missing") | ||
| 138 | if self.ports and self.runtime != "sandbox": | 105 | if self.ports and self.runtime != "sandbox": |
| 139 | raise SpecError( | 106 | raise SpecError( |
| 140 | "[network] ports needs [container] runtime = \"sandbox\" --" | 107 | "[network] ports needs [container] runtime = \"sandbox\" --" |
| @@ -146,9 +113,9 @@ class Container: | |||
| 146 | f"[volumes] {name} must be an absolute path, not {mount!r}" | 113 | f"[volumes] {name} must be an absolute path, not {mount!r}" |
| 147 | ) | 114 | ) |
| 148 | # Mounting over one of these hides what the image already has | 115 | # Mounting over one of these hides what the image already has |
| 149 | - # there -- /nix in particular, where the empty volume would shadow | 116 | + # there: an empty volume would shadow what the image's own build |
| 150 | - # the store the base image spent its build populating. | 117 | + # put in its place. |
| 151 | - if mount.rstrip("/") in ("", "/nix", "/nix/store", "/usr", "/etc"): | 118 | + if mount.rstrip("/") in ("", "/usr", "/etc", "/bin", "/lib"): |
| 152 | raise SpecError( | 119 | raise SpecError( |
| 153 | f"[volumes] {name} may not mount over {mount} --" | 120 | f"[volumes] {name} may not mount over {mount} --" |
| 154 | " it would hide what the image has there" | 121 | " it would hide what the image has there" |
| @@ -161,33 +128,6 @@ class Container: | |||
| 161 | 128 | ||
| 162 | # -- image ----------------------------------------------------------- | 129 | # -- image ----------------------------------------------------------- |
| 163 | 130 | ||
| 164 | - @property | ||
| 165 | - def nix(self) -> str: | ||
| 166 | - """The `nix` command as the container runs it. | ||
| 167 | - | ||
| 168 | - A sandbox on a real VM has a working pty, so the shim buys nothing | ||
| 169 | - there and is left off even when the image was built with it. | ||
| 170 | - """ | ||
| 171 | - if self.use_shim and not self.vm_at_runtime: | ||
| 172 | - return f"LD_PRELOAD={SHIM_SO} nix" | ||
| 173 | - return "nix" | ||
| 174 | - | ||
| 175 | - @property | ||
| 176 | - def vm_at_runtime(self) -> bool: | ||
| 177 | - """True when this container actually runs on a VM rather than gVisor.""" | ||
| 178 | - return (self.runtime == "sandbox" | ||
| 179 | - and bool(self.experimental_options.get("vm_runtime"))) | ||
| 180 | - | ||
| 181 | - @property | ||
| 182 | - def build_nix(self) -> str: | ||
| 183 | - """The `nix` command for BUILD steps, which always run under gVisor. | ||
| 184 | - | ||
| 185 | - Image builds are Functions underneath, so a sandbox container still | ||
| 186 | - needs the shim while its image is being built -- only its run time | ||
| 187 | - gets the VM. | ||
| 188 | - """ | ||
| 189 | - return f"LD_PRELOAD={SHIM_SO} nix" if self.use_shim else "nix" | ||
| 190 | - | ||
| 191 | def _build_image(self) -> modal.Image: | 131 | def _build_image(self) -> modal.Image: |
| 192 | c = self.spec["container"] | 132 | c = self.spec["container"] |
| 193 | build = self.spec.get("build", {}) | 133 | build = self.spec.get("build", {}) |
| @@ -197,45 +137,6 @@ class Container: | |||
| 197 | else: | 137 | else: |
| 198 | image = modal.Image.from_registry(c["registry"]) | 138 | image = modal.Image.from_registry(c["registry"]) |
| 199 | 139 | ||
| 200 | - # `nix profile install` puts things in ~/.nix-profile/bin, which is on | ||
| 201 | - # nobody's PATH here -- so the install succeeds and the very next line | ||
| 202 | - # says `command not found`. Set on the image rather than in a .bashrc, | ||
| 203 | - # because a container's command runs under `sh -c` and reads neither. | ||
| 204 | - # Spelled out rather than prefixed onto $PATH: an image env is a value, | ||
| 205 | - # not a shell expression, so "$PATH" here would be four literal | ||
| 206 | - # characters. | ||
| 207 | - image = image.env({ | ||
| 208 | - "PATH": "/root/.nix-profile/bin:/usr/local/sbin:/usr/local/bin" | ||
| 209 | - ":/usr/sbin:/usr/bin:/sbin:/bin", | ||
| 210 | - }) | ||
| 211 | - | ||
| 212 | - if self.use_shim: | ||
| 213 | - image = image.add_local_file( | ||
| 214 | - PTYSHIM_C, "/opt/ptyshim.c", copy=True | ||
| 215 | - ).run_commands( | ||
| 216 | - f"gcc -shared -fPIC -O2 -o {SHIM_SO} /opt/ptyshim.c -ldl" | ||
| 217 | - ) | ||
| 218 | - | ||
| 219 | - # Warm the devShell BEFORE the source is copied in. Everything the | ||
| 220 | - # shell needs is binary-cached, so the download happens once -- but | ||
| 221 | - # only if this layer survives. Copy the source first and any edit to | ||
| 222 | - # any file invalidates the warm, and the whole closure is fetched | ||
| 223 | - # again on every build. Only flake.nix and flake.lock go in here, so | ||
| 224 | - # the layer is invalidated by a dependency change and nothing else. | ||
| 225 | - if self.use_flake: | ||
| 226 | - for f in ("flake.nix", "flake.lock"): | ||
| 227 | - path = os.path.join(self.dir, f) | ||
| 228 | - if os.path.exists(path): | ||
| 229 | - image = image.add_local_file( | ||
| 230 | - path, f"{self.workdir}/{f}", copy=True | ||
| 231 | - ) | ||
| 232 | - image = image.run_commands( | ||
| 233 | - f"cd {self.workdir} && {self.build_nix} develop" | ||
| 234 | - " --accept-flake-config --command true", | ||
| 235 | - f'echo "store paths after warming:' | ||
| 236 | - f' $({self.build_nix} path-info --all | wc -l)"', | ||
| 237 | - ) | ||
| 238 | - | ||
| 239 | # copy=True throughout: later run_commands need these files present. | 140 | # copy=True throughout: later run_commands need these files present. |
| 240 | # `context` is what include paths are relative to, and it may sit above | 141 | # `context` is what include paths are relative to, and it may sit above |
| 241 | # the container directory -- a container that builds the repo it lives | 142 | # the container directory -- a container that builds the repo it lives |
| @@ -244,19 +145,13 @@ class Container: | |||
| 244 | 145 | ||
| 245 | # Image building and program building, kept apart. | 146 | # Image building and program building, kept apart. |
| 246 | # | 147 | # |
| 247 | - # `[build] commands` run AFTER the source copy, so any edit anywhere in | 148 | + # `[build] commands` run AFTER the source copy, so any edit anywhere |
| 248 | - # the tree invalidates them -- and for a container whose commands warm | 149 | + # in the tree invalidates them. `warm` + `setup` are the same step |
| 249 | - # a devShell, that means minutes of nix on every iteration of a | 150 | + # moved in front of the source: `warm` names only the files the step |
| 250 | - # one-line change. `warm` + `setup` are the same two steps moved in | 151 | + # actually reads, and `setup` runs against those alone, so editing |
| 251 | - # front of the source: `warm` names only the files the flake actually | 152 | + # `flutter/src` invalidates nothing above the final copy. A container |
| 252 | - # evaluates (its own, and whatever the devShell's derivations read), | 153 | + # whose setup is an `apt-get` needs no `warm` at all -- it reads |
| 253 | - # and `setup` runs against those alone. Editing `flutter/src` then | 154 | + # nothing out of the tree, so nothing in the tree can invalidate it. |
| 254 | - # invalidates nothing above the final copy, and the toolchain layer is | ||
| 255 | - # reused until a dependency moves. | ||
| 256 | - # | ||
| 257 | - # Volumes are mounted here for `commands`' reason: a step that can | ||
| 258 | - # reach the binary cache never has to build, which is the one thing | ||
| 259 | - # gVisor will not do. | ||
| 260 | for rel in build.get("warm", []): | 155 | for rel in build.get("warm", []): |
| 261 | src = os.path.normpath(os.path.join(context, rel)) | 156 | src = os.path.normpath(os.path.join(context, rel)) |
| 262 | dest = f"{self.workdir}/{rel}" | 157 | dest = f"{self.workdir}/{rel}" |
| @@ -281,21 +176,17 @@ class Container: | |||
| 281 | else: | 176 | else: |
| 282 | image = image.add_local_file(src, dest, copy=True) | 177 | image = image.add_local_file(src, dest, copy=True) |
| 283 | 178 | ||
| 284 | - # A repo copied in brings its `.git` along, and in a worktree that is a | 179 | + # A repo copied in brings its `.git` along, and in a worktree that is |
| 285 | - # *file* holding `gitdir: <path on the machine that copied it>`. Nix | 180 | + # a *file* holding `gitdir: <path on the machine that copied it>` -- |
| 286 | - # believes it and goes looking for a checkout that is not there -- | 181 | + # which points at nothing out here, so any tool that follows it fails |
| 287 | - # `nix develop` and `nix build .#x` both die before evaluating | 182 | + # in a way that has nothing to do with what it was asked to do. |
| 288 | - # anything. Nothing in a container wants the git metadata, so it goes. | 183 | + # Nothing in a container wants the git metadata, so it goes. |
| 289 | image = image.run_commands(f"rm -rf {self.workdir}/.git") | 184 | image = image.run_commands(f"rm -rf {self.workdir}/.git") |
| 290 | 185 | ||
| 291 | if commands := build.get("commands", []): | 186 | if commands := build.get("commands", []): |
| 292 | - # Volumes mounted for the build too, not just the run. A build step | 187 | + # Volumes mounted for the build too, not just the run, so a step |
| 293 | - # that wants nix to *build* something is the one thing gVisor will | 188 | + # can read a cache the last run filled. The mount is not part of |
| 294 | - # not do -- image builds are Functions underneath, and a derivation | 189 | + # the resulting image; only what the step writes outside it is. |
| 295 | - # there dies on `unexpected EOF reading a line`. Substituting is | ||
| 296 | - # fine, so a step that can reach the cache never has to build, and | ||
| 297 | - # the shim stays retired. The mount is not part of the resulting | ||
| 298 | - # image; only what the step writes outside it is. | ||
| 299 | image = image.run_commands(*commands, volumes=self.volumes) | 190 | image = image.run_commands(*commands, volumes=self.volumes) |
| 300 | 191 | ||
| 301 | # container.py does `from _loader import Container`, and Modal mounts | 192 | # container.py does `from _loader import Container`, and Modal mounts |
| @@ -386,15 +277,10 @@ class Container: | |||
| 386 | return kwargs | 277 | return kwargs |
| 387 | 278 | ||
| 388 | def shell_command(self, override: str = "") -> str: | 279 | def shell_command(self, override: str = "") -> str: |
| 389 | - """The full shell line the container runs, devShell wrapper included.""" | 280 | + """The full shell line the container runs.""" |
| 390 | command = override or self.command | 281 | command = override or self.command |
| 391 | if not command: | 282 | if not command: |
| 392 | raise SpecError("container.toml has no [run] command") | 283 | raise SpecError("container.toml has no [run] command") |
| 393 | - if self.use_flake: | ||
| 394 | - return ( | ||
| 395 | - f"cd {self.workdir} && {self.nix} develop --accept-flake-config" | ||
| 396 | - f" --command sh -c {shlex.quote(command)}" | ||
| 397 | - ) | ||
| 398 | return f"cd {self.workdir} && {command}" | 284 | return f"cd {self.workdir} && {command}" |
| 399 | 285 | ||
| 400 | def run_sandbox(self, override: str = "") -> str: | 286 | def run_sandbox(self, override: str = "") -> str: |
added
.modal/dev/README.md +17 -0 | new file mode 100644 | ||
| @@ -0,0 +1,17 @@ | ||
| 1 | +# `dev` | |
| 2 | + | |
| 3 | + scripts/deploy dev | |
| 4 | + modal run .modal/dev/container.py | |
| 5 | + | |
| 6 | +Defined by `container.toml`; `../_loader.py` is what reads it, and | |
| 7 | +its comments are the spec. | |
| 8 | +Built on `debian:13-slim`. | |
| 9 | + | |
| 10 | +Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command | |
| 11 | +is the sandbox's own process, so it dies when the command exits -- | |
| 12 | +no idle window and nothing to tear down. Note the VM restrictions: | |
| 13 | +no GPU, and memory is exactly what `[resources] memory` asks for. | |
| 14 | + | |
| 15 | +No nix anywhere: the toolchain is `tools/toolchain.sh`, fetched by | |
| 16 | +pinned sha256 onto the `devshell` volume, so nothing is substituted | |
| 17 | +at build time and nothing is evaluated at start. | |
| new file mode 100644 | |||
| @@ -0,0 +1,17 @@ | |||
| 1 | +# `dev` | ||
| 2 | + | ||
| 3 | + scripts/deploy dev | ||
| 4 | + modal run .modal/dev/container.py | ||
| 5 | + | ||
| 6 | +Defined by `container.toml`; `../_loader.py` is what reads it, and | ||
| 7 | +its comments are the spec. | ||
| 8 | +Built on `debian:13-slim`. | ||
| 9 | + | ||
| 10 | +Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command | ||
| 11 | +is the sandbox's own process, so it dies when the command exits -- | ||
| 12 | +no idle window and nothing to tear down. Note the VM restrictions: | ||
| 13 | +no GPU, and memory is exactly what `[resources] memory` asks for. | ||
| 14 | + | ||
| 15 | +No nix anywhere: the toolchain is `tools/toolchain.sh`, fetched by | ||
| 16 | +pinned sha256 onto the `devshell` volume, so nothing is substituted | ||
| 17 | +at build time and nothing is evaluated at start. | ||
renamed
.modal/dev/container.py +5 -4 | similarity index 81% | ||
| rename from .modal/flutter-web/container.py | ||
| rename to .modal/dev/container.py | ||
| @@ -26,10 +26,11 @@ def main(command: str = "", shell: bool = False): | ||
| 26 | 26 | c.run_sandbox(command) |
| 27 | 27 | return |
| 28 | 28 | |
| 29 | - # `modal shell --image` only takes registry references, so it cannot be | |
| 30 | - # pointed at a published Modal image like arch-nix. Attaching to a running | |
| 31 | - # Sandbox can, and that Sandbox is this container: same image, same | |
| 32 | - # volumes, same env. | |
| 29 | + # `modal shell --image` takes a registry reference and nothing else, so | |
| 30 | + # it cannot be pointed at the image this container actually runs -- the | |
| 31 | + # one with its setup steps and its source copy in it. Attaching to a | |
| 32 | + # running Sandbox can, and that Sandbox is this container: same image, | |
| 33 | + # same volumes, same env. | |
| 33 | 34 | sb = c.open_sandbox() |
| 34 | 35 | print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}") |
| 35 | 36 | print(f" attach: modal shell {sb.object_id} (from another terminal)") |
| similarity index 81% | |||
| rename from .modal/flutter-web/container.py | |||
| rename to .modal/dev/container.py | |||
| @@ -26,10 +26,11 @@ def main(command: str = "", shell: bool = False): | |||
| 26 | c.run_sandbox(command) | 26 | c.run_sandbox(command) |
| 27 | return | 27 | return |
| 28 | 28 | ||
| 29 | - # `modal shell --image` only takes registry references, so it cannot be | 29 | + # `modal shell --image` takes a registry reference and nothing else, so |
| 30 | - # pointed at a published Modal image like arch-nix. Attaching to a running | 30 | + # it cannot be pointed at the image this container actually runs -- the |
| 31 | - # Sandbox can, and that Sandbox is this container: same image, same | 31 | + # one with its setup steps and its source copy in it. Attaching to a |
| 32 | - # volumes, same env. | 32 | + # running Sandbox can, and that Sandbox is this container: same image, |
| 33 | + # same volumes, same env. | ||
| 33 | sb = c.open_sandbox() | 34 | sb = c.open_sandbox() |
| 34 | print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}") | 35 | print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}") |
| 35 | print(f" attach: modal shell {sb.object_id} (from another terminal)") | 36 | print(f" attach: modal shell {sb.object_id} (from another terminal)") |
added
.modal/dev/container.toml +140 -0 | new file mode 100644 | ||
| @@ -0,0 +1,140 @@ | ||
| 1 | +[container] | |
| 2 | +name = "frq-dev" | |
| 3 | +description = "the Flutter desktop build, incremental, from a pinned toolchain" | |
| 4 | +# `debian:13-slim` and not `arch-nix`: there is no nix in this container any | |
| 5 | +# more, and no flake left in the tree for it to enter. The build is | |
| 6 | +# `just build desktop`, which gets Flutter, a JDK, the Clojure CLI and Nim out | |
| 7 | +# of `tools/toolchain.sh` by pinned sha256 -- so what the image owes it is the | |
| 8 | +# C and GTK half Flutter's Linux target links against, and nothing else. | |
| 9 | +registry = "debian:13-slim" | |
| 10 | +# A Sandbox, not a Function: runs on a real VM, and the command is | |
| 11 | +# the sandbox's own process so it dies when the command does. | |
| 12 | +runtime = "sandbox" | |
| 13 | + | |
| 14 | +[build] | |
| 15 | +# The container lives inside the repo it builds, so the copy is rooted two | |
| 16 | +# levels up and `.` is the whole tree. | |
| 17 | +context = "../.." | |
| 18 | +include = ["."] | |
| 19 | +# The whole image build, and it is one apt line. What used to be here -- a nix | |
| 20 | +# store to populate and a devShell to print into /etc/devshell.sh, so that a | |
| 21 | +# shell attached to this container *was* the devShell -- is gone with the nix | |
| 22 | +# it was for. The toolchain script now plays that part, and it lives on the | |
| 23 | +# volume rather than in a layer. | |
| 24 | +# | |
| 25 | +# The first half is what the toolchain itself needs: git, because Flutter | |
| 26 | +# shells out to it against its own SDK checkout and refuses to run without | |
| 27 | +# one; unzip and xz-utils for the tarballs; ca-certificates so curl can verify | |
| 28 | +# what it fetches; rsync for the sync below. | |
| 29 | +# | |
| 30 | +# The second half is Flutter's Linux target: CMake, Ninja and pkg-config drive | |
| 31 | +# the build, GTK 3 is what the runner links, and a C toolchain compiles both | |
| 32 | +# that and whatever `nim c` is asked for. libssl is Nim's: nim.cfg is | |
| 33 | +# `-d:ssl`, and std/net resolves -lssl and -lcrypto through dynlib at run time. | |
| 34 | +setup = [ | |
| 35 | + "apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git rsync tar unzip xz-utils && rm -rf /var/lib/apt/lists/*", | |
| 36 | + "apt-get update && apt-get install -y --no-install-recommends build-essential clang cmake ninja-build pkg-config libgtk-3-dev liblzma-dev libssl-dev && rm -rf /var/lib/apt/lists/*", | |
| 37 | +] | |
| 38 | +# The build state a local checkout carries: 395MB of a 441MB repo, uploaded on | |
| 39 | +# every start and wanted by nothing out there. Flutter builds into a volume of | |
| 40 | +# its own, and the clojure caches are this machine's. | |
| 41 | +ignore = [ | |
| 42 | + "flutter/build", "flutter/.home", "flutter/.dart_tool", | |
| 43 | + "flutter/.clojuredart", "flutter/.cpcache", | |
| 44 | + # The ClojureDart compiler's output. Uploading a laptop's copy would make | |
| 45 | + # the rsync below overwrite the one the last container compiled, and every | |
| 46 | + # file whose content differed would look new to the compiler and to | |
| 47 | + # Flutter -- the incremental build undone by the thing meant to feed it. | |
| 48 | + "flutter/lib/cljd-out", | |
| 49 | + # The toolchain, which is a gigabyte of Flutter SDK and lives on the | |
| 50 | + # volume out here. | |
| 51 | + ".toolchain", | |
| 52 | + ".cpcache", "result", "build", ".git", | |
| 53 | + # An editor's linter rewrites this while the upload is reading it, and | |
| 54 | + # Modal fails the whole run with "was modified during build process". | |
| 55 | + ".clj-kondo", | |
| 56 | +] | |
| 57 | + | |
| 58 | +# One volume now, where there were two: the nix binary cache went with nix. | |
| 59 | +# `devshell` is the working state of an incremental loop, shared by every | |
| 60 | +# container that has one -- each gets its own directory under it, named for | |
| 61 | +# what it belongs to, so `web` and `flutter-desktop` never write the | |
| 62 | +# same tree. Modal Volumes have no locking, so those directory names are the | |
| 63 | +# only thing keeping them apart, and two runs of the *same* container must not | |
| 64 | +# overlap. | |
| 65 | +[volumes] | |
| 66 | +devshell = "/devshell" | |
| 67 | + | |
| 68 | +[resources] | |
| 69 | +cpu = 8 | |
| 70 | +memory = 16384 | |
| 71 | +timeout = 3600 | |
| 72 | + | |
| 73 | +[run] | |
| 74 | +workdir = "/app" | |
| 75 | +# Source in, toolchain out of the volume, build in place. None of it | |
| 76 | +# evaluates anything: the old command spent its first minutes entering a | |
| 77 | +# devShell, printing an environment, caching that environment against | |
| 78 | +# flake.lock and copying a nix closure back afterwards, all to arrive at a | |
| 79 | +# PATH. A PATH is what `tools/toolchain.sh env` prints, out of a directory | |
| 80 | +# already on the volume. | |
| 81 | +# | |
| 82 | +# An incremental build is the point, and `nix build` could not give one: a | |
| 83 | +# derivation is all-or-nothing, so any edit is a fresh sandbox and a fresh | |
| 84 | +# compile of everything. Here the toolchain supplies the compiler and | |
| 85 | +# `flutter build` decides what is stale -- which is the whole reason | |
| 86 | +# `just build desktop` exists as the working-tree loop. | |
| 87 | +command = """ | |
| 88 | +set -e | |
| 89 | +SHELL_DIR=/devshell/frq-desktop | |
| 90 | + | |
| 91 | +# Beside the working tree and NOT inside it: the rsync below runs with | |
| 92 | +# --delete, so anything under $SHELL_DIR that is not in /app is removed on | |
| 93 | +# every run. A cache kept in there would be deleted moments before it was | |
| 94 | +# consulted. | |
| 95 | +export FRQ_TOOLCHAIN=/devshell/frq-desktop.toolchain | |
| 96 | +mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN" | |
| 97 | + | |
| 98 | +echo "sync: /app -> $SHELL_DIR" | |
| 99 | +# rsync and not cp, with --checksum and not mtimes: Modal copies the source in | |
| 100 | +# with fresh timestamps every run, so a plain copy looks entirely new to | |
| 101 | +# Flutter and rebuilds the lot. --checksum compares content and leaves the | |
| 102 | +# unchanged files' timestamps alone, which is the whole basis of the | |
| 103 | +# incremental build. | |
| 104 | +# | |
| 105 | +# The excludes are the state we are here to keep -- overwriting them from /app | |
| 106 | +# would defeat the volume. | |
| 107 | +rsync -a --checksum --delete \ | |
| 108 | + --exclude 'flutter/.home/' \ | |
| 109 | + --exclude 'flutter/.clojuredart/' \ | |
| 110 | + --exclude 'flutter/build/' \ | |
| 111 | + --exclude 'flutter/lib/cljd-out/' \ | |
| 112 | + --exclude 'flutter/.dart_tool/' \ | |
| 113 | + --exclude '.toolchain/' \ | |
| 114 | + --exclude '.git' \ | |
| 115 | + /app/ "$SHELL_DIR/" | |
| 116 | + | |
| 117 | +cd "$SHELL_DIR" | |
| 118 | +# What survived from the last run, by presence and not by size: `du` here | |
| 119 | +# walked the pub cache, the toolchain and every object of the last build over | |
| 120 | +# a network volume, for numbers nobody acts on. | |
| 121 | +for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/build; do | |
| 122 | + [ -d "$d" ] && echo " carried over: $d" | |
| 123 | +done | |
| 124 | + | |
| 125 | +# Run from the volume, where the state it reuses lives. `just` is not in this | |
| 126 | +# image and is not worth an apt line for one call: the recipe is a wrapper | |
| 127 | +# around the toolchain, and this is that wrapper. | |
| 128 | +tools/toolchain.sh exec -- bash -euo pipefail -c ' | |
| 129 | + cd flutter | |
| 130 | + clojure -Sdeps "{:mvn/local-repo \\"$FRQ_M2\\"}" -M:cljd compile | |
| 131 | + flutter build linux --debug' | |
| 132 | + | |
| 133 | +echo "built:" | |
| 134 | +du -sh flutter/build | |
| 135 | +""" | |
| 136 | +# Flutter keeps its settings under XDG_CONFIG_HOME and its own caches under | |
| 137 | +# XDG_CACHE_HOME. Both point into the volume so a second run finds what the | |
| 138 | +# first one decided. Set here rather than in the command so a shell into this | |
| 139 | +# container gets them too. | |
| 140 | +env = { XDG_CACHE_HOME = "/devshell/frq-desktop.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-desktop.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-desktop.toolchain" } | |
| new file mode 100644 | |||
| @@ -0,0 +1,140 @@ | |||
| 1 | +[container] | ||
| 2 | +name = "frq-dev" | ||
| 3 | +description = "the Flutter desktop build, incremental, from a pinned toolchain" | ||
| 4 | +# `debian:13-slim` and not `arch-nix`: there is no nix in this container any | ||
| 5 | +# more, and no flake left in the tree for it to enter. The build is | ||
| 6 | +# `just build desktop`, which gets Flutter, a JDK, the Clojure CLI and Nim out | ||
| 7 | +# of `tools/toolchain.sh` by pinned sha256 -- so what the image owes it is the | ||
| 8 | +# C and GTK half Flutter's Linux target links against, and nothing else. | ||
| 9 | +registry = "debian:13-slim" | ||
| 10 | +# A Sandbox, not a Function: runs on a real VM, and the command is | ||
| 11 | +# the sandbox's own process so it dies when the command does. | ||
| 12 | +runtime = "sandbox" | ||
| 13 | + | ||
| 14 | +[build] | ||
| 15 | +# The container lives inside the repo it builds, so the copy is rooted two | ||
| 16 | +# levels up and `.` is the whole tree. | ||
| 17 | +context = "../.." | ||
| 18 | +include = ["."] | ||
| 19 | +# The whole image build, and it is one apt line. What used to be here -- a nix | ||
| 20 | +# store to populate and a devShell to print into /etc/devshell.sh, so that a | ||
| 21 | +# shell attached to this container *was* the devShell -- is gone with the nix | ||
| 22 | +# it was for. The toolchain script now plays that part, and it lives on the | ||
| 23 | +# volume rather than in a layer. | ||
| 24 | +# | ||
| 25 | +# The first half is what the toolchain itself needs: git, because Flutter | ||
| 26 | +# shells out to it against its own SDK checkout and refuses to run without | ||
| 27 | +# one; unzip and xz-utils for the tarballs; ca-certificates so curl can verify | ||
| 28 | +# what it fetches; rsync for the sync below. | ||
| 29 | +# | ||
| 30 | +# The second half is Flutter's Linux target: CMake, Ninja and pkg-config drive | ||
| 31 | +# the build, GTK 3 is what the runner links, and a C toolchain compiles both | ||
| 32 | +# that and whatever `nim c` is asked for. libssl is Nim's: nim.cfg is | ||
| 33 | +# `-d:ssl`, and std/net resolves -lssl and -lcrypto through dynlib at run time. | ||
| 34 | +setup = [ | ||
| 35 | + "apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git rsync tar unzip xz-utils && rm -rf /var/lib/apt/lists/*", | ||
| 36 | + "apt-get update && apt-get install -y --no-install-recommends build-essential clang cmake ninja-build pkg-config libgtk-3-dev liblzma-dev libssl-dev && rm -rf /var/lib/apt/lists/*", | ||
| 37 | +] | ||
| 38 | +# The build state a local checkout carries: 395MB of a 441MB repo, uploaded on | ||
| 39 | +# every start and wanted by nothing out there. Flutter builds into a volume of | ||
| 40 | +# its own, and the clojure caches are this machine's. | ||
| 41 | +ignore = [ | ||
| 42 | + "flutter/build", "flutter/.home", "flutter/.dart_tool", | ||
| 43 | + "flutter/.clojuredart", "flutter/.cpcache", | ||
| 44 | + # The ClojureDart compiler's output. Uploading a laptop's copy would make | ||
| 45 | + # the rsync below overwrite the one the last container compiled, and every | ||
| 46 | + # file whose content differed would look new to the compiler and to | ||
| 47 | + # Flutter -- the incremental build undone by the thing meant to feed it. | ||
| 48 | + "flutter/lib/cljd-out", | ||
| 49 | + # The toolchain, which is a gigabyte of Flutter SDK and lives on the | ||
| 50 | + # volume out here. | ||
| 51 | + ".toolchain", | ||
| 52 | + ".cpcache", "result", "build", ".git", | ||
| 53 | + # An editor's linter rewrites this while the upload is reading it, and | ||
| 54 | + # Modal fails the whole run with "was modified during build process". | ||
| 55 | + ".clj-kondo", | ||
| 56 | +] | ||
| 57 | + | ||
| 58 | +# One volume now, where there were two: the nix binary cache went with nix. | ||
| 59 | +# `devshell` is the working state of an incremental loop, shared by every | ||
| 60 | +# container that has one -- each gets its own directory under it, named for | ||
| 61 | +# what it belongs to, so `web` and `flutter-desktop` never write the | ||
| 62 | +# same tree. Modal Volumes have no locking, so those directory names are the | ||
| 63 | +# only thing keeping them apart, and two runs of the *same* container must not | ||
| 64 | +# overlap. | ||
| 65 | +[volumes] | ||
| 66 | +devshell = "/devshell" | ||
| 67 | + | ||
| 68 | +[resources] | ||
| 69 | +cpu = 8 | ||
| 70 | +memory = 16384 | ||
| 71 | +timeout = 3600 | ||
| 72 | + | ||
| 73 | +[run] | ||
| 74 | +workdir = "/app" | ||
| 75 | +# Source in, toolchain out of the volume, build in place. None of it | ||
| 76 | +# evaluates anything: the old command spent its first minutes entering a | ||
| 77 | +# devShell, printing an environment, caching that environment against | ||
| 78 | +# flake.lock and copying a nix closure back afterwards, all to arrive at a | ||
| 79 | +# PATH. A PATH is what `tools/toolchain.sh env` prints, out of a directory | ||
| 80 | +# already on the volume. | ||
| 81 | +# | ||
| 82 | +# An incremental build is the point, and `nix build` could not give one: a | ||
| 83 | +# derivation is all-or-nothing, so any edit is a fresh sandbox and a fresh | ||
| 84 | +# compile of everything. Here the toolchain supplies the compiler and | ||
| 85 | +# `flutter build` decides what is stale -- which is the whole reason | ||
| 86 | +# `just build desktop` exists as the working-tree loop. | ||
| 87 | +command = """ | ||
| 88 | +set -e | ||
| 89 | +SHELL_DIR=/devshell/frq-desktop | ||
| 90 | + | ||
| 91 | +# Beside the working tree and NOT inside it: the rsync below runs with | ||
| 92 | +# --delete, so anything under $SHELL_DIR that is not in /app is removed on | ||
| 93 | +# every run. A cache kept in there would be deleted moments before it was | ||
| 94 | +# consulted. | ||
| 95 | +export FRQ_TOOLCHAIN=/devshell/frq-desktop.toolchain | ||
| 96 | +mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN" | ||
| 97 | + | ||
| 98 | +echo "sync: /app -> $SHELL_DIR" | ||
| 99 | +# rsync and not cp, with --checksum and not mtimes: Modal copies the source in | ||
| 100 | +# with fresh timestamps every run, so a plain copy looks entirely new to | ||
| 101 | +# Flutter and rebuilds the lot. --checksum compares content and leaves the | ||
| 102 | +# unchanged files' timestamps alone, which is the whole basis of the | ||
| 103 | +# incremental build. | ||
| 104 | +# | ||
| 105 | +# The excludes are the state we are here to keep -- overwriting them from /app | ||
| 106 | +# would defeat the volume. | ||
| 107 | +rsync -a --checksum --delete \ | ||
| 108 | + --exclude 'flutter/.home/' \ | ||
| 109 | + --exclude 'flutter/.clojuredart/' \ | ||
| 110 | + --exclude 'flutter/build/' \ | ||
| 111 | + --exclude 'flutter/lib/cljd-out/' \ | ||
| 112 | + --exclude 'flutter/.dart_tool/' \ | ||
| 113 | + --exclude '.toolchain/' \ | ||
| 114 | + --exclude '.git' \ | ||
| 115 | + /app/ "$SHELL_DIR/" | ||
| 116 | + | ||
| 117 | +cd "$SHELL_DIR" | ||
| 118 | +# What survived from the last run, by presence and not by size: `du` here | ||
| 119 | +# walked the pub cache, the toolchain and every object of the last build over | ||
| 120 | +# a network volume, for numbers nobody acts on. | ||
| 121 | +for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/build; do | ||
| 122 | + [ -d "$d" ] && echo " carried over: $d" | ||
| 123 | +done | ||
| 124 | + | ||
| 125 | +# Run from the volume, where the state it reuses lives. `just` is not in this | ||
| 126 | +# image and is not worth an apt line for one call: the recipe is a wrapper | ||
| 127 | +# around the toolchain, and this is that wrapper. | ||
| 128 | +tools/toolchain.sh exec -- bash -euo pipefail -c ' | ||
| 129 | + cd flutter | ||
| 130 | + clojure -Sdeps "{:mvn/local-repo \\"$FRQ_M2\\"}" -M:cljd compile | ||
| 131 | + flutter build linux --debug' | ||
| 132 | + | ||
| 133 | +echo "built:" | ||
| 134 | +du -sh flutter/build | ||
| 135 | +""" | ||
| 136 | +# Flutter keeps its settings under XDG_CONFIG_HOME and its own caches under | ||
| 137 | +# XDG_CACHE_HOME. Both point into the volume so a second run finds what the | ||
| 138 | +# first one decided. Set here rather than in the command so a shell into this | ||
| 139 | +# container gets them too. | ||
| 140 | +env = { XDG_CACHE_HOME = "/devshell/frq-desktop.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-desktop.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-desktop.toolchain" } | ||
deleted
.modal/flutter-dev/README.md +0 -16 | deleted file mode 100644 | ||
| @@ -1,16 +0,0 @@ | ||
| 1 | -# `flutter-dev` | |
| 2 | - | |
| 3 | - scripts/deploy flutter-dev | |
| 4 | - modal run .modal/flutter-dev/container.py | |
| 5 | - | |
| 6 | -Defined by `container.toml`; see `../spec.md` for the keys. | |
| 7 | -Built on the published `arch-nix` image. | |
| 8 | - | |
| 9 | -Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command | |
| 10 | -is the sandbox's own process, so it dies when the command exits -- | |
| 11 | -no idle window and nothing to tear down. Note the VM restrictions: | |
| 12 | -no GPU, and memory is exactly what `[resources] memory` asks for. | |
| 13 | - | |
| 14 | -No nix at run time: nothing is substituted at build time and nothing | |
| 15 | -is evaluated at start. Add a `flake.nix` and set `[nix] flake`/`shim` | |
| 16 | -together if you want a devShell, knowing what it costs. | |
| deleted file mode 100644 | |||
| @@ -1,16 +0,0 @@ | |||
| 1 | -# `flutter-dev` | ||
| 2 | - | ||
| 3 | - scripts/deploy flutter-dev | ||
| 4 | - modal run .modal/flutter-dev/container.py | ||
| 5 | - | ||
| 6 | -Defined by `container.toml`; see `../spec.md` for the keys. | ||
| 7 | -Built on the published `arch-nix` image. | ||
| 8 | - | ||
| 9 | -Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command | ||
| 10 | -is the sandbox's own process, so it dies when the command exits -- | ||
| 11 | -no idle window and nothing to tear down. Note the VM restrictions: | ||
| 12 | -no GPU, and memory is exactly what `[resources] memory` asks for. | ||
| 13 | - | ||
| 14 | -No nix at run time: nothing is substituted at build time and nothing | ||
| 15 | -is evaluated at start. Add a `flake.nix` and set `[nix] flake`/`shim` | ||
| 16 | -together if you want a devShell, knowing what it costs. | ||
deleted
.modal/flutter-dev/container.toml +0 -160 | deleted file mode 100644 | ||
| @@ -1,160 +0,0 @@ | ||
| 1 | -[container] | |
| 2 | -name = "frq-flutter-dev" | |
| 3 | -description = "the Flutter desktop build, incremental, in a nix devShell" | |
| 4 | -base = "arch-nix" | |
| 5 | -# A Sandbox, not a Function: runs on a real VM, and the command is | |
| 6 | -# the sandbox's own process so it dies when the command does. | |
| 7 | -runtime = "sandbox" | |
| 8 | - | |
| 9 | -[build] | |
| 10 | -# The container lives inside the repo it builds, so the copy is rooted two | |
| 11 | -# levels up and `.` is the whole tree. | |
| 12 | -context = "../.." | |
| 13 | -include = ["."] | |
| 14 | -# `dev` is the shell you actually want. There is no default shell in this | |
| 15 | -# flake any more -- the one that used to be there belonged to the retired | |
| 16 | -# libcosmic frontend -- so a bare `nix develop` here fails rather than | |
| 17 | -# resolving to the wrong tree. | |
| 18 | -# The devShell, baked in rather than entered. `print-dev-env` writes the whole | |
| 19 | -# environment out as shell -- PATH, the compiler, every variable mkShell sets | |
| 20 | -# -- and realises its inputs on the way, so the closure becomes an image layer | |
| 21 | -# instead of a fetch every container pays for. Sourcing it from .bashrc means a | |
| 22 | -# shell attached to this container *is* the devShell: no `nix develop`, no | |
| 23 | -# clone of the flake's git inputs, no wait. | |
| 24 | -# | |
| 25 | -# `dev` stays for the case where the baked env is stale against a flake edit. | |
| 26 | -commands = [ | |
| 27 | - "nix print-dev-env /app#flutter-desktop --accept-flake-config --extra-substituters file:///nix-cache > /etc/devshell.sh", | |
| 28 | - "echo '. /etc/devshell.sh' >> /root/.bashrc", | |
| 29 | - "printf '#!/bin/sh\\nexec nix develop /app#flutter-desktop \"$@\"\\n' > /usr/local/bin/dev && chmod +x /usr/local/bin/dev", | |
| 30 | -] | |
| 31 | -# The build state a local checkout carries: 395MB of a 441MB repo, uploaded on | |
| 32 | -# every start and wanted by nothing out there. Flutter builds into a volume of | |
| 33 | -# its own, and the clojure caches are this machine's. | |
| 34 | -ignore = [ | |
| 35 | - "flutter/build", "flutter/.home", "flutter/.dart_tool", | |
| 36 | - "flutter/.clojuredart", "flutter/.cpcache", | |
| 37 | - ".cpcache", "result", "build", ".git", | |
| 38 | -] | |
| 39 | - | |
| 40 | -# Two volumes doing two different jobs. `nix-cache` is the binary cache every | |
| 41 | -# container here reads from and writes back to. `devshell` is the working | |
| 42 | -# state of a `nix develop` loop, and it is shared by every container that has | |
| 43 | -# one -- each gets its own directory under it, named for the devShell it | |
| 44 | -# belongs to, so two projects (or two shells of one project) never write the | |
| 45 | -# same tree. Modal Volumes have no locking, so the directories are the only | |
| 46 | -# thing keeping them apart, and two runs of the *same* devshell must not | |
| 47 | -# overlap. | |
| 48 | -[volumes] | |
| 49 | -nix-cache = "/nix-cache" | |
| 50 | -devshell = "/devshell" | |
| 51 | - | |
| 52 | -[resources] | |
| 53 | -cpu = 8 | |
| 54 | -memory = 16384 | |
| 55 | -timeout = 3600 | |
| 56 | - | |
| 57 | -[run] | |
| 58 | -workdir = "/app" | |
| 59 | -# Nix for the dependencies, the ordinary toolchain for the build. `nix build` | |
| 60 | -# cannot do this: a derivation is all-or-nothing, so any edit is a fresh | |
| 61 | -# sandbox and a fresh compile of everything. Here the devShell supplies the | |
| 62 | -# compiler and the libraries, and `flutter build` decides what is stale -- | |
| 63 | -# which is the whole reason `just flutter-desktop` exists as the working-tree | |
| 64 | -# loop rather than as another `nix build`. | |
| 65 | -# | |
| 66 | -# rsync and not cp, with --checksum and not mtimes: Modal copies the source in | |
| 67 | -# with fresh timestamps on every run, so a plain copy would look entirely new | |
| 68 | -# to Flutter and rebuild the lot. --checksum compares content, leaves the | |
| 69 | -# unchanged files' timestamps alone, and lets the incremental build work. | |
| 70 | -# | |
| 71 | -# The excludes are the state that must NOT be overwritten from /app -- it is | |
| 72 | -# what we are here to keep. `just flutter-desktop` seeds those caches only | |
| 73 | -# when they are missing, so finding them warm is all it takes. | |
| 74 | -command = """ | |
| 75 | -set -e | |
| 76 | -# This container's own directory on the shared devshell volume, named for the | |
| 77 | -# devShell it keeps the state of. Anything else using this volume picks its | |
| 78 | -# own name and the two never meet. | |
| 79 | -SHELL_DIR=/devshell/frq-flutter-desktop | |
| 80 | -mkdir -p "$SHELL_DIR" "$SHELL_DIR/.cache" | |
| 81 | - | |
| 82 | -# A worktree's `.git` is a *file* naming a gitdir back on the machine that | |
| 83 | -# copied it in, and nix believes it and goes looking for a path that is not | |
| 84 | -# here. It has to go before any flake reference to /app. | |
| 85 | -rm -rf /app/.git | |
| 86 | - | |
| 87 | -echo "sync: /app -> $SHELL_DIR" | |
| 88 | -# `nix shell --command` and not `nix profile install`: a profile install puts | |
| 89 | -# rsync in ~/.nix-profile/bin, which is not on the PATH of the shell already | |
| 90 | -# running, so the very next line said `rsync: command not found`. | |
| 91 | -# | |
| 92 | -# rsync and not cp, with --checksum and not mtimes: Modal copies the source in | |
| 93 | -# with fresh timestamps every run, so a plain copy looks entirely new to | |
| 94 | -# Flutter and rebuilds the lot. --checksum compares content and leaves the | |
| 95 | -# unchanged files' timestamps alone, which is the whole basis of the | |
| 96 | -# incremental build. | |
| 97 | -# | |
| 98 | -# The excludes are the state we are here to keep -- overwriting them from /app | |
| 99 | -# would defeat the volume. `just flutter-desktop` seeds those caches only when | |
| 100 | -# they are missing, so finding them warm is all it takes. | |
| 101 | -nix shell nixpkgs#rsync --accept-flake-config \ | |
| 102 | - --extra-substituters file:///nix-cache --command \ | |
| 103 | - rsync -a --checksum --delete \ | |
| 104 | - --exclude 'flutter/.home/' \ | |
| 105 | - --exclude 'flutter/.clojuredart/' \ | |
| 106 | - --exclude 'flutter/build/' \ | |
| 107 | - --exclude 'flutter/.dart_tool/' \ | |
| 108 | - --exclude '.git' \ | |
| 109 | - /app/ "$SHELL_DIR/" | |
| 110 | - | |
| 111 | -cd "$SHELL_DIR" | |
| 112 | -echo "state carried over:" | |
| 113 | -du -sh flutter/.home flutter/.clojuredart flutter/build 2>/dev/null \ | |
| 114 | - || echo " (none yet -- first run)" | |
| 115 | - | |
| 116 | -# Nix for the dependencies, the ordinary toolchain for the build. `nix build` | |
| 117 | -# cannot do this: a derivation is all-or-nothing, so any edit is a fresh | |
| 118 | -# sandbox and a fresh compile of everything. Here the devShell supplies the | |
| 119 | -# compiler and the libraries and `flutter build` decides what is stale. | |
| 120 | -# Evaluated from /app and built in the volume. Both halves matter: /app is the | |
| 121 | -# pristine copy, so nix stores a source tree of the repo rather than one | |
| 122 | -# carrying gigabytes of flutter/build, while the recipe still runs where the | |
| 123 | -# state it reuses lives -- `just -f` is what puts it there, since the recipe | |
| 124 | -# cds to its own justfile's directory. | |
| 125 | -nix develop /app#flutter-desktop --accept-flake-config \ | |
| 126 | - --extra-substituters file:///nix-cache \ | |
| 127 | - --max-jobs auto --command just -f "$SHELL_DIR/justfile" flutter-desktop | |
| 128 | - | |
| 129 | -echo "built:" | |
| 130 | -du -sh flutter/build | |
| 131 | - | |
| 132 | -# The devShell's closure is gigabytes of Flutter, Dart, clang and GTK, and the | |
| 133 | -# store it landed in belongs to the image rather than to a volume -- so | |
| 134 | -# without this every run re-fetches it from upstream. Written back, the next | |
| 135 | -# run substitutes it from file:///nix-cache instead. | |
| 136 | -if [ -f /nix-cache/nix-cache-info ]; then | |
| 137 | - echo "cache: writing the devShell closure back" | |
| 138 | - nix copy --no-check-sigs --all --to file:///nix-cache | |
| 139 | -fi | |
| 140 | -""" | |
| 141 | -# Nix's own cache, on the volume rather than in the container. Without it | |
| 142 | -# every Sandbox starts empty and `nix develop` re-clones the flake's git | |
| 143 | -# inputs -- nixgl and its transitives -- | |
| 144 | -# because flake.lock pins which revision to fetch, not whether it is already | |
| 145 | -# on disk. Set here rather than in the command so an interactive shell into | |
| 146 | -# this container gets it too. | |
| 147 | -env = { XDG_CACHE_HOME = "/devshell/frq-flutter-desktop/.cache" } | |
| 148 | - | |
| 149 | -[nix] | |
| 150 | -# Every nix command in the container reads the mounted cache, including one | |
| 151 | -# typed by hand in a shell. Passing --extra-substituters per command only ever | |
| 152 | -# covered the scripts. | |
| 153 | -substituters = ["file:///nix-cache"] | |
| 154 | -# No devShell warming at image build time: this enters `nix develop` at run | |
| 155 | -# time, on the VM, where the cache answers for its closure. The ptyshim that | |
| 156 | -# warming would need under gVisor is deprecated and does not come back. | |
| 157 | -flake = false | |
| 158 | -shim = false | |
| 159 | - | |
| 160 | -# [experimental] overrides the sandbox default of vm_runtime = true. | |
| deleted file mode 100644 | |||
| @@ -1,160 +0,0 @@ | |||
| 1 | -[container] | ||
| 2 | -name = "frq-flutter-dev" | ||
| 3 | -description = "the Flutter desktop build, incremental, in a nix devShell" | ||
| 4 | -base = "arch-nix" | ||
| 5 | -# A Sandbox, not a Function: runs on a real VM, and the command is | ||
| 6 | -# the sandbox's own process so it dies when the command does. | ||
| 7 | -runtime = "sandbox" | ||
| 8 | - | ||
| 9 | -[build] | ||
| 10 | -# The container lives inside the repo it builds, so the copy is rooted two | ||
| 11 | -# levels up and `.` is the whole tree. | ||
| 12 | -context = "../.." | ||
| 13 | -include = ["."] | ||
| 14 | -# `dev` is the shell you actually want. There is no default shell in this | ||
| 15 | -# flake any more -- the one that used to be there belonged to the retired | ||
| 16 | -# libcosmic frontend -- so a bare `nix develop` here fails rather than | ||
| 17 | -# resolving to the wrong tree. | ||
| 18 | -# The devShell, baked in rather than entered. `print-dev-env` writes the whole | ||
| 19 | -# environment out as shell -- PATH, the compiler, every variable mkShell sets | ||
| 20 | -# -- and realises its inputs on the way, so the closure becomes an image layer | ||
| 21 | -# instead of a fetch every container pays for. Sourcing it from .bashrc means a | ||
| 22 | -# shell attached to this container *is* the devShell: no `nix develop`, no | ||
| 23 | -# clone of the flake's git inputs, no wait. | ||
| 24 | -# | ||
| 25 | -# `dev` stays for the case where the baked env is stale against a flake edit. | ||
| 26 | -commands = [ | ||
| 27 | - "nix print-dev-env /app#flutter-desktop --accept-flake-config --extra-substituters file:///nix-cache > /etc/devshell.sh", | ||
| 28 | - "echo '. /etc/devshell.sh' >> /root/.bashrc", | ||
| 29 | - "printf '#!/bin/sh\\nexec nix develop /app#flutter-desktop \"$@\"\\n' > /usr/local/bin/dev && chmod +x /usr/local/bin/dev", | ||
| 30 | -] | ||
| 31 | -# The build state a local checkout carries: 395MB of a 441MB repo, uploaded on | ||
| 32 | -# every start and wanted by nothing out there. Flutter builds into a volume of | ||
| 33 | -# its own, and the clojure caches are this machine's. | ||
| 34 | -ignore = [ | ||
| 35 | - "flutter/build", "flutter/.home", "flutter/.dart_tool", | ||
| 36 | - "flutter/.clojuredart", "flutter/.cpcache", | ||
| 37 | - ".cpcache", "result", "build", ".git", | ||
| 38 | -] | ||
| 39 | - | ||
| 40 | -# Two volumes doing two different jobs. `nix-cache` is the binary cache every | ||
| 41 | -# container here reads from and writes back to. `devshell` is the working | ||
| 42 | -# state of a `nix develop` loop, and it is shared by every container that has | ||
| 43 | -# one -- each gets its own directory under it, named for the devShell it | ||
| 44 | -# belongs to, so two projects (or two shells of one project) never write the | ||
| 45 | -# same tree. Modal Volumes have no locking, so the directories are the only | ||
| 46 | -# thing keeping them apart, and two runs of the *same* devshell must not | ||
| 47 | -# overlap. | ||
| 48 | -[volumes] | ||
| 49 | -nix-cache = "/nix-cache" | ||
| 50 | -devshell = "/devshell" | ||
| 51 | - | ||
| 52 | -[resources] | ||
| 53 | -cpu = 8 | ||
| 54 | -memory = 16384 | ||
| 55 | -timeout = 3600 | ||
| 56 | - | ||
| 57 | -[run] | ||
| 58 | -workdir = "/app" | ||
| 59 | -# Nix for the dependencies, the ordinary toolchain for the build. `nix build` | ||
| 60 | -# cannot do this: a derivation is all-or-nothing, so any edit is a fresh | ||
| 61 | -# sandbox and a fresh compile of everything. Here the devShell supplies the | ||
| 62 | -# compiler and the libraries, and `flutter build` decides what is stale -- | ||
| 63 | -# which is the whole reason `just flutter-desktop` exists as the working-tree | ||
| 64 | -# loop rather than as another `nix build`. | ||
| 65 | -# | ||
| 66 | -# rsync and not cp, with --checksum and not mtimes: Modal copies the source in | ||
| 67 | -# with fresh timestamps on every run, so a plain copy would look entirely new | ||
| 68 | -# to Flutter and rebuild the lot. --checksum compares content, leaves the | ||
| 69 | -# unchanged files' timestamps alone, and lets the incremental build work. | ||
| 70 | -# | ||
| 71 | -# The excludes are the state that must NOT be overwritten from /app -- it is | ||
| 72 | -# what we are here to keep. `just flutter-desktop` seeds those caches only | ||
| 73 | -# when they are missing, so finding them warm is all it takes. | ||
| 74 | -command = """ | ||
| 75 | -set -e | ||
| 76 | -# This container's own directory on the shared devshell volume, named for the | ||
| 77 | -# devShell it keeps the state of. Anything else using this volume picks its | ||
| 78 | -# own name and the two never meet. | ||
| 79 | -SHELL_DIR=/devshell/frq-flutter-desktop | ||
| 80 | -mkdir -p "$SHELL_DIR" "$SHELL_DIR/.cache" | ||
| 81 | - | ||
| 82 | -# A worktree's `.git` is a *file* naming a gitdir back on the machine that | ||
| 83 | -# copied it in, and nix believes it and goes looking for a path that is not | ||
| 84 | -# here. It has to go before any flake reference to /app. | ||
| 85 | -rm -rf /app/.git | ||
| 86 | - | ||
| 87 | -echo "sync: /app -> $SHELL_DIR" | ||
| 88 | -# `nix shell --command` and not `nix profile install`: a profile install puts | ||
| 89 | -# rsync in ~/.nix-profile/bin, which is not on the PATH of the shell already | ||
| 90 | -# running, so the very next line said `rsync: command not found`. | ||
| 91 | -# | ||
| 92 | -# rsync and not cp, with --checksum and not mtimes: Modal copies the source in | ||
| 93 | -# with fresh timestamps every run, so a plain copy looks entirely new to | ||
| 94 | -# Flutter and rebuilds the lot. --checksum compares content and leaves the | ||
| 95 | -# unchanged files' timestamps alone, which is the whole basis of the | ||
| 96 | -# incremental build. | ||
| 97 | -# | ||
| 98 | -# The excludes are the state we are here to keep -- overwriting them from /app | ||
| 99 | -# would defeat the volume. `just flutter-desktop` seeds those caches only when | ||
| 100 | -# they are missing, so finding them warm is all it takes. | ||
| 101 | -nix shell nixpkgs#rsync --accept-flake-config \ | ||
| 102 | - --extra-substituters file:///nix-cache --command \ | ||
| 103 | - rsync -a --checksum --delete \ | ||
| 104 | - --exclude 'flutter/.home/' \ | ||
| 105 | - --exclude 'flutter/.clojuredart/' \ | ||
| 106 | - --exclude 'flutter/build/' \ | ||
| 107 | - --exclude 'flutter/.dart_tool/' \ | ||
| 108 | - --exclude '.git' \ | ||
| 109 | - /app/ "$SHELL_DIR/" | ||
| 110 | - | ||
| 111 | -cd "$SHELL_DIR" | ||
| 112 | -echo "state carried over:" | ||
| 113 | -du -sh flutter/.home flutter/.clojuredart flutter/build 2>/dev/null \ | ||
| 114 | - || echo " (none yet -- first run)" | ||
| 115 | - | ||
| 116 | -# Nix for the dependencies, the ordinary toolchain for the build. `nix build` | ||
| 117 | -# cannot do this: a derivation is all-or-nothing, so any edit is a fresh | ||
| 118 | -# sandbox and a fresh compile of everything. Here the devShell supplies the | ||
| 119 | -# compiler and the libraries and `flutter build` decides what is stale. | ||
| 120 | -# Evaluated from /app and built in the volume. Both halves matter: /app is the | ||
| 121 | -# pristine copy, so nix stores a source tree of the repo rather than one | ||
| 122 | -# carrying gigabytes of flutter/build, while the recipe still runs where the | ||
| 123 | -# state it reuses lives -- `just -f` is what puts it there, since the recipe | ||
| 124 | -# cds to its own justfile's directory. | ||
| 125 | -nix develop /app#flutter-desktop --accept-flake-config \ | ||
| 126 | - --extra-substituters file:///nix-cache \ | ||
| 127 | - --max-jobs auto --command just -f "$SHELL_DIR/justfile" flutter-desktop | ||
| 128 | - | ||
| 129 | -echo "built:" | ||
| 130 | -du -sh flutter/build | ||
| 131 | - | ||
| 132 | -# The devShell's closure is gigabytes of Flutter, Dart, clang and GTK, and the | ||
| 133 | -# store it landed in belongs to the image rather than to a volume -- so | ||
| 134 | -# without this every run re-fetches it from upstream. Written back, the next | ||
| 135 | -# run substitutes it from file:///nix-cache instead. | ||
| 136 | -if [ -f /nix-cache/nix-cache-info ]; then | ||
| 137 | - echo "cache: writing the devShell closure back" | ||
| 138 | - nix copy --no-check-sigs --all --to file:///nix-cache | ||
| 139 | -fi | ||
| 140 | -""" | ||
| 141 | -# Nix's own cache, on the volume rather than in the container. Without it | ||
| 142 | -# every Sandbox starts empty and `nix develop` re-clones the flake's git | ||
| 143 | -# inputs -- nixgl and its transitives -- | ||
| 144 | -# because flake.lock pins which revision to fetch, not whether it is already | ||
| 145 | -# on disk. Set here rather than in the command so an interactive shell into | ||
| 146 | -# this container gets it too. | ||
| 147 | -env = { XDG_CACHE_HOME = "/devshell/frq-flutter-desktop/.cache" } | ||
| 148 | - | ||
| 149 | -[nix] | ||
| 150 | -# Every nix command in the container reads the mounted cache, including one | ||
| 151 | -# typed by hand in a shell. Passing --extra-substituters per command only ever | ||
| 152 | -# covered the scripts. | ||
| 153 | -substituters = ["file:///nix-cache"] | ||
| 154 | -# No devShell warming at image build time: this enters `nix develop` at run | ||
| 155 | -# time, on the VM, where the cache answers for its closure. The ptyshim that | ||
| 156 | -# warming would need under gVisor is deprecated and does not come back. | ||
| 157 | -flake = false | ||
| 158 | -shim = false | ||
| 159 | - | ||
| 160 | -# [experimental] overrides the sandbox default of vm_runtime = true. | ||
renamed
.modal/web/README.md +13 -12 | similarity index 82% | ||
| rename from .modal/flutter-web/README.md | ||
| rename to .modal/web/README.md | ||
| @@ -1,9 +1,10 @@ | ||
| 1 | -# `flutter-web` | |
| 1 | +# `web` | |
| 2 | 2 | |
| 3 | - modal run .modal/flutter-web/container.py | |
| 4 | - just modal flutter-web | |
| 3 | + modal run .modal/web/container.py | |
| 4 | + just modal web | |
| 5 | 5 | |
| 6 | -Defined by `container.toml`; see `../spec.md` for the keys. | |
| 6 | +Defined by `container.toml`; `../_loader.py` is what reads it, and | |
| 7 | +its comments are the spec. | |
| 7 | 8 | Built on `debian:13-slim`. |
| 8 | 9 | |
| 9 | 10 | No nix, and that is the point of this container rather than an |
| @@ -18,14 +19,14 @@ copying a nix closure back to a volume afterwards -- does not happen | ||
| 18 | 19 | at all. The toolchain lands on the volume and the second run finds |
| 19 | 20 | it there. |
| 20 | 21 | |
| 21 | -The other two Flutter targets keep their devShells: `apk` needs the | |
| 22 | -Android SDK and `flutter-desktop` needs GTK and a C++ toolchain, and | |
| 23 | -a host toolchain is what nix is better at than a tarball. The web | |
| 24 | -target needs a Dart and a JVM, which is what a tarball is for. | |
| 22 | +The `dev` container beside it works the same way now, and so does a | |
| 23 | +laptop: one script, one pinned set of tarballs, and whatever the | |
| 24 | +host has to bring for a given target -- GTK and a C++ toolchain for | |
| 25 | +the desktop build, Google's command-line tools for the APK. | |
| 25 | 26 | |
| 26 | 27 | Same incremental shape as before -- the working tree, the generated |
| 27 | 28 | Dart under `flutter/lib/cljd-out` and Flutter's caches live on the |
| 28 | -`devshell` volume, under `frq-flutter-web/` so the desktop | |
| 29 | +`devshell` volume, under `frq-web/` so the desktop | |
| 29 | 30 | container's directory beside it is untouched. None of them is copied |
| 30 | 31 | in from the laptop: a checkout's copy of the compiler's output is |
| 31 | 32 | not this container's, and overwriting the volume's with it is how an |
| @@ -44,8 +45,8 @@ To look at what it built, ask for the serve action -- `[network] | ||
| 44 | 45 | ports` tunnels 8080 out, and the URL is printed once the sandbox is |
| 45 | 46 | scheduled: |
| 46 | 47 | |
| 47 | - modal run .modal/flutter-web/container.py \ | |
| 48 | - --command 'cd /devshell/frq-flutter-web && tools/build-web.sh serve 8080' | |
| 48 | + modal run .modal/web/container.py \ | |
| 49 | + --command 'cd /devshell/frq-web && tools/build-web.sh serve 8080' | |
| 49 | 50 | |
| 50 | 51 | That blocks until you Ctrl-C it, and it bills until you do. |
| 51 | 52 | |
| @@ -73,7 +74,7 @@ browser has no raw socket, so the IRC connection wants a WebSocket. | ||
| 73 | 74 | and hands out `flutter/build/web`, so a rebuild in the Sandbox is |
| 74 | 75 | picked up by the next cold start with nothing redeployed. |
| 75 | 76 | |
| 76 | - modal deploy .modal/flutter-web/serve.py | |
| 77 | + modal deploy .modal/web/serve.py | |
| 77 | 78 | |
| 78 | 79 | `[network] ports` tunnels 8080 out of the Sandbox as well, for the |
| 79 | 80 | case where you want the build and the server to be one process. |
| similarity index 82% | |||
| rename from .modal/flutter-web/README.md | |||
| rename to .modal/web/README.md | |||
| @@ -1,9 +1,10 @@ | |||
| 1 | -# `flutter-web` | 1 | +# `web` |
| 2 | 2 | ||
| 3 | - modal run .modal/flutter-web/container.py | 3 | + modal run .modal/web/container.py |
| 4 | - just modal flutter-web | 4 | + just modal web |
| 5 | 5 | ||
| 6 | -Defined by `container.toml`; see `../spec.md` for the keys. | 6 | +Defined by `container.toml`; `../_loader.py` is what reads it, and |
| 7 | +its comments are the spec. | ||
| 7 | Built on `debian:13-slim`. | 8 | Built on `debian:13-slim`. |
| 8 | 9 | ||
| 9 | No nix, and that is the point of this container rather than an | 10 | No nix, and that is the point of this container rather than an |
| @@ -18,14 +19,14 @@ copying a nix closure back to a volume afterwards -- does not happen | |||
| 18 | at all. The toolchain lands on the volume and the second run finds | 19 | at all. The toolchain lands on the volume and the second run finds |
| 19 | it there. | 20 | it there. |
| 20 | 21 | ||
| 21 | -The other two Flutter targets keep their devShells: `apk` needs the | 22 | +The `dev` container beside it works the same way now, and so does a |
| 22 | -Android SDK and `flutter-desktop` needs GTK and a C++ toolchain, and | 23 | +laptop: one script, one pinned set of tarballs, and whatever the |
| 23 | -a host toolchain is what nix is better at than a tarball. The web | 24 | +host has to bring for a given target -- GTK and a C++ toolchain for |
| 24 | -target needs a Dart and a JVM, which is what a tarball is for. | 25 | +the desktop build, Google's command-line tools for the APK. |
| 25 | 26 | ||
| 26 | Same incremental shape as before -- the working tree, the generated | 27 | Same incremental shape as before -- the working tree, the generated |
| 27 | Dart under `flutter/lib/cljd-out` and Flutter's caches live on the | 28 | Dart under `flutter/lib/cljd-out` and Flutter's caches live on the |
| 28 | -`devshell` volume, under `frq-flutter-web/` so the desktop | 29 | +`devshell` volume, under `frq-web/` so the desktop |
| 29 | container's directory beside it is untouched. None of them is copied | 30 | container's directory beside it is untouched. None of them is copied |
| 30 | in from the laptop: a checkout's copy of the compiler's output is | 31 | in from the laptop: a checkout's copy of the compiler's output is |
| 31 | not this container's, and overwriting the volume's with it is how an | 32 | not this container's, and overwriting the volume's with it is how an |
| @@ -44,8 +45,8 @@ To look at what it built, ask for the serve action -- `[network] | |||
| 44 | ports` tunnels 8080 out, and the URL is printed once the sandbox is | 45 | ports` tunnels 8080 out, and the URL is printed once the sandbox is |
| 45 | scheduled: | 46 | scheduled: |
| 46 | 47 | ||
| 47 | - modal run .modal/flutter-web/container.py \ | 48 | + modal run .modal/web/container.py \ |
| 48 | - --command 'cd /devshell/frq-flutter-web && tools/build-web.sh serve 8080' | 49 | + --command 'cd /devshell/frq-web && tools/build-web.sh serve 8080' |
| 49 | 50 | ||
| 50 | That blocks until you Ctrl-C it, and it bills until you do. | 51 | That blocks until you Ctrl-C it, and it bills until you do. |
| 51 | 52 | ||
| @@ -73,7 +74,7 @@ browser has no raw socket, so the IRC connection wants a WebSocket. | |||
| 73 | and hands out `flutter/build/web`, so a rebuild in the Sandbox is | 74 | and hands out `flutter/build/web`, so a rebuild in the Sandbox is |
| 74 | picked up by the next cold start with nothing redeployed. | 75 | picked up by the next cold start with nothing redeployed. |
| 75 | 76 | ||
| 76 | - modal deploy .modal/flutter-web/serve.py | 77 | + modal deploy .modal/web/serve.py |
| 77 | 78 | ||
| 78 | `[network] ports` tunnels 8080 out of the Sandbox as well, for the | 79 | `[network] ports` tunnels 8080 out of the Sandbox as well, for the |
| 79 | case where you want the build and the server to be one process. | 80 | case where you want the build and the server to be one process. |
renamed
.modal/web/container.py +5 -4 | similarity index 81% | ||
| rename from .modal/flutter-dev/container.py | ||
| rename to .modal/web/container.py | ||
| @@ -26,10 +26,11 @@ def main(command: str = "", shell: bool = False): | ||
| 26 | 26 | c.run_sandbox(command) |
| 27 | 27 | return |
| 28 | 28 | |
| 29 | - # `modal shell --image` only takes registry references, so it cannot be | |
| 30 | - # pointed at a published Modal image like arch-nix. Attaching to a running | |
| 31 | - # Sandbox can, and that Sandbox is this container: same image, same | |
| 32 | - # volumes, same env. | |
| 29 | + # `modal shell --image` takes a registry reference and nothing else, so | |
| 30 | + # it cannot be pointed at the image this container actually runs -- the | |
| 31 | + # one with its setup steps and its source copy in it. Attaching to a | |
| 32 | + # running Sandbox can, and that Sandbox is this container: same image, | |
| 33 | + # same volumes, same env. | |
| 33 | 34 | sb = c.open_sandbox() |
| 34 | 35 | print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}") |
| 35 | 36 | print(f" attach: modal shell {sb.object_id} (from another terminal)") |
| similarity index 81% | |||
| rename from .modal/flutter-dev/container.py | |||
| rename to .modal/web/container.py | |||
| @@ -26,10 +26,11 @@ def main(command: str = "", shell: bool = False): | |||
| 26 | c.run_sandbox(command) | 26 | c.run_sandbox(command) |
| 27 | return | 27 | return |
| 28 | 28 | ||
| 29 | - # `modal shell --image` only takes registry references, so it cannot be | 29 | + # `modal shell --image` takes a registry reference and nothing else, so |
| 30 | - # pointed at a published Modal image like arch-nix. Attaching to a running | 30 | + # it cannot be pointed at the image this container actually runs -- the |
| 31 | - # Sandbox can, and that Sandbox is this container: same image, same | 31 | + # one with its setup steps and its source copy in it. Attaching to a |
| 32 | - # volumes, same env. | 32 | + # running Sandbox can, and that Sandbox is this container: same image, |
| 33 | + # same volumes, same env. | ||
| 33 | sb = c.open_sandbox() | 34 | sb = c.open_sandbox() |
| 34 | print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}") | 35 | print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}") |
| 35 | print(f" attach: modal shell {sb.object_id} (from another terminal)") | 36 | print(f" attach: modal shell {sb.object_id} (from another terminal)") |
renamed
.modal/web/container.toml +6 -6 | similarity index 92% | ||
| rename from .modal/flutter-web/container.toml | ||
| rename to .modal/web/container.toml | ||
| @@ -1,5 +1,5 @@ | ||
| 1 | 1 | [container] |
| 2 | -name = "frq-flutter-web" | |
| 2 | +name = "frq-web" | |
| 3 | 3 | description = "the Flutter web build, incremental, from a pinned toolchain" |
| 4 | 4 | # `debian:13-slim` and not `arch-nix`: there is no nix in this container any |
| 5 | 5 | # more. The build is `tools/build-web.sh`, which fetches its own Flutter, JDK |
| @@ -57,7 +57,7 @@ ignore = [ | ||
| 57 | 57 | # One volume now, where there were two: the nix binary cache went with nix. |
| 58 | 58 | # `devshell` is the working state of an incremental loop, shared by every |
| 59 | 59 | # container that has one -- each gets its own directory under it, named for |
| 60 | -# what it belongs to, so `flutter-web` and `flutter-desktop` never write the | |
| 60 | +# what it belongs to, so `web` and `flutter-desktop` never write the | |
| 61 | 61 | # same tree. Modal Volumes have no locking, so those directory names are the |
| 62 | 62 | # only thing keeping them apart, and two runs of the *same* container must |
| 63 | 63 | # not overlap. |
| @@ -87,13 +87,13 @@ workdir = "/app" | ||
| 87 | 87 | # directory that is already on the volume. |
| 88 | 88 | command = """ |
| 89 | 89 | set -e |
| 90 | -SHELL_DIR=/devshell/frq-flutter-web | |
| 90 | +SHELL_DIR=/devshell/frq-web | |
| 91 | 91 | |
| 92 | 92 | # Beside the working tree and NOT inside it: the rsync below runs with |
| 93 | 93 | # --delete, so anything under $SHELL_DIR that is not in /app is removed on |
| 94 | 94 | # every run. A cache kept in there would be deleted moments before it was |
| 95 | 95 | # consulted -- which is what happened to the last one that tried. |
| 96 | -export FRQ_TOOLCHAIN=/devshell/frq-flutter-web.toolchain | |
| 96 | +export FRQ_TOOLCHAIN=/devshell/frq-web.toolchain | |
| 97 | 97 | mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN" |
| 98 | 98 | |
| 99 | 99 | echo "sync: /app -> $SHELL_DIR" |
| @@ -125,7 +125,7 @@ for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/buil | ||
| 125 | 125 | [ -d "$d" ] && echo " carried over: $d" |
| 126 | 126 | done |
| 127 | 127 | |
| 128 | -# The same script `just flutter-web` runs, with the same single build mode: | |
| 128 | +# The same script `just build web` runs, with the same single build mode: | |
| 129 | 129 | # what is served and what a laptop compiles are the same bundle. |
| 130 | 130 | tools/build-web.sh build |
| 131 | 131 | """ |
| @@ -133,4 +133,4 @@ tools/build-web.sh build | ||
| 133 | 133 | # XDG_CONFIG_HOME, and its own caches under XDG_CACHE_HOME. Both point into |
| 134 | 134 | # the volume so a second run finds what the first one decided. Set here |
| 135 | 135 | # rather than in the command so a shell into this container gets them too. |
| 136 | -env = { XDG_CACHE_HOME = "/devshell/frq-flutter-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-flutter-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-flutter-web.toolchain" } | |
| 136 | +env = { XDG_CACHE_HOME = "/devshell/frq-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-web.toolchain" } | |
| similarity index 92% | |||
| rename from .modal/flutter-web/container.toml | |||
| rename to .modal/web/container.toml | |||
| @@ -1,5 +1,5 @@ | |||
| 1 | [container] | 1 | [container] |
| 2 | -name = "frq-flutter-web" | 2 | +name = "frq-web" |
| 3 | description = "the Flutter web build, incremental, from a pinned toolchain" | 3 | description = "the Flutter web build, incremental, from a pinned toolchain" |
| 4 | # `debian:13-slim` and not `arch-nix`: there is no nix in this container any | 4 | # `debian:13-slim` and not `arch-nix`: there is no nix in this container any |
| 5 | # more. The build is `tools/build-web.sh`, which fetches its own Flutter, JDK | 5 | # more. The build is `tools/build-web.sh`, which fetches its own Flutter, JDK |
| @@ -57,7 +57,7 @@ ignore = [ | |||
| 57 | # One volume now, where there were two: the nix binary cache went with nix. | 57 | # One volume now, where there were two: the nix binary cache went with nix. |
| 58 | # `devshell` is the working state of an incremental loop, shared by every | 58 | # `devshell` is the working state of an incremental loop, shared by every |
| 59 | # container that has one -- each gets its own directory under it, named for | 59 | # container that has one -- each gets its own directory under it, named for |
| 60 | -# what it belongs to, so `flutter-web` and `flutter-desktop` never write the | 60 | +# what it belongs to, so `web` and `flutter-desktop` never write the |
| 61 | # same tree. Modal Volumes have no locking, so those directory names are the | 61 | # same tree. Modal Volumes have no locking, so those directory names are the |
| 62 | # only thing keeping them apart, and two runs of the *same* container must | 62 | # only thing keeping them apart, and two runs of the *same* container must |
| 63 | # not overlap. | 63 | # not overlap. |
| @@ -87,13 +87,13 @@ workdir = "/app" | |||
| 87 | # directory that is already on the volume. | 87 | # directory that is already on the volume. |
| 88 | command = """ | 88 | command = """ |
| 89 | set -e | 89 | set -e |
| 90 | -SHELL_DIR=/devshell/frq-flutter-web | 90 | +SHELL_DIR=/devshell/frq-web |
| 91 | 91 | ||
| 92 | # Beside the working tree and NOT inside it: the rsync below runs with | 92 | # Beside the working tree and NOT inside it: the rsync below runs with |
| 93 | # --delete, so anything under $SHELL_DIR that is not in /app is removed on | 93 | # --delete, so anything under $SHELL_DIR that is not in /app is removed on |
| 94 | # every run. A cache kept in there would be deleted moments before it was | 94 | # every run. A cache kept in there would be deleted moments before it was |
| 95 | # consulted -- which is what happened to the last one that tried. | 95 | # consulted -- which is what happened to the last one that tried. |
| 96 | -export FRQ_TOOLCHAIN=/devshell/frq-flutter-web.toolchain | 96 | +export FRQ_TOOLCHAIN=/devshell/frq-web.toolchain |
| 97 | mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN" | 97 | mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN" |
| 98 | 98 | ||
| 99 | echo "sync: /app -> $SHELL_DIR" | 99 | echo "sync: /app -> $SHELL_DIR" |
| @@ -125,7 +125,7 @@ for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/buil | |||
| 125 | [ -d "$d" ] && echo " carried over: $d" | 125 | [ -d "$d" ] && echo " carried over: $d" |
| 126 | done | 126 | done |
| 127 | 127 | ||
| 128 | -# The same script `just flutter-web` runs, with the same single build mode: | 128 | +# The same script `just build web` runs, with the same single build mode: |
| 129 | # what is served and what a laptop compiles are the same bundle. | 129 | # what is served and what a laptop compiles are the same bundle. |
| 130 | tools/build-web.sh build | 130 | tools/build-web.sh build |
| 131 | """ | 131 | """ |
| @@ -133,4 +133,4 @@ tools/build-web.sh build | |||
| 133 | # XDG_CONFIG_HOME, and its own caches under XDG_CACHE_HOME. Both point into | 133 | # XDG_CONFIG_HOME, and its own caches under XDG_CACHE_HOME. Both point into |
| 134 | # the volume so a second run finds what the first one decided. Set here | 134 | # the volume so a second run finds what the first one decided. Set here |
| 135 | # rather than in the command so a shell into this container gets them too. | 135 | # rather than in the command so a shell into this container gets them too. |
| 136 | -env = { XDG_CACHE_HOME = "/devshell/frq-flutter-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-flutter-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-flutter-web.toolchain" } | 136 | +env = { XDG_CACHE_HOME = "/devshell/frq-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-web.toolchain" } |
renamed
.modal/web/serve.py +5 -5 | similarity index 97% | ||
| rename from .modal/flutter-web/serve.py | ||
| rename to .modal/web/serve.py | ||
| @@ -2,13 +2,13 @@ | ||
| 2 | 2 | |
| 3 | 3 | Not part of `container.py`, and not a key in `container.toml`, because it is |
| 4 | 4 | the other half of a split the build already makes: the Sandbox compiles into |
| 5 | -`/devshell/frq-flutter-web` and exits, and what it leaves behind is a | |
| 5 | +`/devshell/frq-web` and exits, and what it leaves behind is a | |
| 6 | 6 | directory of static files that outlives it. A Function mounting the same |
| 7 | 7 | volume can hand those out without rebuilding anything, and can scale to zero |
| 8 | 8 | between readers -- which a Sandbox holding a tunnel open cannot. |
| 9 | 9 | |
| 10 | - modal serve .modal/flutter-web/serve.py # while editing, auto-reloads | |
| 11 | - modal deploy .modal/flutter-web/serve.py # a URL that stays | |
| 10 | + modal serve .modal/web/serve.py # while editing, auto-reloads | |
| 11 | + modal deploy .modal/web/serve.py # a URL that stays | |
| 12 | 12 | |
| 13 | 13 | Deliberately NOT built on the container's own image. That image carries the |
| 14 | 14 | repo and a few gigabytes of devShell closure, all of it for a compile that |
| @@ -26,9 +26,9 @@ import modal | ||
| 26 | 26 | # lazy, so naming it here costs nothing until a container actually mounts it. |
| 27 | 27 | DEVSHELL = modal.Volume.from_name("devshell", create_if_missing=True) |
| 28 | 28 | |
| 29 | -# Where `just flutter-web` leaves its output, under this devShell's own | |
| 29 | +# Where `just web` leaves its output, under this devShell's own | |
| 30 | 30 | # directory on the shared volume -- the same path container.toml builds in. |
| 31 | -WEB_ROOT = "/devshell/frq-flutter-web/flutter/build/web" | |
| 31 | +WEB_ROOT = "/devshell/frq-web/flutter/build/web" | |
| 32 | 32 | |
| 33 | 33 | PORT = 8080 |
| 34 | 34 | |
| similarity index 97% | |||
| rename from .modal/flutter-web/serve.py | |||
| rename to .modal/web/serve.py | |||
| @@ -2,13 +2,13 @@ | |||
| 2 | 2 | ||
| 3 | Not part of `container.py`, and not a key in `container.toml`, because it is | 3 | Not part of `container.py`, and not a key in `container.toml`, because it is |
| 4 | the other half of a split the build already makes: the Sandbox compiles into | 4 | the other half of a split the build already makes: the Sandbox compiles into |
| 5 | -`/devshell/frq-flutter-web` and exits, and what it leaves behind is a | 5 | +`/devshell/frq-web` and exits, and what it leaves behind is a |
| 6 | directory of static files that outlives it. A Function mounting the same | 6 | directory of static files that outlives it. A Function mounting the same |
| 7 | volume can hand those out without rebuilding anything, and can scale to zero | 7 | volume can hand those out without rebuilding anything, and can scale to zero |
| 8 | between readers -- which a Sandbox holding a tunnel open cannot. | 8 | between readers -- which a Sandbox holding a tunnel open cannot. |
| 9 | 9 | ||
| 10 | - modal serve .modal/flutter-web/serve.py # while editing, auto-reloads | 10 | + modal serve .modal/web/serve.py # while editing, auto-reloads |
| 11 | - modal deploy .modal/flutter-web/serve.py # a URL that stays | 11 | + modal deploy .modal/web/serve.py # a URL that stays |
| 12 | 12 | ||
| 13 | Deliberately NOT built on the container's own image. That image carries the | 13 | Deliberately NOT built on the container's own image. That image carries the |
| 14 | repo and a few gigabytes of devShell closure, all of it for a compile that | 14 | repo and a few gigabytes of devShell closure, all of it for a compile that |
| @@ -26,9 +26,9 @@ import modal | |||
| 26 | # lazy, so naming it here costs nothing until a container actually mounts it. | 26 | # lazy, so naming it here costs nothing until a container actually mounts it. |
| 27 | DEVSHELL = modal.Volume.from_name("devshell", create_if_missing=True) | 27 | DEVSHELL = modal.Volume.from_name("devshell", create_if_missing=True) |
| 28 | 28 | ||
| 29 | -# Where `just flutter-web` leaves its output, under this devShell's own | 29 | +# Where `just web` leaves its output, under this devShell's own |
| 30 | # directory on the shared volume -- the same path container.toml builds in. | 30 | # directory on the shared volume -- the same path container.toml builds in. |
| 31 | -WEB_ROOT = "/devshell/frq-flutter-web/flutter/build/web" | 31 | +WEB_ROOT = "/devshell/frq-web/flutter/build/web" |
| 32 | 32 | ||
| 33 | PORT = 8080 | 33 | PORT = 8080 |
| 34 | 34 | ||
modified
.rickub/workflows/build.yml +3 -3 | @@ -148,7 +148,7 @@ jobs: | ||
| 148 | 148 | # nowhere else, and `modal app logs` cannot reach an ephemeral run — |
| 149 | 149 | # so this terminal is the only place the build is visible. tee, not |
| 150 | 150 | # tail: a run killed mid-pipe through tail takes its output with it. |
| 151 | - run: modal run .modal/flutter-web/container.py 2>&1 | tee /tmp/frq-build.log | |
| 151 | + run: modal run .modal/web/container.py 2>&1 | tee /tmp/frq-build.log | |
| 152 | 152 | |
| 153 | 153 | # The Sandbox leaves the bundle on the devshell volume rather than |
| 154 | 154 | # anywhere a runner can see, so fetch it back out. |
| @@ -157,13 +157,13 @@ jobs: | ||
| 157 | 157 | MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }} |
| 158 | 158 | MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }} |
| 159 | 159 | # Into a directory that already exists, which `modal volume get` then |
| 160 | - # creates `web/` inside — the same shape `just web-local` uses. Naming | |
| 160 | + # creates `web/` inside — the same shape `just serve` uses. Naming | |
| 161 | 161 | # `web` as the destination itself is what failed, with `[Errno 21] Is a |
| 162 | 162 | # directory`, after the Modal build had already succeeded. |
| 163 | 163 | run: | |
| 164 | 164 | mkdir -p dist |
| 165 | 165 | modal volume get --force devshell \ |
| 166 | - frq-flutter-web/flutter/build/web dist | |
| 166 | + frq-web/flutter/build/web dist | |
| 167 | 167 | |
| 168 | 168 | - uses: actions/upload-artifact@v4 |
| 169 | 169 | with: |
| @@ -148,7 +148,7 @@ jobs: | |||
| 148 | # nowhere else, and `modal app logs` cannot reach an ephemeral run — | 148 | # nowhere else, and `modal app logs` cannot reach an ephemeral run — |
| 149 | # so this terminal is the only place the build is visible. tee, not | 149 | # so this terminal is the only place the build is visible. tee, not |
| 150 | # tail: a run killed mid-pipe through tail takes its output with it. | 150 | # tail: a run killed mid-pipe through tail takes its output with it. |
| 151 | - run: modal run .modal/flutter-web/container.py 2>&1 | tee /tmp/frq-build.log | 151 | + run: modal run .modal/web/container.py 2>&1 | tee /tmp/frq-build.log |
| 152 | 152 | ||
| 153 | # The Sandbox leaves the bundle on the devshell volume rather than | 153 | # The Sandbox leaves the bundle on the devshell volume rather than |
| 154 | # anywhere a runner can see, so fetch it back out. | 154 | # anywhere a runner can see, so fetch it back out. |
| @@ -157,13 +157,13 @@ jobs: | |||
| 157 | MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }} | 157 | MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }} |
| 158 | MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }} | 158 | MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }} |
| 159 | # Into a directory that already exists, which `modal volume get` then | 159 | # Into a directory that already exists, which `modal volume get` then |
| 160 | - # creates `web/` inside — the same shape `just web-local` uses. Naming | 160 | + # creates `web/` inside — the same shape `just serve` uses. Naming |
| 161 | # `web` as the destination itself is what failed, with `[Errno 21] Is a | 161 | # `web` as the destination itself is what failed, with `[Errno 21] Is a |
| 162 | # directory`, after the Modal build had already succeeded. | 162 | # directory`, after the Modal build had already succeeded. |
| 163 | run: | | 163 | run: | |
| 164 | mkdir -p dist | 164 | mkdir -p dist |
| 165 | modal volume get --force devshell \ | 165 | modal volume get --force devshell \ |
| 166 | - frq-flutter-web/flutter/build/web dist | 166 | + frq-web/flutter/build/web dist |
| 167 | 167 | ||
| 168 | - uses: actions/upload-artifact@v4 | 168 | - uses: actions/upload-artifact@v4 |
| 169 | with: | 169 | with: |
modified
CLAUDE.md +31 -40 | @@ -1,44 +1,36 @@ | ||
| 1 | 1 | # Working in this repo |
| 2 | 2 | |
| 3 | -## Nix | |
| 3 | +## The toolchain, and where builds happen | |
| 4 | 4 | |
| 5 | -**STOP BUILDING LOCALLY. Build on Modal.** This machine is for editing and for | |
| 6 | -evaluating — `nix flake check`, `nix eval`, `nix build --dry-run`, `nix repl` | |
| 7 | -— and not for realising a derivation. A local build of the whole graph gets | |
| 8 | -killed for memory long before it finishes, and the minutes spent finding that | |
| 9 | -out are minutes not spent on the change. So: | |
| 5 | +There is no nix in the build any more. `tools/toolchain.sh` fetches Flutter | |
| 6 | +(which carries Dart), a JDK, the Clojure CLI and Nim as sha256-pinned tarballs | |
| 7 | +into `.toolchain/`, and every `just` recipe runs inside the environment that | |
| 8 | +script prints. `just tools android` adds Google's command-line tools, which is | |
| 9 | +what `just build apk` needs before Gradle can have sdkmanager finish the SDK | |
| 10 | +off. The host still brings a C compiler, OpenSSL, git, curl, unzip and | |
| 11 | +python3 — and GTK with the usual CMake/Ninja/pkg-config for the Linux targets. | |
| 12 | + | |
| 13 | +**Prefer Modal for a long build.** A cold Flutter toolchain plus a full | |
| 14 | +compile is a lot of laptop, and the containers in `.modal/` do it on a real | |
| 15 | +machine: | |
| 10 | 16 | |
| 11 | 17 | ```bash |
| 12 | -modal run .modal/flutter-web/container.py # the web bundle, on Modal | |
| 13 | -modal run .modal/flutter-dev/container.py # the incremental Flutter loop | |
| 18 | +just modal web # the web bundle, on Modal | |
| 19 | +just modal dev # the incremental Flutter loop | |
| 14 | 20 | ``` |
| 15 | 21 | |
| 16 | 22 | The containers in `.modal/` are not a third source tree: they are CI config |
| 17 | -that happens to live here, the way `.github/` would be. | |
| 18 | - | |
| 19 | -`--dry-run` locally to see what *would* be built, then hand the build to Modal. | |
| 20 | -The one exception is a derivation you already know is trivial and already | |
| 21 | -substitutable; if you are unsure, it is not the exception. | |
| 23 | +that happens to live here, the way `.github/` would be. They run the very same | |
| 24 | +`tools/toolchain.sh`, which is why a plain Debian image is enough. | |
| 22 | 25 | |
| 23 | 26 | `modal app logs` is no substitute for watching that command: it resolves |
| 24 | 27 | deployed apps by name, not the ephemeral one a `modal run` creates, and carries |
| 25 | 28 | nothing until the Sandbox starts — the image build streams to the client and |
| 26 | 29 | nowhere else. |
| 27 | 30 | |
| 28 | -You are already running inside the Arch distrobox, where `nix` lives, so run | |
| 29 | -the evaluating commands directly — do not wrap them in `distrobox enter`. | |
| 30 | - | |
| 31 | -The containers run as Modal **Sandboxes on a real VM**, which is what makes a | |
| 32 | -build work out there at all: the ptyshim that used to stand in for a working | |
| 33 | -pty under gVisor is deprecated, and nothing here should reintroduce it. Neither | |
| 34 | -container carries nix: `tools/toolchain.sh` fetches Flutter, a JDK and the | |
| 35 | -Clojure CLI by sha256 onto the `devshell` Volume, and the build runs out of | |
| 36 | -those. | |
| 37 | - | |
| 38 | -A remote builder (`eu.nixbuild.net`) is also configured here, for the case | |
| 39 | -where you want a derivation built somewhere other than Modal: `--store | |
| 40 | -ssh-ng://eu.nixbuild.net --eval-store auto` rather than a `builders` entry, so | |
| 41 | -the whole graph stays there and only .drv files go up. | |
| 31 | +The containers run as Modal **Sandboxes on a real VM** rather than under | |
| 32 | +gVisor: a real kernel, a working pty, and memory that is exactly what | |
| 33 | +`[resources] memory` asks for. | |
| 42 | 34 | |
| 43 | 35 | One thing this container is *not* representative of: `/etc/localtime` is a |
| 44 | 36 | regular file here rather than a symlink, so anything that reads the zone out |
| @@ -56,7 +48,7 @@ Let them write to the terminal, or `tee` them if you want a copy to grep | ||
| 56 | 48 | afterwards: |
| 57 | 49 | |
| 58 | 50 | ```bash |
| 59 | -modal run .modal/flutter-web/container.py 2>&1 | tee /tmp/frq-build.log | |
| 51 | +modal run .modal/web/container.py 2>&1 | tee /tmp/frq-build.log | |
| 60 | 52 | ``` |
| 61 | 53 | |
| 62 | 54 | Trim afterwards, on the file, where the whole run is still there to re-read. |
| @@ -83,7 +75,7 @@ ClojureDart — the core exists to have less Clojure in the tree, and adding | ||
| 83 | 75 | more of it to call the thing replacing it is the wrong direction. ClojureDart |
| 84 | 76 | shrinks from both ends. |
| 85 | 77 | |
| 86 | -`just nim-test` and `just dart-test` need no Flutter, which is most of the | |
| 78 | +`just test nim` and `just test dart` need no Flutter, which is most of the | |
| 87 | 79 | point: the whole boundary is checkable in about a second. |
| 88 | 80 | |
| 89 | 81 | A module is not deleted from `common/` when its Nim version lands: the web |
| @@ -120,19 +112,18 @@ every push. | ||
| 120 | 112 | |
| 121 | 113 | `flutter/` builds three things, from one `clojure -M:cljd compile`: |
| 122 | 114 | |
| 123 | -`just apk`, out of the flake's own `.#flutter` shell (clojure, jdk17, flutter) | |
| 124 | -and its `.#android-sdk` package. Impure on purpose: Gradle fetches its own | |
| 125 | -dependencies and writes into `ANDROID_HOME`, so the recipe copies the store SDK | |
| 126 | -to `flutter/.home` and lets it finish there. | |
| 115 | +`just build apk`. Impure on purpose: Gradle resolves its own dependencies over | |
| 116 | +the network and has sdkmanager install a platform and build-tools into | |
| 117 | +`ANDROID_HOME` as it goes, which is why that SDK lives in `.toolchain/` and is | |
| 118 | +ours to write to. | |
| 127 | 119 | |
| 128 | -`just flutter-desktop`, out of `.#flutter-desktop` (the same clojure and | |
| 129 | -flutter, with cmake, ninja, pkg-config and gtk3 where the JDK and the SDK are). | |
| 130 | -Impure for the network half of the same reasons and no writable-SDK dance, | |
| 131 | -since nothing writes into the store. nixGL off NixOS. | |
| 120 | +`just build desktop`, Flutter's Linux target — CMake, Ninja, pkg-config and | |
| 121 | +GTK from the host where the APK wants a JDK and an SDK. Impure for the network | |
| 122 | +half of the same reasons. | |
| 132 | 123 | |
| 133 | -`just flutter-web`, out of no nix shell at all — `tools/toolchain.sh` fetches | |
| 134 | -the three pinned tarballs it needs, which is what lets `.modal/flutter-web/` | |
| 135 | -run the same script on a plain Debian image. | |
| 124 | +`just build web`, which needs least of all: a Dart, a JVM and a browser, and | |
| 125 | +the browser is not ours. That is what lets `.modal/web/` run the same | |
| 126 | +`tools/build-web.sh` on a plain Debian image. | |
| 136 | 127 | |
| 137 | 128 | The consequence for `common/` is that "the phone" is not a synonym for "the |
| 138 | 129 | ClojureDart side": three targets compile it. An implementation that branches on |
| @@ -1,44 +1,36 @@ | |||
| 1 | # Working in this repo | 1 | # Working in this repo |
| 2 | 2 | ||
| 3 | -## Nix | 3 | +## The toolchain, and where builds happen |
| 4 | 4 | ||
| 5 | -**STOP BUILDING LOCALLY. Build on Modal.** This machine is for editing and for | 5 | +There is no nix in the build any more. `tools/toolchain.sh` fetches Flutter |
| 6 | -evaluating — `nix flake check`, `nix eval`, `nix build --dry-run`, `nix repl` | 6 | +(which carries Dart), a JDK, the Clojure CLI and Nim as sha256-pinned tarballs |
| 7 | -— and not for realising a derivation. A local build of the whole graph gets | 7 | +into `.toolchain/`, and every `just` recipe runs inside the environment that |
| 8 | -killed for memory long before it finishes, and the minutes spent finding that | 8 | +script prints. `just tools android` adds Google's command-line tools, which is |
| 9 | -out are minutes not spent on the change. So: | 9 | +what `just build apk` needs before Gradle can have sdkmanager finish the SDK |
| 10 | +off. The host still brings a C compiler, OpenSSL, git, curl, unzip and | ||
| 11 | +python3 — and GTK with the usual CMake/Ninja/pkg-config for the Linux targets. | ||
| 12 | + | ||
| 13 | +**Prefer Modal for a long build.** A cold Flutter toolchain plus a full | ||
| 14 | +compile is a lot of laptop, and the containers in `.modal/` do it on a real | ||
| 15 | +machine: | ||
| 10 | 16 | ||
| 11 | ```bash | 17 | ```bash |
| 12 | -modal run .modal/flutter-web/container.py # the web bundle, on Modal | 18 | +just modal web # the web bundle, on Modal |
| 13 | -modal run .modal/flutter-dev/container.py # the incremental Flutter loop | 19 | +just modal dev # the incremental Flutter loop |
| 14 | ``` | 20 | ``` |
| 15 | 21 | ||
| 16 | The containers in `.modal/` are not a third source tree: they are CI config | 22 | The containers in `.modal/` are not a third source tree: they are CI config |
| 17 | -that happens to live here, the way `.github/` would be. | 23 | +that happens to live here, the way `.github/` would be. They run the very same |
| 18 | - | 24 | +`tools/toolchain.sh`, which is why a plain Debian image is enough. |
| 19 | -`--dry-run` locally to see what *would* be built, then hand the build to Modal. | ||
| 20 | -The one exception is a derivation you already know is trivial and already | ||
| 21 | -substitutable; if you are unsure, it is not the exception. | ||
| 22 | 25 | ||
| 23 | `modal app logs` is no substitute for watching that command: it resolves | 26 | `modal app logs` is no substitute for watching that command: it resolves |
| 24 | deployed apps by name, not the ephemeral one a `modal run` creates, and carries | 27 | deployed apps by name, not the ephemeral one a `modal run` creates, and carries |
| 25 | nothing until the Sandbox starts — the image build streams to the client and | 28 | nothing until the Sandbox starts — the image build streams to the client and |
| 26 | nowhere else. | 29 | nowhere else. |
| 27 | 30 | ||
| 28 | -You are already running inside the Arch distrobox, where `nix` lives, so run | 31 | +The containers run as Modal **Sandboxes on a real VM** rather than under |
| 29 | -the evaluating commands directly — do not wrap them in `distrobox enter`. | 32 | +gVisor: a real kernel, a working pty, and memory that is exactly what |
| 30 | - | 33 | +`[resources] memory` asks for. |
| 31 | -The containers run as Modal **Sandboxes on a real VM**, which is what makes a | ||
| 32 | -build work out there at all: the ptyshim that used to stand in for a working | ||
| 33 | -pty under gVisor is deprecated, and nothing here should reintroduce it. Neither | ||
| 34 | -container carries nix: `tools/toolchain.sh` fetches Flutter, a JDK and the | ||
| 35 | -Clojure CLI by sha256 onto the `devshell` Volume, and the build runs out of | ||
| 36 | -those. | ||
| 37 | - | ||
| 38 | -A remote builder (`eu.nixbuild.net`) is also configured here, for the case | ||
| 39 | -where you want a derivation built somewhere other than Modal: `--store | ||
| 40 | -ssh-ng://eu.nixbuild.net --eval-store auto` rather than a `builders` entry, so | ||
| 41 | -the whole graph stays there and only .drv files go up. | ||
| 42 | 34 | ||
| 43 | One thing this container is *not* representative of: `/etc/localtime` is a | 35 | One thing this container is *not* representative of: `/etc/localtime` is a |
| 44 | regular file here rather than a symlink, so anything that reads the zone out | 36 | regular file here rather than a symlink, so anything that reads the zone out |
| @@ -56,7 +48,7 @@ Let them write to the terminal, or `tee` them if you want a copy to grep | |||
| 56 | afterwards: | 48 | afterwards: |
| 57 | 49 | ||
| 58 | ```bash | 50 | ```bash |
| 59 | -modal run .modal/flutter-web/container.py 2>&1 | tee /tmp/frq-build.log | 51 | +modal run .modal/web/container.py 2>&1 | tee /tmp/frq-build.log |
| 60 | ``` | 52 | ``` |
| 61 | 53 | ||
| 62 | Trim afterwards, on the file, where the whole run is still there to re-read. | 54 | Trim afterwards, on the file, where the whole run is still there to re-read. |
| @@ -83,7 +75,7 @@ ClojureDart — the core exists to have less Clojure in the tree, and adding | |||
| 83 | more of it to call the thing replacing it is the wrong direction. ClojureDart | 75 | more of it to call the thing replacing it is the wrong direction. ClojureDart |
| 84 | shrinks from both ends. | 76 | shrinks from both ends. |
| 85 | 77 | ||
| 86 | -`just nim-test` and `just dart-test` need no Flutter, which is most of the | 78 | +`just test nim` and `just test dart` need no Flutter, which is most of the |
| 87 | point: the whole boundary is checkable in about a second. | 79 | point: the whole boundary is checkable in about a second. |
| 88 | 80 | ||
| 89 | A module is not deleted from `common/` when its Nim version lands: the web | 81 | A module is not deleted from `common/` when its Nim version lands: the web |
| @@ -120,19 +112,18 @@ every push. | |||
| 120 | 112 | ||
| 121 | `flutter/` builds three things, from one `clojure -M:cljd compile`: | 113 | `flutter/` builds three things, from one `clojure -M:cljd compile`: |
| 122 | 114 | ||
| 123 | -`just apk`, out of the flake's own `.#flutter` shell (clojure, jdk17, flutter) | 115 | +`just build apk`. Impure on purpose: Gradle resolves its own dependencies over |
| 124 | -and its `.#android-sdk` package. Impure on purpose: Gradle fetches its own | 116 | +the network and has sdkmanager install a platform and build-tools into |
| 125 | -dependencies and writes into `ANDROID_HOME`, so the recipe copies the store SDK | 117 | +`ANDROID_HOME` as it goes, which is why that SDK lives in `.toolchain/` and is |
| 126 | -to `flutter/.home` and lets it finish there. | 118 | +ours to write to. |
| 127 | 119 | ||
| 128 | -`just flutter-desktop`, out of `.#flutter-desktop` (the same clojure and | 120 | +`just build desktop`, Flutter's Linux target — CMake, Ninja, pkg-config and |
| 129 | -flutter, with cmake, ninja, pkg-config and gtk3 where the JDK and the SDK are). | 121 | +GTK from the host where the APK wants a JDK and an SDK. Impure for the network |
| 130 | -Impure for the network half of the same reasons and no writable-SDK dance, | 122 | +half of the same reasons. |
| 131 | -since nothing writes into the store. nixGL off NixOS. | ||
| 132 | 123 | ||
| 133 | -`just flutter-web`, out of no nix shell at all — `tools/toolchain.sh` fetches | 124 | +`just build web`, which needs least of all: a Dart, a JVM and a browser, and |
| 134 | -the three pinned tarballs it needs, which is what lets `.modal/flutter-web/` | 125 | +the browser is not ours. That is what lets `.modal/web/` run the same |
| 135 | -run the same script on a plain Debian image. | 126 | +`tools/build-web.sh` on a plain Debian image. |
| 136 | 127 | ||
| 137 | The consequence for `common/` is that "the phone" is not a synonym for "the | 128 | The consequence for `common/` is that "the phone" is not a synonym for "the |
| 138 | ClojureDart side": three targets compile it. An implementation that branches on | 129 | ClojureDart side": three targets compile it. An implementation that branches on |
modified
README.md +11 -12 | @@ -53,7 +53,7 @@ There used to be a third tree, `src/`, and another runtime under it: jolt, with | ||
| 53 | 53 | **libcosmic** in a desktop window and by `libjolttui` in a terminal, plus an |
| 54 | 54 | AV media plane over MoQ. It is gone. Flutter is the only frontend now, which is |
| 55 | 55 | why `common/` no longer carries `#?(:jolt ...)` reader conditionals and why the |
| 56 | -calls, terminal and `nix run .#frq` sections that used to be here are not. | |
| 56 | +calls and terminal sections that used to be here are not. | |
| 57 | 57 | |
| 58 | 58 | ## Tracing |
| 59 | 59 | |
| @@ -63,18 +63,17 @@ Android is logcat. | ||
| 63 | 63 | ## Running |
| 64 | 64 | |
| 65 | 65 | ```bash |
| 66 | -just flutter-desktop run # the Linux window | |
| 67 | -just apk run # onto a connected Android device | |
| 68 | -just flutter-web serve # a browser, on :8080 | |
| 66 | +just run desktop # the Linux window | |
| 67 | +just run apk # onto a connected Android device | |
| 68 | +just run web # a browser, on :8080 | |
| 69 | 69 | ``` |
| 70 | 70 | |
| 71 | -Every recipe lives in the `justfile` itself. The two that need a toolchain from | |
| 72 | -Nix re-enter `nix develop` and come back to the same recipe, so `just | |
| 73 | -flutter-desktop` and `nix develop .#flutter-desktop --command just | |
| 74 | -flutter-desktop` are one code path rather than two. `just flutter-web` needs no | |
| 75 | -Nix at all: `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by | |
| 76 | -sha256, which is what lets `.modal/flutter-web/` run the same script on a plain | |
| 77 | -Debian image. | |
| 71 | +Every recipe lives in the `justfile` itself, and none of them needs Nix: | |
| 72 | +`tools/toolchain.sh` fetches Flutter, a JDK, the Clojure CLI and Nim by | |
| 73 | +sha256 into `.toolchain/`, and every recipe runs inside that. It is the same | |
| 74 | +script `.modal/` runs, which is what lets a plain Debian image build this. | |
| 75 | +`just build apk` additionally asks it for Google's command-line tools, and | |
| 76 | +sdkmanager finishes the Android SDK off. | |
| 78 | 77 | |
| 79 | 78 | All three are one `clojure -M:cljd compile` over `flutter/src` and `common/`, |
| 80 | 79 | and differ only in which Flutter target runs afterwards. |
| @@ -90,7 +89,7 @@ cargo run --release --bin freeq-server # in the freeq checkout | ||
| 90 | 89 | A browser has no TCP, so the web build wants a WebSocket URL in the Server |
| 91 | 90 | field — `wss://irc.freeq.at/irc`. And Bluesky sign-in only completes on |
| 92 | 91 | `localhost`, because that is the one origin freeq's auth broker will redirect |
| 93 | -back to: `just web-local` is what serves the Modal-built bundle there. | |
| 92 | +back to: `just serve` is what serves the Modal-built bundle there. | |
| 94 | 93 | |
| 95 | 94 | ## Signing in |
| 96 | 95 | |
| @@ -53,7 +53,7 @@ There used to be a third tree, `src/`, and another runtime under it: jolt, with | |||
| 53 | **libcosmic** in a desktop window and by `libjolttui` in a terminal, plus an | 53 | **libcosmic** in a desktop window and by `libjolttui` in a terminal, plus an |
| 54 | AV media plane over MoQ. It is gone. Flutter is the only frontend now, which is | 54 | AV media plane over MoQ. It is gone. Flutter is the only frontend now, which is |
| 55 | why `common/` no longer carries `#?(:jolt ...)` reader conditionals and why the | 55 | why `common/` no longer carries `#?(:jolt ...)` reader conditionals and why the |
| 56 | -calls, terminal and `nix run .#frq` sections that used to be here are not. | 56 | +calls and terminal sections that used to be here are not. |
| 57 | 57 | ||
| 58 | ## Tracing | 58 | ## Tracing |
| 59 | 59 | ||
| @@ -63,18 +63,17 @@ Android is logcat. | |||
| 63 | ## Running | 63 | ## Running |
| 64 | 64 | ||
| 65 | ```bash | 65 | ```bash |
| 66 | -just flutter-desktop run # the Linux window | 66 | +just run desktop # the Linux window |
| 67 | -just apk run # onto a connected Android device | 67 | +just run apk # onto a connected Android device |
| 68 | -just flutter-web serve # a browser, on :8080 | 68 | +just run web # a browser, on :8080 |
| 69 | ``` | 69 | ``` |
| 70 | 70 | ||
| 71 | -Every recipe lives in the `justfile` itself. The two that need a toolchain from | 71 | +Every recipe lives in the `justfile` itself, and none of them needs Nix: |
| 72 | -Nix re-enter `nix develop` and come back to the same recipe, so `just | 72 | +`tools/toolchain.sh` fetches Flutter, a JDK, the Clojure CLI and Nim by |
| 73 | -flutter-desktop` and `nix develop .#flutter-desktop --command just | 73 | +sha256 into `.toolchain/`, and every recipe runs inside that. It is the same |
| 74 | -flutter-desktop` are one code path rather than two. `just flutter-web` needs no | 74 | +script `.modal/` runs, which is what lets a plain Debian image build this. |
| 75 | -Nix at all: `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by | 75 | +`just build apk` additionally asks it for Google's command-line tools, and |
| 76 | -sha256, which is what lets `.modal/flutter-web/` run the same script on a plain | 76 | +sdkmanager finishes the Android SDK off. |
| 77 | -Debian image. | ||
| 78 | 77 | ||
| 79 | All three are one `clojure -M:cljd compile` over `flutter/src` and `common/`, | 78 | All three are one `clojure -M:cljd compile` over `flutter/src` and `common/`, |
| 80 | and differ only in which Flutter target runs afterwards. | 79 | and differ only in which Flutter target runs afterwards. |
| @@ -90,7 +89,7 @@ cargo run --release --bin freeq-server # in the freeq checkout | |||
| 90 | A browser has no TCP, so the web build wants a WebSocket URL in the Server | 89 | A browser has no TCP, so the web build wants a WebSocket URL in the Server |
| 91 | field — `wss://irc.freeq.at/irc`. And Bluesky sign-in only completes on | 90 | field — `wss://irc.freeq.at/irc`. And Bluesky sign-in only completes on |
| 92 | `localhost`, because that is the one origin freeq's auth broker will redirect | 91 | `localhost`, because that is the one origin freeq's auth broker will redirect |
| 93 | -back to: `just web-local` is what serves the Modal-built bundle there. | 92 | +back to: `just serve` is what serves the Modal-built bundle there. |
| 94 | 93 | ||
| 95 | ## Signing in | 94 | ## Signing in |
| 96 | 95 | ||
modified
dart/README.md +1 -1 | @@ -11,7 +11,7 @@ things: | ||
| 11 | 11 | * **It tests on the plain Dart VM.** `flutter/pubspec.yaml` depends on the |
| 12 | 12 | Flutter SDK, so `dart pub get` cannot resolve it at all — anything living |
| 13 | 13 | there needs a Flutter toolchain to run one assertion about a string. This |
| 14 | - package resolves and tests in a second. `just dart-test`. | |
| 14 | + package resolves and tests in a second. `just test dart`. | |
| 15 | 15 | * **It says which way the dependency goes.** The Flutter app depends on this |
| 16 | 16 | by path. Nothing here may depend on Flutter, and if that ever becomes |
| 17 | 17 | tempting the thing being written belongs on the other side of the line. |
| @@ -11,7 +11,7 @@ things: | |||
| 11 | * **It tests on the plain Dart VM.** `flutter/pubspec.yaml` depends on the | 11 | * **It tests on the plain Dart VM.** `flutter/pubspec.yaml` depends on the |
| 12 | Flutter SDK, so `dart pub get` cannot resolve it at all — anything living | 12 | Flutter SDK, so `dart pub get` cannot resolve it at all — anything living |
| 13 | there needs a Flutter toolchain to run one assertion about a string. This | 13 | there needs a Flutter toolchain to run one assertion about a string. This |
| 14 | - package resolves and tests in a second. `just dart-test`. | 14 | + package resolves and tests in a second. `just test dart`. |
| 15 | * **It says which way the dependency goes.** The Flutter app depends on this | 15 | * **It says which way the dependency goes.** The Flutter app depends on this |
| 16 | by path. Nothing here may depend on Flutter, and if that ever becomes | 16 | by path. Nothing here may depend on Flutter, and if that ever becomes |
| 17 | tempting the thing being written belongs on the other side of the line. | 17 | tempting the thing being written belongs on the other side of the line. |
deleted
flake.lock +0 -82 | deleted file mode 100644 | ||
| @@ -1,82 +0,0 @@ | ||
| 1 | -{ | |
| 2 | - "nodes": { | |
| 3 | - "flake-utils": { | |
| 4 | - "inputs": { | |
| 5 | - "systems": "systems" | |
| 6 | - }, | |
| 7 | - "locked": { | |
| 8 | - "lastModified": 1731533236, | |
| 9 | - "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=", | |
| 10 | - "owner": "numtide", | |
| 11 | - "repo": "flake-utils", | |
| 12 | - "rev": "11707dc2f618dd54ca8739b309ec4fc024de578b", | |
| 13 | - "type": "github" | |
| 14 | - }, | |
| 15 | - "original": { | |
| 16 | - "owner": "numtide", | |
| 17 | - "repo": "flake-utils", | |
| 18 | - "type": "github" | |
| 19 | - } | |
| 20 | - }, | |
| 21 | - "nixgl": { | |
| 22 | - "inputs": { | |
| 23 | - "flake-utils": "flake-utils", | |
| 24 | - "nixpkgs": [ | |
| 25 | - "nixpkgs" | |
| 26 | - ] | |
| 27 | - }, | |
| 28 | - "locked": { | |
| 29 | - "lastModified": 1762090880, | |
| 30 | - "narHash": "sha256-fbRQzIGPkjZa83MowjbD2ALaJf9y6KMDdJBQMKFeY/8=", | |
| 31 | - "owner": "nix-community", | |
| 32 | - "repo": "nixGL", | |
| 33 | - "rev": "b6105297e6f0cd041670c3e8628394d4ee247ed5", | |
| 34 | - "type": "github" | |
| 35 | - }, | |
| 36 | - "original": { | |
| 37 | - "owner": "nix-community", | |
| 38 | - "repo": "nixGL", | |
| 39 | - "type": "github" | |
| 40 | - } | |
| 41 | - }, | |
| 42 | - "nixpkgs": { | |
| 43 | - "locked": { | |
| 44 | - "lastModified": 1788039129, | |
| 45 | - "narHash": "sha256-pa4Q0qErvCvzCaaUph7Sm37RhR4xvPrYI8Lgz6k85+A=", | |
| 46 | - "owner": "NixOS", | |
| 47 | - "repo": "nixpkgs", | |
| 48 | - "rev": "d2f67949798825fe853f7c5d0492b8bf016d3f88", | |
| 49 | - "type": "github" | |
| 50 | - }, | |
| 51 | - "original": { | |
| 52 | - "owner": "NixOS", | |
| 53 | - "ref": "nixos-unstable", | |
| 54 | - "repo": "nixpkgs", | |
| 55 | - "type": "github" | |
| 56 | - } | |
| 57 | - }, | |
| 58 | - "root": { | |
| 59 | - "inputs": { | |
| 60 | - "nixgl": "nixgl", | |
| 61 | - "nixpkgs": "nixpkgs" | |
| 62 | - } | |
| 63 | - }, | |
| 64 | - "systems": { | |
| 65 | - "locked": { | |
| 66 | - "lastModified": 1681028828, | |
| 67 | - "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=", | |
| 68 | - "owner": "nix-systems", | |
| 69 | - "repo": "default", | |
| 70 | - "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e", | |
| 71 | - "type": "github" | |
| 72 | - }, | |
| 73 | - "original": { | |
| 74 | - "owner": "nix-systems", | |
| 75 | - "repo": "default", | |
| 76 | - "type": "github" | |
| 77 | - } | |
| 78 | - } | |
| 79 | - }, | |
| 80 | - "root": "root", | |
| 81 | - "version": 7 | |
| 82 | -} | |
| deleted file mode 100644 | |||
| @@ -1,82 +0,0 @@ | |||
| 1 | -{ | ||
| 2 | - "nodes": { | ||
| 3 | - "flake-utils": { | ||
| 4 | - "inputs": { | ||
| 5 | - "systems": "systems" | ||
| 6 | - }, | ||
| 7 | - "locked": { | ||
| 8 | - "lastModified": 1731533236, | ||
| 9 | - "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=", | ||
| 10 | - "owner": "numtide", | ||
| 11 | - "repo": "flake-utils", | ||
| 12 | - "rev": "11707dc2f618dd54ca8739b309ec4fc024de578b", | ||
| 13 | - "type": "github" | ||
| 14 | - }, | ||
| 15 | - "original": { | ||
| 16 | - "owner": "numtide", | ||
| 17 | - "repo": "flake-utils", | ||
| 18 | - "type": "github" | ||
| 19 | - } | ||
| 20 | - }, | ||
| 21 | - "nixgl": { | ||
| 22 | - "inputs": { | ||
| 23 | - "flake-utils": "flake-utils", | ||
| 24 | - "nixpkgs": [ | ||
| 25 | - "nixpkgs" | ||
| 26 | - ] | ||
| 27 | - }, | ||
| 28 | - "locked": { | ||
| 29 | - "lastModified": 1762090880, | ||
| 30 | - "narHash": "sha256-fbRQzIGPkjZa83MowjbD2ALaJf9y6KMDdJBQMKFeY/8=", | ||
| 31 | - "owner": "nix-community", | ||
| 32 | - "repo": "nixGL", | ||
| 33 | - "rev": "b6105297e6f0cd041670c3e8628394d4ee247ed5", | ||
| 34 | - "type": "github" | ||
| 35 | - }, | ||
| 36 | - "original": { | ||
| 37 | - "owner": "nix-community", | ||
| 38 | - "repo": "nixGL", | ||
| 39 | - "type": "github" | ||
| 40 | - } | ||
| 41 | - }, | ||
| 42 | - "nixpkgs": { | ||
| 43 | - "locked": { | ||
| 44 | - "lastModified": 1788039129, | ||
| 45 | - "narHash": "sha256-pa4Q0qErvCvzCaaUph7Sm37RhR4xvPrYI8Lgz6k85+A=", | ||
| 46 | - "owner": "NixOS", | ||
| 47 | - "repo": "nixpkgs", | ||
| 48 | - "rev": "d2f67949798825fe853f7c5d0492b8bf016d3f88", | ||
| 49 | - "type": "github" | ||
| 50 | - }, | ||
| 51 | - "original": { | ||
| 52 | - "owner": "NixOS", | ||
| 53 | - "ref": "nixos-unstable", | ||
| 54 | - "repo": "nixpkgs", | ||
| 55 | - "type": "github" | ||
| 56 | - } | ||
| 57 | - }, | ||
| 58 | - "root": { | ||
| 59 | - "inputs": { | ||
| 60 | - "nixgl": "nixgl", | ||
| 61 | - "nixpkgs": "nixpkgs" | ||
| 62 | - } | ||
| 63 | - }, | ||
| 64 | - "systems": { | ||
| 65 | - "locked": { | ||
| 66 | - "lastModified": 1681028828, | ||
| 67 | - "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=", | ||
| 68 | - "owner": "nix-systems", | ||
| 69 | - "repo": "default", | ||
| 70 | - "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e", | ||
| 71 | - "type": "github" | ||
| 72 | - }, | ||
| 73 | - "original": { | ||
| 74 | - "owner": "nix-systems", | ||
| 75 | - "repo": "default", | ||
| 76 | - "type": "github" | ||
| 77 | - } | ||
| 78 | - } | ||
| 79 | - }, | ||
| 80 | - "root": "root", | ||
| 81 | - "version": 7 | ||
| 82 | -} | ||
deleted
flake.nix +0 -642 | deleted file mode 100644 | ||
| @@ -1,642 +0,0 @@ | ||
| 1 | -{ | |
| 2 | - # frq is a Flutter app whose source is ClojureDart, so what this flake | |
| 3 | - # provides is toolchains rather than a built program: the `flutter` shell | |
| 4 | - # that `just apk` compiles in, the `flutter-desktop` shell for the Linux | |
| 5 | - # target, and the Android SDK the first of those copies somewhere writable. | |
| 6 | - # | |
| 7 | - # nix develop .#flutter-desktop --command just flutter-desktop run | |
| 8 | - # | |
| 9 | - # On a machine that is not NixOS the GL driver is the host's and the loader | |
| 10 | - # will not find it, so the window never opens ("GL display: argument does not | |
| 11 | - # name a valid config"). The recipes handle that themselves: off NixOS they | |
| 12 | - # hand the process to nixGL, which puts the host's driver ahead of the | |
| 13 | - # store's. A distrobox/container Arch is the same case as a bare one. | |
| 14 | - description = "frq — a freeq client in Flutter"; | |
| 15 | - | |
| 16 | - inputs = { | |
| 17 | - nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; | |
| 18 | - | |
| 19 | - # Only ever used off NixOS, to put the host GL driver on the loader path. | |
| 20 | - nixgl = { | |
| 21 | - url = "github:nix-community/nixGL"; | |
| 22 | - inputs.nixpkgs.follows = "nixpkgs"; | |
| 23 | - }; | |
| 24 | - }; | |
| 25 | - | |
| 26 | - outputs = { self, nixpkgs, nixgl }: | |
| 27 | - let | |
| 28 | - systems = [ "x86_64-linux" "aarch64-linux" ]; | |
| 29 | - forEachSystem = f: | |
| 30 | - nixpkgs.lib.genAttrs systems (system: f nixpkgs.legacyPackages.${system}); | |
| 31 | - | |
| 32 | - # Mesa, despite the name: it covers Intel and AMD alike. The NVIDIA | |
| 33 | - # wrappers are the ones that need --impure (they read the host kernel | |
| 34 | - # module's version), which is why this only ever reaches for Intel. | |
| 35 | - # | |
| 36 | - # Built from nixGL's default.nix rather than taken from its flake | |
| 37 | - # outputs, for the one argument the flake hardcodes on: `enable32bits`, | |
| 38 | - # which on x86_64 puts a second, i686 copy of mesa, its LLVM, and | |
| 39 | - # intel-media-driver into the wrapper. frq is 64-bit on both halves — | |
| 40 | - # the Rust cdylibs and the Chez runtime — so nothing here ever opens the | |
| 41 | - # 32-bit driver, and carrying it is most of the dev shell's closure. | |
| 42 | - nixGLFor = pkgs: (import nixgl { | |
| 43 | - inherit pkgs; | |
| 44 | - enable32bits = false; | |
| 45 | - }).nixGLIntel; | |
| 46 | - | |
| 47 | - # The Android SDK wants two things `nixpkgs.legacyPackages` cannot give: | |
| 48 | - # `allowUnfree`, because the SDK's own licence is not free, and | |
| 49 | - # `android_sdk.accept_license`, which is how you say so in a file rather | |
| 50 | - # than at a prompt a build has no terminal for. Neither can be set on a | |
| 51 | - # legacyPackages attribute after the fact, so this is a second import of | |
| 52 | - # the same locked nixpkgs rather than a second nixpkgs. | |
| 53 | - # | |
| 54 | - # This used to live in `just apk` as a `nix build --impure --expr` with | |
| 55 | - # `builtins.getFlake "github:NixOS/nixpkgs/nixos-unstable"` inside it — | |
| 56 | - # which fetched whatever nixos-unstable was that morning, not what | |
| 57 | - # flake.lock pins, so the SDK under the APK and the nixpkgs under | |
| 58 | - # everything else were free to drift apart. Here they are the same rev. | |
| 59 | - androidPkgsFor = system: import nixpkgs { | |
| 60 | - inherit system; | |
| 61 | - config = { | |
| 62 | - allowUnfree = true; | |
| 63 | - android_sdk.accept_license = true; | |
| 64 | - }; | |
| 65 | - }; | |
| 66 | - | |
| 67 | - # Only the floor Gradle stands on. It installs build-tools and a platform | |
| 68 | - # into ANDROID_HOME itself as it goes — see `just apk` for why that means | |
| 69 | - # a writable copy — so composing more of them here buys nothing. | |
| 70 | - # | |
| 71 | - # includeNDK = false deliberately: the app is Dart and path_provider is | |
| 72 | - # platform channels, so there is no native code to need one, and asking | |
| 73 | - # for it is a few hundred megabytes and a Gradle fetch of that exact NDK. | |
| 74 | - androidSdkFor = system: | |
| 75 | - let android = androidPkgsFor system; in | |
| 76 | - (android.androidenv.composeAndroidPackages { | |
| 77 | - cmdLineToolsVersion = "13.0"; | |
| 78 | - buildToolsVersions = [ "34.0.0" ]; | |
| 79 | - platformVersions = [ "35" "34" ]; | |
| 80 | - includeNDK = false; | |
| 81 | - }).androidsdk; | |
| 82 | - | |
| 83 | - in | |
| 84 | - { | |
| 85 | - packages = forEachSystem (pkgs: | |
| 86 | - let | |
| 87 | - inherit (pkgs) lib; | |
| 88 | - in | |
| 89 | - { | |
| 90 | - # The Android SDK `just apk` copies into flutter/.home. A package | |
| 91 | - # rather than something the recipe evaluates inline, so that | |
| 92 | - # `nix build .#android-sdk` is how you pre-warm it and `nix flake | |
| 93 | - # show` admits it exists. | |
| 94 | - android-sdk = androidSdkFor pkgs.stdenv.hostPlatform.system; | |
| 95 | - | |
| 96 | - # There were `appimage` outputs here, and what they were for was a | |
| 97 | - # host without Nix: they squashed the whole closure into one | |
| 98 | - # runnable file, Mesa included, and the Mesa was not waste — off | |
| 99 | - # NixOS the launcher goes through nixGL, which needs a store Mesa to | |
| 100 | - # put the host's driver in front of. Nothing asks for that shape any | |
| 101 | - # more, and they were the last thing evaluating nix-appimage, which | |
| 102 | - # is why that input is gone too. | |
| 103 | - | |
| 104 | - # Everything `clojure -M:cljd compile` would otherwise reach the | |
| 105 | - # network for, fetched once and hashed. | |
| 106 | - # | |
| 107 | - # The compile needs three caches, and the reason this is one | |
| 108 | - # derivation rather than three is that only one of them is obvious. | |
| 109 | - # Maven and gitlibs are the ordinary tools.deps pair. The third is | |
| 110 | - # ClojureDart's own: `ensure-cljd-analyzer!` writes a *second*, whole | |
| 111 | - # pub project to `.clojuredart/cache/<cljd sha>/cljd_helper`, runs | |
| 112 | - # `pub add analyzer` in it, and then runs `bin/analyzer.dart` out of | |
| 113 | - # it for the duration of the compile — so a sandbox needs that | |
| 114 | - # project already resolved, not just the app's dependencies. | |
| 115 | - # | |
| 116 | - # Fixed-output, so it is allowed the network the rest of the build is | |
| 117 | - # not. What that costs is a hash to maintain, and the thing worth | |
| 118 | - # being exact about is *when*: this derivation never sees frq's | |
| 119 | - # source. It compiles a three-line throwaway project against the same | |
| 120 | - # `flutter/deps.edn` and the same `flutter/pubspec.yaml`, so the hash | |
| 121 | - # moves when a dependency moves and not when a screen changes. A | |
| 122 | - # stub, rather than `-P` and a hand-built analyzer dir, because | |
| 123 | - # running the real compiler once is the only way to be sure the | |
| 124 | - # caches are the ones it actually wants. | |
| 125 | - # | |
| 126 | - # PUB_CACHE lands in $out on purpose. The package_config.json inside | |
| 127 | - # cljd_helper carries absolute paths to whatever resolved it, so | |
| 128 | - # resolving into a build directory would bake in paths that stop | |
| 129 | - # existing the moment this derivation finishes. Pointed at $out they | |
| 130 | - # are store paths, and still true. | |
| 131 | - cljd-deps = | |
| 132 | - let | |
| 133 | - flutterPkg = pkgs.flutter; | |
| 134 | - in | |
| 135 | - pkgs.stdenvNoCC.mkDerivation { | |
| 136 | - name = "frq-cljd-deps"; | |
| 137 | - dontUnpack = true; | |
| 138 | - | |
| 139 | - nativeBuildInputs = [ | |
| 140 | - pkgs.clojure | |
| 141 | - pkgs.jdk17 | |
| 142 | - flutterPkg | |
| 143 | - pkgs.git | |
| 144 | - pkgs.cacert | |
| 145 | - ]; | |
| 146 | - | |
| 147 | - buildCommand = '' | |
| 148 | - export HOME="$NIX_BUILD_TOP/home" | |
| 149 | - # Resolved in the build directory and copied to $out at the | |
| 150 | - # end, never written there directly. A fixed-output derivation | |
| 151 | - # may not reference a store path and its own output is a store | |
| 152 | - # path, so pub writing its cache's absolute location into its | |
| 153 | - # own metadata is enough to fail the check. | |
| 154 | - cache="$NIX_BUILD_TOP/cache" | |
| 155 | - export PUB_CACHE="$cache/pub-cache" | |
| 156 | - export GITLIBS="$cache/gitlibs" | |
| 157 | - export SSL_CERT_FILE="${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt" | |
| 158 | - mkdir -p "$HOME" "$PUB_CACHE" "$GITLIBS" "$cache/m2" | |
| 159 | - | |
| 160 | - # The stub: our dependency files, nothing of our source. | |
| 161 | - # ../common is a :local/root dependency now, so it has to exist | |
| 162 | - # *and* carry a deps.edn for tools.deps to resolve — an empty | |
| 163 | - # directory with that one file in it is enough. | |
| 164 | - proj="$NIX_BUILD_TOP/stub" | |
| 165 | - mkdir -p "$proj/src/stub" "$NIX_BUILD_TOP/common" | |
| 166 | - cp ${./common/deps.edn} "$NIX_BUILD_TOP/common/deps.edn" | |
| 167 | - cp ${./flutter/deps.edn} "$proj/deps.edn" | |
| 168 | - cp ${./flutter/pubspec.yaml} "$proj/pubspec.yaml" | |
| 169 | - # The lock, or `pub get` resolves against pub.dev and takes | |
| 170 | - # whatever satisfies the ranges today. Every build then fetches | |
| 171 | - # a slightly different set and the fixed-output hash is a | |
| 172 | - # promise nothing can keep. | |
| 173 | - cp ${./flutter/pubspec.lock} "$proj/pubspec.lock" | |
| 174 | - chmod u+w "$proj/deps.edn" "$proj/pubspec.yaml" "$proj/pubspec.lock" | |
| 175 | - cat > "$proj/src/stub/main.cljd" <<'EOF' | |
| 176 | - (ns stub.main) | |
| 177 | - (defn main [] nil) | |
| 178 | - EOF | |
| 179 | - | |
| 180 | - cd "$proj" | |
| 181 | - # `:main` has to name the stub, or the compiler goes looking for | |
| 182 | - # frq.main in a tree that is not here. | |
| 183 | - sed -i 's/:main frq\.main/:main stub.main/' deps.edn | |
| 184 | - | |
| 185 | - flutter config --no-analytics &>/dev/null || true | |
| 186 | - flutter config --enable-linux-desktop >/dev/null || true | |
| 187 | - | |
| 188 | - clojure -Sdeps '{:mvn/local-repo "'"$cache"'/m2"}' -M:cljd compile | |
| 189 | - | |
| 190 | - # What the compile left behind, and only that. The analyzer | |
| 191 | - # project is keyed by the ClojureDart sha, so the directory | |
| 192 | - # under cache/ is copied wholesale rather than named here. | |
| 193 | - mkdir -p "$out/clojuredart" | |
| 194 | - cp -r .clojuredart/cache "$out/clojuredart/cache" | |
| 195 | - cp -r "$cache/m2" "$out/m2" | |
| 196 | - cp -r "$cache/gitlibs" "$out/gitlibs" | |
| 197 | - cp -r "$PUB_CACHE" "$out/pub-cache" | |
| 198 | - | |
| 199 | - # A fixed-output derivation may not reference a store path, and | |
| 200 | - # a resolved pub project is nothing but store paths: | |
| 201 | - # package_config.json names the Flutter SDK and every package | |
| 202 | - # in the cache by absolute path. So the analyzer project ships | |
| 203 | - # *unresolved* — its pubspec and its analyzer.dart and nothing | |
| 204 | - # else — and `flutter pub get --offline` re-resolves it against | |
| 205 | - # this cache at build time, where naming the store is allowed. | |
| 206 | - find "$out" \( -name '.dart_tool' -o -name '.flutter-plugins' \ | |
| 207 | - -o -name '.flutter-plugins-dependencies' \) -prune -exec rm -rf {} + | |
| 208 | - find "$out" -name '.packages' -delete | |
| 209 | - | |
| 210 | - | |
| 211 | - # A fixed-output hash is a promise that two runs agree, so | |
| 212 | - # everything a tool writes *about* a run rather than about a | |
| 213 | - # dependency has to go: pub's log carries timestamps, Maven | |
| 214 | - # rewrites its resolution metadata on every resolve, and | |
| 215 | - # tools.gitlibs keeps bare clones it only needs in order to | |
| 216 | - # make a checkout. None of it is read offline. | |
| 217 | - # | |
| 218 | - # active_roots is the one that was actually breaking this. Pub | |
| 219 | - # records the project directories using the cache, sharded by | |
| 220 | - # a hash of the path, and $NIX_BUILD_TOP is different on every | |
| 221 | - # run -- so two builds whose hosted/ trees were byte-identical | |
| 222 | - # still disagreed, purely over which directory had asked. Four | |
| 223 | - # builds gave four hashes until this went. | |
| 224 | - rm -rf "$out/pub-cache/log" "$out/pub-cache/_temp" \ | |
| 225 | - "$out/pub-cache/git" "$out/pub-cache/global_packages" \ | |
| 226 | - "$out/pub-cache/bin" "$out/pub-cache/active_roots" | |
| 227 | - | |
| 228 | - # tools.gitlibs keeps a bare clone per URL under _repos/, and a | |
| 229 | - # bare clone is packfiles — which two runs of the same fetch do | |
| 230 | - # not have to produce byte for byte. It cannot simply be | |
| 231 | - # deleted, because `procure` calls `ensure-git-dir` before it | |
| 232 | - # looks at anything else and would clone it again, over a | |
| 233 | - # network this has and the build that uses it does not. | |
| 234 | - # | |
| 235 | - # It does not need the objects, though. `procure` finds the sha | |
| 236 | - # with `match-exact` against the checkout already in libs/, so | |
| 237 | - # the bare repo only has to exist. Emptied and re-initialised, | |
| 238 | - # it is a fixed handful of files from the pinned git and the | |
| 239 | - # same on every run. | |
| 240 | - find "$out/gitlibs/_repos" -name HEAD | while read -r head; do | |
| 241 | - repo="$(dirname "$head")" | |
| 242 | - rm -rf "$repo" | |
| 243 | - git init --bare -q "$repo" | |
| 244 | - # The sample hooks are shell scripts, so they carry a | |
| 245 | - # `#!/nix/store/.../bash` line — which is exactly the kind of | |
| 246 | - # store reference a fixed-output derivation may not hold. An | |
| 247 | - # empty bare repo nothing ever runs has no use for them. | |
| 248 | - rm -rf "$repo/hooks" | |
| 249 | - done | |
| 250 | - # pub's version listings, which record when they were fetched. | |
| 251 | - # This is the one that actually moved between two runs of this | |
| 252 | - # derivation: the package sources under hosted/ were identical | |
| 253 | - # and the listings beside them were not. Nothing offline reads | |
| 254 | - # them — a resolution that already has every package on disk | |
| 255 | - # never asks pub.dev what versions exist. | |
| 256 | - find "$out/pub-cache" -name '.cache' -type d -prune -exec rm -rf {} + | |
| 257 | - find "$out/m2" \( -name '*.lastUpdated' -o -name '_remote.repositories' \ | |
| 258 | - -o -name 'resolver-status.properties' -o -name '*.part' \ | |
| 259 | - -o -name 'maven-metadata-*.xml*' \) -delete | |
| 260 | - find "$out" \( -name '.DS_Store' -o -name '*.log' -o -name '.git' \) \ | |
| 261 | - -prune -exec rm -rf {} + | |
| 262 | - find "$out" -type d -empty -delete | |
| 263 | - chmod -R u+w "$out" | |
| 264 | - | |
| 265 | - # Last, after every cleanup above: anything still naming the | |
| 266 | - # store fails the fixed-output check, and the error names one | |
| 267 | - # path out of thousands of files. This names the files. | |
| 268 | - if refs="$(grep -rlI /nix/store "$out" 2>/dev/null)" && [ -n "$refs" ]; then | |
| 269 | - echo "cljd-deps: these still reference the store:" >&2 | |
| 270 | - echo "$refs" | head -20 >&2 | |
| 271 | - fi | |
| 272 | - | |
| 273 | - # If two runs disagree, this says which half to look in. Cheap, | |
| 274 | - # and the alternative is a hash mismatch with nothing attached. | |
| 275 | - for d in "$out"/*; do | |
| 276 | - echo "cljd-deps subtree $(basename "$d") $( (cd "$d" && find . -type f \ | |
| 277 | - -exec sha256sum {} + | sort -k2 | sha256sum) )" >&2 | |
| 278 | - done | |
| 279 | - for d in "$out"/pub-cache/*/*; do | |
| 280 | - [ -d "$d" ] || continue | |
| 281 | - # The path relative to $out, not the basename: two runs that | |
| 282 | - # disagree here disagree about *which* directory exists, and | |
| 283 | - # a bare `09` names nothing you can go and look at. | |
| 284 | - echo "cljd-deps pub ''${d#$out/} $( (cd "$d" && find . -type f \ | |
| 285 | - -exec sha256sum {} + | sort -k2 | sha256sum) )" >&2 | |
| 286 | - done | |
| 287 | - ''; | |
| 288 | - | |
| 289 | - outputHashMode = "recursive"; | |
| 290 | - outputHashAlgo = "sha256"; | |
| 291 | - # Moves when flutter/deps.edn or flutter/pubspec.yaml move, and | |
| 292 | - # not when frq's own source does — see the stub above. | |
| 293 | - outputHash = "sha256-qSGx7WFdVyV7yu4R+EjiQHZcLqwDjYSlohiZcXB43DY="; | |
| 294 | - }; | |
| 295 | - | |
| 296 | - # The Flutter desktop GUI, built rather than run out of the tree. | |
| 297 | - # | |
| 298 | - # `just flutter-desktop` is the working-tree loop and this is its | |
| 299 | - # opposite number: the source is the flake's, the output is a store | |
| 300 | - # path, and the build is a sandbox with no network. It is the first thing here that builds | |
| 301 | - # purely — the APK cannot, because Gradle fetches as it goes. | |
| 302 | - # | |
| 303 | - # Two stages, because the Dart does not exist until ClojureDart writes | |
| 304 | - # it. `preBuild` runs the compiler over `flutter/src` and `common/` | |
| 305 | - # with `--offline`, out of the caches `cljd-deps` fetched; everything | |
| 306 | - # after that is an ordinary Flutter application as far as nixpkgs is | |
| 307 | - # concerned. | |
| 308 | - # | |
| 309 | - # The caches are copied in rather than used where they lie. Maven, | |
| 310 | - # tools.gitlibs and pub all expect to be able to write to their own | |
| 311 | - # cache — a lock file, a resolved marker — and the store is read-only, | |
| 312 | - # so pointing them at $out of a fixed-output derivation fails in three | |
| 313 | - # different ways at three different depths. | |
| 314 | - # | |
| 315 | - # `src` is the whole tree and not `flutter/`: `flutter/deps.edn` puts | |
| 316 | - # `../common` on the classpath, which is the entire point of that | |
| 317 | - # directory, and a source root of `flutter/` would leave the screens | |
| 318 | - # outside it. | |
| 319 | - flutter-desktop-unwrapped = pkgs.flutter.buildFlutterApplication rec { | |
| 320 | - pname = "frq-flutter"; | |
| 321 | - version = "0.1.0"; | |
| 322 | - | |
| 323 | - src = lib.cleanSourceWith { | |
| 324 | - src = ./.; | |
| 325 | - # Two trees, and only two: `flutter/` is the app and `common/` | |
| 326 | - # is the screens its deps.edn puts on the classpath. The root is | |
| 327 | - # still the source root because of that `../common`, but letting | |
| 328 | - # the *whole* root in means every file in the repo is an input — | |
| 329 | - # so editing flake.nix or CLAUDE.md | |
| 330 | - # invalidated the entire Dart compile and paid ten minutes for a | |
| 331 | - # change the Flutter build cannot even see. | |
| 332 | - # | |
| 333 | - # Matched on the path relative to the root rather than on | |
| 334 | - # basename: `src` as a basename would also exclude `flutter/src` | |
| 335 | - # and `common/src`, which is everything that matters. | |
| 336 | - filter = | |
| 337 | - let root = toString ./.; in | |
| 338 | - path: type: | |
| 339 | - let | |
| 340 | - rel = lib.removePrefix (root + "/") (toString path); | |
| 341 | - inTree = d: rel == d || lib.hasPrefix (d + "/") rel; | |
| 342 | - in | |
| 343 | - (inTree "flutter" || inTree "common") | |
| 344 | - # Build trees and caches, which are large, machine-specific | |
| 345 | - # and would make every one of them a new store path. | |
| 346 | - && !(builtins.elem (baseNameOf path) [ | |
| 347 | - "build" ".home" ".clojuredart" ".cpcache" "cljd-out" | |
| 348 | - ".dart_tool" "result" ".git" "buck-out" | |
| 349 | - ]); | |
| 350 | - }; | |
| 351 | - sourceRoot = "source/flutter"; | |
| 352 | - | |
| 353 | - # Read at eval time, so the lock in git is the lock that is built. | |
| 354 | - autoPubspecLock = ./flutter/pubspec.lock; | |
| 355 | - | |
| 356 | - # git, because tools.deps resolves the ClojureDart dependency through | |
| 357 | - # tools.gitlibs even when every byte of it is already on disk — see | |
| 358 | - # the _repos note in cljd-deps. | |
| 359 | - nativeBuildInputs = [ pkgs.clojure pkgs.jdk17 pkgs.git ]; | |
| 360 | - | |
| 361 | - preBuild = '' | |
| 362 | - export PUB_CACHE="$NIX_BUILD_TOP/pub-cache" | |
| 363 | - export GITLIBS="$NIX_BUILD_TOP/gitlibs" | |
| 364 | - cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/pub-cache "$PUB_CACHE" | |
| 365 | - cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/gitlibs "$GITLIBS" | |
| 366 | - cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/m2 "$NIX_BUILD_TOP/m2" | |
| 367 | - mkdir -p .clojuredart | |
| 368 | - cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/clojuredart/cache .clojuredart/cache | |
| 369 | - chmod -R u+w "$PUB_CACHE" "$GITLIBS" "$NIX_BUILD_TOP/m2" .clojuredart | |
| 370 | - | |
| 371 | - # Resolve the analyzer project here rather than in cljd-deps, | |
| 372 | - # which was not allowed to name the store. Offline, out of the | |
| 373 | - # cache that derivation did fetch. ClojureDart only reaches for | |
| 374 | - # the network when `bin/analyzer.dart` is missing, and it is not. | |
| 375 | - for helper in .clojuredart/cache/*/cljd_helper; do | |
| 376 | - ( cd "$helper" && flutter pub get --offline ) | |
| 377 | - done | |
| 378 | - | |
| 379 | - # --offline is what keeps `pub get` out of a sandbox that has no | |
| 380 | - # network; the analyzer project it would otherwise resolve is | |
| 381 | - # already in .clojuredart, put there by cljd-deps. | |
| 382 | - clojure -Sdeps "{:mvn/local-repo \"$NIX_BUILD_TOP/m2\"}" \ | |
| 383 | - -M:cljd compile --offline | |
| 384 | - ''; | |
| 385 | - | |
| 386 | - meta = { | |
| 387 | - description = "frq's screens on Flutter's Linux target (no GL launcher)"; | |
| 388 | - mainProgram = "frq"; | |
| 389 | - platforms = systems; | |
| 390 | - }; | |
| 391 | - }; | |
| 392 | - | |
| 393 | - # The same shape as `frq` above: a launcher, and a package that is a | |
| 394 | - # symlink to it. The reason is the same one `frqScript` gives — on | |
| 395 | - # NixOS the store's Mesa is the system's and the window opens, and | |
| 396 | - # anywhere else the real driver is the host's, so the process is | |
| 397 | - # handed to nixGL. Without it the store build dies on a distrobox | |
| 398 | - # Arch with "No provider of eglGetPlatformDisplayEXT found", which is | |
| 399 | - # that failure wearing an EGL hat. | |
| 400 | - # | |
| 401 | - # A wrapper *around* the built application rather than a `postFixup` | |
| 402 | - # inside it, because buildFlutterApplication's own dartFixupHook runs | |
| 403 | - # after postFixup and rewrites `bin/frq` — so anything done to that | |
| 404 | - # path from inside is undone on the way out. | |
| 405 | - flutter-desktop = | |
| 406 | - let | |
| 407 | - unwrapped = | |
| 408 | - self.packages.${pkgs.stdenv.hostPlatform.system}.flutter-desktop-unwrapped; | |
| 409 | - | |
| 410 | - # buildFlutterApplication's own wrapper appends a bare `/lib` to | |
| 411 | - # LD_LIBRARY_PATH — the host's, not the bundle's. Off NixOS that | |
| 412 | - # is a foreign library directory in front of nothing, and the app | |
| 413 | - # dies in the loader before main: first | |
| 414 | - # | |
| 415 | - # /lib/libc.so.6: undefined symbol: __pointer_chk_guard | |
| 416 | - # | |
| 417 | - # and, once the store's glibc is put ahead of it, | |
| 418 | - # | |
| 419 | - # libc.so.6: version `GLIBC_2.43' not found | |
| 420 | - # (required by /lib/libglib-2.0.so.0) | |
| 421 | - # | |
| 422 | - # which is the same bug wearing the other hat: the host's glib | |
| 423 | - # against the store's glibc. Ordering cannot fix a mixture, so | |
| 424 | - # the entry goes rather than moves. The wrapper is generated, so | |
| 425 | - # this edits a copy and asserts the edit landed — a silent miss | |
| 426 | - # here is a runtime failure on someone else's machine. | |
| 427 | - fixed = pkgs.runCommand "frq-flutter-wrapper" { } '' | |
| 428 | - mkdir -p "$out/bin" | |
| 429 | - sed "s|'/lib'||g" ${unwrapped}/bin/frq > "$out/bin/frq" | |
| 430 | - chmod +x "$out/bin/frq" | |
| 431 | - if grep -q "'/lib'" "$out/bin/frq"; then | |
| 432 | - echo "the /lib entry outlived the edit; look at the wrapper" >&2 | |
| 433 | - exit 1 | |
| 434 | - fi | |
| 435 | - ''; | |
| 436 | - | |
| 437 | - script = pkgs.writeShellScript "frq" '' | |
| 438 | - runner="" | |
| 439 | - [ -e /run/current-system ] || runner="${nixGLFor pkgs}/bin/nixGLIntel" | |
| 440 | - exec ''${runner} ${fixed}/bin/frq "$@" | |
| 441 | - ''; | |
| 442 | - in | |
| 443 | - pkgs.runCommand "frq-flutter-0.1.0" | |
| 444 | - { | |
| 445 | - meta = { | |
| 446 | - description = "frq's screens on Flutter's Linux target"; | |
| 447 | - mainProgram = "frq"; | |
| 448 | - platforms = systems; | |
| 449 | - }; | |
| 450 | - } | |
| 451 | - '' | |
| 452 | - mkdir -p "$out/bin" | |
| 453 | - ln -s ${script} "$out/bin/frq" | |
| 454 | - ''; | |
| 455 | - }); | |
| 456 | - | |
| 457 | - devShells = forEachSystem (pkgs: | |
| 458 | - let | |
| 459 | - inherit (pkgs) lib; | |
| 460 | - in | |
| 461 | - { | |
| 462 | - # Dart on its own, for the tests that need no Flutter: the FFI | |
| 463 | - # binding to the Nim core runs on the plain VM, and making it wait | |
| 464 | - # for a Flutter toolchain would throw away the reason it is fast. | |
| 465 | - # | |
| 466 | - # Flutter bundles a Dart, so this is a duplicate in one sense. It is | |
| 467 | - # also thirty times smaller, and the point of the boundary is that | |
| 468 | - # you can check it without the thing on the other side of it. | |
| 469 | - dart = pkgs.mkShellNoCC { | |
| 470 | - name = "frq-dart"; | |
| 471 | - packages = [ pkgs.dart pkgs.just ]; | |
| 472 | - | |
| 473 | - # libfrqcore.so is linked against OpenSSL, and the process that | |
| 474 | - # dlopens it has to be able to find one. Named here rather than | |
| 475 | - # left to the host: a machine whose libssl is a different soname | |
| 476 | - # fails at `frq_init` with a message about the wrong library. | |
| 477 | - LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.openssl ]; | |
| 478 | - | |
| 479 | - FRQ_DART = "1"; | |
| 480 | - }; | |
| 481 | - | |
| 482 | - # Nim, for `nim/` — the portable core as a native library. Just the | |
| 483 | - # compiler: the core has no dependencies outside Nim's own standard | |
| 484 | - # library, deliberately, because a dependency here is one that has | |
| 485 | - # to cross-compile to every target the Dart side runs on. | |
| 486 | - # | |
| 487 | - # `nim c` shells out to a C compiler, so this is mkShell and not | |
| 488 | - # mkShellNoCC: stdenv brings the one Nim will find. | |
| 489 | - nim = pkgs.mkShell { | |
| 490 | - name = "frq-nim"; | |
| 491 | - packages = [ pkgs.nim pkgs.just ]; | |
| 492 | - | |
| 493 | - # OpenSSL, because `-d:ssl` in nim/nim.cfg makes std/net link | |
| 494 | - # -lssl and -lcrypto: the IRC connection is TLS on :6697, which is | |
| 495 | - # the only port freeq actually listens on. | |
| 496 | - buildInputs = [ pkgs.openssl ]; | |
| 497 | - | |
| 498 | - # And on the loader path as well as the linker's. Nim resolves the | |
| 499 | - # OpenSSL entry points through dynlib at run time, so without this | |
| 500 | - # `newContext` finds nothing behind the symbol and dies with a | |
| 501 | - # SIGSEGV that says nothing about SSL at all. | |
| 502 | - LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.openssl ]; | |
| 503 | - | |
| 504 | - # The recipe's re-entry test, the way FRQ_FLUTTER_DESKTOP is the | |
| 505 | - # desktop one's. | |
| 506 | - FRQ_NIM = "1"; | |
| 507 | - }; | |
| 508 | - | |
| 509 | - # The APK toolchain. It is not in a default shell any more because | |
| 510 | - # there is no default shell: what used to be one belonged to the | |
| 511 | - # libcosmic frontend, and a Flutter build asks for a toolchain by | |
| 512 | - # name. | |
| 513 | - # | |
| 514 | - # Flutter brings its own Dart, Gradle and a JDK's worth of | |
| 515 | - # closure: Flutter brings its own Dart, Gradle and a JDK's worth of | |
| 516 | - # closure, and a desktop build has no use for any of it. | |
| 517 | - # | |
| 518 | - # `just apk` used to name these as `nix shell nixpkgs#clojure | |
| 519 | - # nixpkgs#jdk17 nixpkgs#flutter`, which is the flake registry's | |
| 520 | - # nixpkgs and not this flake's — so the Flutter under the APK | |
| 521 | - # floated while everything else was locked. Same three packages, | |
| 522 | - # from flake.lock now. | |
| 523 | - # | |
| 524 | - # JDK 17 and not newer on purpose: the Flutter template's Gradle | |
| 525 | - # plugin pins a Gradle that rejects a JDK it was released before, | |
| 526 | - # and the failure reads as an unsupported class file version rather | |
| 527 | - # than as a version mismatch. | |
| 528 | - flutter = pkgs.mkShellNoCC { | |
| 529 | - name = "frq-flutter"; | |
| 530 | - | |
| 531 | - # git, because tools.deps resolves the ClojureDart dependency | |
| 532 | - # through tools.gitlibs even when every byte of it is already in | |
| 533 | - # the seeded cache — the same reason flutter-desktop-unwrapped | |
| 534 | - # names it. The host's git has always been there to answer; naming | |
| 535 | - # it means the shell does not depend on that. | |
| 536 | - packages = [ pkgs.clojure pkgs.jdk17 pkgs.flutter pkgs.just pkgs.git ]; | |
| 537 | - | |
| 538 | - # Where the recipe copies from. Naming it here is also what makes | |
| 539 | - # entering the shell build it, so the first `just apk` does not | |
| 540 | - # stop for a few hundred megabytes of SDK with nothing said about | |
| 541 | - # why. | |
| 542 | - FRQ_ANDROID_SDK = | |
| 543 | - "${androidSdkFor pkgs.stdenv.hostPlatform.system}/libexec/android-sdk"; | |
| 544 | - | |
| 545 | - # The Maven, gitlibs and pub caches the ClojureDart compile would | |
| 546 | - # otherwise fetch, plus the analyzer project it writes under | |
| 547 | - # .clojuredart. Here for both of FRQ_ANDROID_SDK's reasons: it is | |
| 548 | - # where the recipe copies from, and naming it is what makes | |
| 549 | - # entering the shell build it. | |
| 550 | - # | |
| 551 | - # The compile still runs online. These are a warm start and not a | |
| 552 | - # pin — `--offline` would be, and would turn adding a line to | |
| 553 | - # flutter/deps.edn into a re-hash of cljd-deps before anything | |
| 554 | - # compiled again. The sandbox build takes that trade because it | |
| 555 | - # has no network; the loop someone edits in should not. | |
| 556 | - FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}"; | |
| 557 | - }; | |
| 558 | - | |
| 559 | - # The other desktop GUI. Same ClojureDart half as the APK — one | |
| 560 | - # `clojure -M:cljd compile`, one flutter/src — over Flutter's Linux | |
| 561 | - # target instead of its Android one, so `flutter/linux/` is the | |
| 562 | - # runner and CMake and Ninja are the build rather than Gradle. | |
| 563 | - # | |
| 564 | - # A separate shell from `flutter` rather than one that carries both, | |
| 565 | - # because the halves are disjoint: this wants GTK and a C++ toolchain | |
| 566 | - # and no JDK, and the APK wants a JDK and an SDK and no GTK. Sharing | |
| 567 | - # them would mean every desktop build paying for a few hundred | |
| 568 | - # megabytes of Android SDK it never opens, which is the same argument | |
| 569 | - # that keeps Flutter out of the default shell. | |
| 570 | - # | |
| 571 | - # mkShell and not mkShellNoCC, unlike every other shell here: this is | |
| 572 | - # the one that actually compiles C++. stdenv brings the compiler, and | |
| 573 | - # gtk3 in buildInputs is what puts its .pc file where the runner's | |
| 574 | - # `pkg_check_modules(GTK gtk+-3.0)` can find it. | |
| 575 | - flutter-desktop = pkgs.mkShell { | |
| 576 | - name = "frq-flutter-desktop"; | |
| 577 | - | |
| 578 | - # clojure and flutter are the APK shell's, and deliberately the | |
| 579 | - # same two: the Dart that runs here is generated by the same | |
| 580 | - # compiler from the same source, and a second Flutter version | |
| 581 | - # under it would be a second set of engine artifacts and a second | |
| 582 | - # answer to "does the phone build match the desktop one". | |
| 583 | - nativeBuildInputs = [ | |
| 584 | - pkgs.clojure | |
| 585 | - pkgs.flutter | |
| 586 | - pkgs.just | |
| 587 | - pkgs.cmake | |
| 588 | - pkgs.ninja | |
| 589 | - pkgs.pkg-config | |
| 590 | - ]; | |
| 591 | - | |
| 592 | - # gtk3 is the runner's own dependency; the rest are url_launcher's | |
| 593 | - # Linux implementation, which is a GTK plugin compiled into the | |
| 594 | - # bundle. path_provider needs nothing here — its Linux half is | |
| 595 | - # pure Dart over the XDG directories. | |
| 596 | - buildInputs = [ pkgs.gtk3 pkgs.glib ]; | |
| 597 | - | |
| 598 | - # Where libfrqcore.so's OpenSSL lives. A named variable and NOT | |
| 599 | - # LD_LIBRARY_PATH, deliberately: this shell also runs Flutter | |
| 600 | - # through nixGL, which does its own careful things to the loader | |
| 601 | - # path, and a blanket LD_LIBRARY_PATH here is the sort of thing | |
| 602 | - # that breaks GL on one machine and not another. The `nim-spike` | |
| 603 | - # recipe prepends this for the app it launches and nothing else. | |
| 604 | - FRQ_OPENSSL_LIB = lib.makeLibraryPath [ pkgs.openssl ]; | |
| 605 | - | |
| 606 | - # Flutter paints through GL, and off NixOS the driver that can do | |
| 607 | - # that is the host's, not the store's. | |
| 608 | - NIXGL = "${nixGLFor pkgs}/bin/nixGLIntel"; | |
| 609 | - | |
| 610 | - # The `flutter` shell's, deliberately the same one and for the | |
| 611 | - # same reason clojure and flutter are: the ClojureDart half of | |
| 612 | - # both builds is one compile over one deps.edn, so a second set of | |
| 613 | - # caches would be a second answer to what it resolved against. | |
| 614 | - # This recipe reads the variable directly — it is inside this | |
| 615 | - # shell before it does any of the work — where `just apk` reaches | |
| 616 | - # for the flake output itself. | |
| 617 | - FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}"; | |
| 618 | - | |
| 619 | - # The recipe's re-entry test. Nothing else sets it, so `just flutter-desktop` | |
| 620 | - # outside the shell re-enters and lands back on the same recipe — | |
| 621 | - # no flag to forget, and no second code path for someone who runs | |
| 622 | - # `nix develop .#flutter-desktop --command just flutter-desktop` | |
| 623 | - # by hand. | |
| 624 | - FRQ_FLUTTER_DESKTOP = "1"; | |
| 625 | - }; | |
| 626 | - | |
| 627 | - # No `flutter-web` shell here any more. The web target was the one | |
| 628 | - # that needed nothing of the host -- no JDK and no Android SDK as | |
| 629 | - # the APK wants, no GTK and no C++ and no nixGL as the desktop one | |
| 630 | - # does -- and a devShell whose only job is to hand over a Dart and | |
| 631 | - # a JVM is a devShell that a pinned tarball can replace. It did: | |
| 632 | - # `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by | |
| 633 | - # sha256, `tools/build-web.sh` builds out of them, and | |
| 634 | - # `.modal/flutter-web/` runs that same script on a plain Debian | |
| 635 | - # image with no store to populate. | |
| 636 | - # | |
| 637 | - # The two shells above stay. What they supply is a host toolchain, | |
| 638 | - # which is exactly what nix is better at than a tarball. | |
| 639 | - }); | |
| 640 | - | |
| 641 | - }; | |
| 642 | -} | |
| deleted file mode 100644 | |||
| @@ -1,642 +0,0 @@ | |||
| 1 | -{ | ||
| 2 | - # frq is a Flutter app whose source is ClojureDart, so what this flake | ||
| 3 | - # provides is toolchains rather than a built program: the `flutter` shell | ||
| 4 | - # that `just apk` compiles in, the `flutter-desktop` shell for the Linux | ||
| 5 | - # target, and the Android SDK the first of those copies somewhere writable. | ||
| 6 | - # | ||
| 7 | - # nix develop .#flutter-desktop --command just flutter-desktop run | ||
| 8 | - # | ||
| 9 | - # On a machine that is not NixOS the GL driver is the host's and the loader | ||
| 10 | - # will not find it, so the window never opens ("GL display: argument does not | ||
| 11 | - # name a valid config"). The recipes handle that themselves: off NixOS they | ||
| 12 | - # hand the process to nixGL, which puts the host's driver ahead of the | ||
| 13 | - # store's. A distrobox/container Arch is the same case as a bare one. | ||
| 14 | - description = "frq — a freeq client in Flutter"; | ||
| 15 | - | ||
| 16 | - inputs = { | ||
| 17 | - nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; | ||
| 18 | - | ||
| 19 | - # Only ever used off NixOS, to put the host GL driver on the loader path. | ||
| 20 | - nixgl = { | ||
| 21 | - url = "github:nix-community/nixGL"; | ||
| 22 | - inputs.nixpkgs.follows = "nixpkgs"; | ||
| 23 | - }; | ||
| 24 | - }; | ||
| 25 | - | ||
| 26 | - outputs = { self, nixpkgs, nixgl }: | ||
| 27 | - let | ||
| 28 | - systems = [ "x86_64-linux" "aarch64-linux" ]; | ||
| 29 | - forEachSystem = f: | ||
| 30 | - nixpkgs.lib.genAttrs systems (system: f nixpkgs.legacyPackages.${system}); | ||
| 31 | - | ||
| 32 | - # Mesa, despite the name: it covers Intel and AMD alike. The NVIDIA | ||
| 33 | - # wrappers are the ones that need --impure (they read the host kernel | ||
| 34 | - # module's version), which is why this only ever reaches for Intel. | ||
| 35 | - # | ||
| 36 | - # Built from nixGL's default.nix rather than taken from its flake | ||
| 37 | - # outputs, for the one argument the flake hardcodes on: `enable32bits`, | ||
| 38 | - # which on x86_64 puts a second, i686 copy of mesa, its LLVM, and | ||
| 39 | - # intel-media-driver into the wrapper. frq is 64-bit on both halves — | ||
| 40 | - # the Rust cdylibs and the Chez runtime — so nothing here ever opens the | ||
| 41 | - # 32-bit driver, and carrying it is most of the dev shell's closure. | ||
| 42 | - nixGLFor = pkgs: (import nixgl { | ||
| 43 | - inherit pkgs; | ||
| 44 | - enable32bits = false; | ||
| 45 | - }).nixGLIntel; | ||
| 46 | - | ||
| 47 | - # The Android SDK wants two things `nixpkgs.legacyPackages` cannot give: | ||
| 48 | - # `allowUnfree`, because the SDK's own licence is not free, and | ||
| 49 | - # `android_sdk.accept_license`, which is how you say so in a file rather | ||
| 50 | - # than at a prompt a build has no terminal for. Neither can be set on a | ||
| 51 | - # legacyPackages attribute after the fact, so this is a second import of | ||
| 52 | - # the same locked nixpkgs rather than a second nixpkgs. | ||
| 53 | - # | ||
| 54 | - # This used to live in `just apk` as a `nix build --impure --expr` with | ||
| 55 | - # `builtins.getFlake "github:NixOS/nixpkgs/nixos-unstable"` inside it — | ||
| 56 | - # which fetched whatever nixos-unstable was that morning, not what | ||
| 57 | - # flake.lock pins, so the SDK under the APK and the nixpkgs under | ||
| 58 | - # everything else were free to drift apart. Here they are the same rev. | ||
| 59 | - androidPkgsFor = system: import nixpkgs { | ||
| 60 | - inherit system; | ||
| 61 | - config = { | ||
| 62 | - allowUnfree = true; | ||
| 63 | - android_sdk.accept_license = true; | ||
| 64 | - }; | ||
| 65 | - }; | ||
| 66 | - | ||
| 67 | - # Only the floor Gradle stands on. It installs build-tools and a platform | ||
| 68 | - # into ANDROID_HOME itself as it goes — see `just apk` for why that means | ||
| 69 | - # a writable copy — so composing more of them here buys nothing. | ||
| 70 | - # | ||
| 71 | - # includeNDK = false deliberately: the app is Dart and path_provider is | ||
| 72 | - # platform channels, so there is no native code to need one, and asking | ||
| 73 | - # for it is a few hundred megabytes and a Gradle fetch of that exact NDK. | ||
| 74 | - androidSdkFor = system: | ||
| 75 | - let android = androidPkgsFor system; in | ||
| 76 | - (android.androidenv.composeAndroidPackages { | ||
| 77 | - cmdLineToolsVersion = "13.0"; | ||
| 78 | - buildToolsVersions = [ "34.0.0" ]; | ||
| 79 | - platformVersions = [ "35" "34" ]; | ||
| 80 | - includeNDK = false; | ||
| 81 | - }).androidsdk; | ||
| 82 | - | ||
| 83 | - in | ||
| 84 | - { | ||
| 85 | - packages = forEachSystem (pkgs: | ||
| 86 | - let | ||
| 87 | - inherit (pkgs) lib; | ||
| 88 | - in | ||
| 89 | - { | ||
| 90 | - # The Android SDK `just apk` copies into flutter/.home. A package | ||
| 91 | - # rather than something the recipe evaluates inline, so that | ||
| 92 | - # `nix build .#android-sdk` is how you pre-warm it and `nix flake | ||
| 93 | - # show` admits it exists. | ||
| 94 | - android-sdk = androidSdkFor pkgs.stdenv.hostPlatform.system; | ||
| 95 | - | ||
| 96 | - # There were `appimage` outputs here, and what they were for was a | ||
| 97 | - # host without Nix: they squashed the whole closure into one | ||
| 98 | - # runnable file, Mesa included, and the Mesa was not waste — off | ||
| 99 | - # NixOS the launcher goes through nixGL, which needs a store Mesa to | ||
| 100 | - # put the host's driver in front of. Nothing asks for that shape any | ||
| 101 | - # more, and they were the last thing evaluating nix-appimage, which | ||
| 102 | - # is why that input is gone too. | ||
| 103 | - | ||
| 104 | - # Everything `clojure -M:cljd compile` would otherwise reach the | ||
| 105 | - # network for, fetched once and hashed. | ||
| 106 | - # | ||
| 107 | - # The compile needs three caches, and the reason this is one | ||
| 108 | - # derivation rather than three is that only one of them is obvious. | ||
| 109 | - # Maven and gitlibs are the ordinary tools.deps pair. The third is | ||
| 110 | - # ClojureDart's own: `ensure-cljd-analyzer!` writes a *second*, whole | ||
| 111 | - # pub project to `.clojuredart/cache/<cljd sha>/cljd_helper`, runs | ||
| 112 | - # `pub add analyzer` in it, and then runs `bin/analyzer.dart` out of | ||
| 113 | - # it for the duration of the compile — so a sandbox needs that | ||
| 114 | - # project already resolved, not just the app's dependencies. | ||
| 115 | - # | ||
| 116 | - # Fixed-output, so it is allowed the network the rest of the build is | ||
| 117 | - # not. What that costs is a hash to maintain, and the thing worth | ||
| 118 | - # being exact about is *when*: this derivation never sees frq's | ||
| 119 | - # source. It compiles a three-line throwaway project against the same | ||
| 120 | - # `flutter/deps.edn` and the same `flutter/pubspec.yaml`, so the hash | ||
| 121 | - # moves when a dependency moves and not when a screen changes. A | ||
| 122 | - # stub, rather than `-P` and a hand-built analyzer dir, because | ||
| 123 | - # running the real compiler once is the only way to be sure the | ||
| 124 | - # caches are the ones it actually wants. | ||
| 125 | - # | ||
| 126 | - # PUB_CACHE lands in $out on purpose. The package_config.json inside | ||
| 127 | - # cljd_helper carries absolute paths to whatever resolved it, so | ||
| 128 | - # resolving into a build directory would bake in paths that stop | ||
| 129 | - # existing the moment this derivation finishes. Pointed at $out they | ||
| 130 | - # are store paths, and still true. | ||
| 131 | - cljd-deps = | ||
| 132 | - let | ||
| 133 | - flutterPkg = pkgs.flutter; | ||
| 134 | - in | ||
| 135 | - pkgs.stdenvNoCC.mkDerivation { | ||
| 136 | - name = "frq-cljd-deps"; | ||
| 137 | - dontUnpack = true; | ||
| 138 | - | ||
| 139 | - nativeBuildInputs = [ | ||
| 140 | - pkgs.clojure | ||
| 141 | - pkgs.jdk17 | ||
| 142 | - flutterPkg | ||
| 143 | - pkgs.git | ||
| 144 | - pkgs.cacert | ||
| 145 | - ]; | ||
| 146 | - | ||
| 147 | - buildCommand = '' | ||
| 148 | - export HOME="$NIX_BUILD_TOP/home" | ||
| 149 | - # Resolved in the build directory and copied to $out at the | ||
| 150 | - # end, never written there directly. A fixed-output derivation | ||
| 151 | - # may not reference a store path and its own output is a store | ||
| 152 | - # path, so pub writing its cache's absolute location into its | ||
| 153 | - # own metadata is enough to fail the check. | ||
| 154 | - cache="$NIX_BUILD_TOP/cache" | ||
| 155 | - export PUB_CACHE="$cache/pub-cache" | ||
| 156 | - export GITLIBS="$cache/gitlibs" | ||
| 157 | - export SSL_CERT_FILE="${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt" | ||
| 158 | - mkdir -p "$HOME" "$PUB_CACHE" "$GITLIBS" "$cache/m2" | ||
| 159 | - | ||
| 160 | - # The stub: our dependency files, nothing of our source. | ||
| 161 | - # ../common is a :local/root dependency now, so it has to exist | ||
| 162 | - # *and* carry a deps.edn for tools.deps to resolve — an empty | ||
| 163 | - # directory with that one file in it is enough. | ||
| 164 | - proj="$NIX_BUILD_TOP/stub" | ||
| 165 | - mkdir -p "$proj/src/stub" "$NIX_BUILD_TOP/common" | ||
| 166 | - cp ${./common/deps.edn} "$NIX_BUILD_TOP/common/deps.edn" | ||
| 167 | - cp ${./flutter/deps.edn} "$proj/deps.edn" | ||
| 168 | - cp ${./flutter/pubspec.yaml} "$proj/pubspec.yaml" | ||
| 169 | - # The lock, or `pub get` resolves against pub.dev and takes | ||
| 170 | - # whatever satisfies the ranges today. Every build then fetches | ||
| 171 | - # a slightly different set and the fixed-output hash is a | ||
| 172 | - # promise nothing can keep. | ||
| 173 | - cp ${./flutter/pubspec.lock} "$proj/pubspec.lock" | ||
| 174 | - chmod u+w "$proj/deps.edn" "$proj/pubspec.yaml" "$proj/pubspec.lock" | ||
| 175 | - cat > "$proj/src/stub/main.cljd" <<'EOF' | ||
| 176 | - (ns stub.main) | ||
| 177 | - (defn main [] nil) | ||
| 178 | - EOF | ||
| 179 | - | ||
| 180 | - cd "$proj" | ||
| 181 | - # `:main` has to name the stub, or the compiler goes looking for | ||
| 182 | - # frq.main in a tree that is not here. | ||
| 183 | - sed -i 's/:main frq\.main/:main stub.main/' deps.edn | ||
| 184 | - | ||
| 185 | - flutter config --no-analytics &>/dev/null || true | ||
| 186 | - flutter config --enable-linux-desktop >/dev/null || true | ||
| 187 | - | ||
| 188 | - clojure -Sdeps '{:mvn/local-repo "'"$cache"'/m2"}' -M:cljd compile | ||
| 189 | - | ||
| 190 | - # What the compile left behind, and only that. The analyzer | ||
| 191 | - # project is keyed by the ClojureDart sha, so the directory | ||
| 192 | - # under cache/ is copied wholesale rather than named here. | ||
| 193 | - mkdir -p "$out/clojuredart" | ||
| 194 | - cp -r .clojuredart/cache "$out/clojuredart/cache" | ||
| 195 | - cp -r "$cache/m2" "$out/m2" | ||
| 196 | - cp -r "$cache/gitlibs" "$out/gitlibs" | ||
| 197 | - cp -r "$PUB_CACHE" "$out/pub-cache" | ||
| 198 | - | ||
| 199 | - # A fixed-output derivation may not reference a store path, and | ||
| 200 | - # a resolved pub project is nothing but store paths: | ||
| 201 | - # package_config.json names the Flutter SDK and every package | ||
| 202 | - # in the cache by absolute path. So the analyzer project ships | ||
| 203 | - # *unresolved* — its pubspec and its analyzer.dart and nothing | ||
| 204 | - # else — and `flutter pub get --offline` re-resolves it against | ||
| 205 | - # this cache at build time, where naming the store is allowed. | ||
| 206 | - find "$out" \( -name '.dart_tool' -o -name '.flutter-plugins' \ | ||
| 207 | - -o -name '.flutter-plugins-dependencies' \) -prune -exec rm -rf {} + | ||
| 208 | - find "$out" -name '.packages' -delete | ||
| 209 | - | ||
| 210 | - | ||
| 211 | - # A fixed-output hash is a promise that two runs agree, so | ||
| 212 | - # everything a tool writes *about* a run rather than about a | ||
| 213 | - # dependency has to go: pub's log carries timestamps, Maven | ||
| 214 | - # rewrites its resolution metadata on every resolve, and | ||
| 215 | - # tools.gitlibs keeps bare clones it only needs in order to | ||
| 216 | - # make a checkout. None of it is read offline. | ||
| 217 | - # | ||
| 218 | - # active_roots is the one that was actually breaking this. Pub | ||
| 219 | - # records the project directories using the cache, sharded by | ||
| 220 | - # a hash of the path, and $NIX_BUILD_TOP is different on every | ||
| 221 | - # run -- so two builds whose hosted/ trees were byte-identical | ||
| 222 | - # still disagreed, purely over which directory had asked. Four | ||
| 223 | - # builds gave four hashes until this went. | ||
| 224 | - rm -rf "$out/pub-cache/log" "$out/pub-cache/_temp" \ | ||
| 225 | - "$out/pub-cache/git" "$out/pub-cache/global_packages" \ | ||
| 226 | - "$out/pub-cache/bin" "$out/pub-cache/active_roots" | ||
| 227 | - | ||
| 228 | - # tools.gitlibs keeps a bare clone per URL under _repos/, and a | ||
| 229 | - # bare clone is packfiles — which two runs of the same fetch do | ||
| 230 | - # not have to produce byte for byte. It cannot simply be | ||
| 231 | - # deleted, because `procure` calls `ensure-git-dir` before it | ||
| 232 | - # looks at anything else and would clone it again, over a | ||
| 233 | - # network this has and the build that uses it does not. | ||
| 234 | - # | ||
| 235 | - # It does not need the objects, though. `procure` finds the sha | ||
| 236 | - # with `match-exact` against the checkout already in libs/, so | ||
| 237 | - # the bare repo only has to exist. Emptied and re-initialised, | ||
| 238 | - # it is a fixed handful of files from the pinned git and the | ||
| 239 | - # same on every run. | ||
| 240 | - find "$out/gitlibs/_repos" -name HEAD | while read -r head; do | ||
| 241 | - repo="$(dirname "$head")" | ||
| 242 | - rm -rf "$repo" | ||
| 243 | - git init --bare -q "$repo" | ||
| 244 | - # The sample hooks are shell scripts, so they carry a | ||
| 245 | - # `#!/nix/store/.../bash` line — which is exactly the kind of | ||
| 246 | - # store reference a fixed-output derivation may not hold. An | ||
| 247 | - # empty bare repo nothing ever runs has no use for them. | ||
| 248 | - rm -rf "$repo/hooks" | ||
| 249 | - done | ||
| 250 | - # pub's version listings, which record when they were fetched. | ||
| 251 | - # This is the one that actually moved between two runs of this | ||
| 252 | - # derivation: the package sources under hosted/ were identical | ||
| 253 | - # and the listings beside them were not. Nothing offline reads | ||
| 254 | - # them — a resolution that already has every package on disk | ||
| 255 | - # never asks pub.dev what versions exist. | ||
| 256 | - find "$out/pub-cache" -name '.cache' -type d -prune -exec rm -rf {} + | ||
| 257 | - find "$out/m2" \( -name '*.lastUpdated' -o -name '_remote.repositories' \ | ||
| 258 | - -o -name 'resolver-status.properties' -o -name '*.part' \ | ||
| 259 | - -o -name 'maven-metadata-*.xml*' \) -delete | ||
| 260 | - find "$out" \( -name '.DS_Store' -o -name '*.log' -o -name '.git' \) \ | ||
| 261 | - -prune -exec rm -rf {} + | ||
| 262 | - find "$out" -type d -empty -delete | ||
| 263 | - chmod -R u+w "$out" | ||
| 264 | - | ||
| 265 | - # Last, after every cleanup above: anything still naming the | ||
| 266 | - # store fails the fixed-output check, and the error names one | ||
| 267 | - # path out of thousands of files. This names the files. | ||
| 268 | - if refs="$(grep -rlI /nix/store "$out" 2>/dev/null)" && [ -n "$refs" ]; then | ||
| 269 | - echo "cljd-deps: these still reference the store:" >&2 | ||
| 270 | - echo "$refs" | head -20 >&2 | ||
| 271 | - fi | ||
| 272 | - | ||
| 273 | - # If two runs disagree, this says which half to look in. Cheap, | ||
| 274 | - # and the alternative is a hash mismatch with nothing attached. | ||
| 275 | - for d in "$out"/*; do | ||
| 276 | - echo "cljd-deps subtree $(basename "$d") $( (cd "$d" && find . -type f \ | ||
| 277 | - -exec sha256sum {} + | sort -k2 | sha256sum) )" >&2 | ||
| 278 | - done | ||
| 279 | - for d in "$out"/pub-cache/*/*; do | ||
| 280 | - [ -d "$d" ] || continue | ||
| 281 | - # The path relative to $out, not the basename: two runs that | ||
| 282 | - # disagree here disagree about *which* directory exists, and | ||
| 283 | - # a bare `09` names nothing you can go and look at. | ||
| 284 | - echo "cljd-deps pub ''${d#$out/} $( (cd "$d" && find . -type f \ | ||
| 285 | - -exec sha256sum {} + | sort -k2 | sha256sum) )" >&2 | ||
| 286 | - done | ||
| 287 | - ''; | ||
| 288 | - | ||
| 289 | - outputHashMode = "recursive"; | ||
| 290 | - outputHashAlgo = "sha256"; | ||
| 291 | - # Moves when flutter/deps.edn or flutter/pubspec.yaml move, and | ||
| 292 | - # not when frq's own source does — see the stub above. | ||
| 293 | - outputHash = "sha256-qSGx7WFdVyV7yu4R+EjiQHZcLqwDjYSlohiZcXB43DY="; | ||
| 294 | - }; | ||
| 295 | - | ||
| 296 | - # The Flutter desktop GUI, built rather than run out of the tree. | ||
| 297 | - # | ||
| 298 | - # `just flutter-desktop` is the working-tree loop and this is its | ||
| 299 | - # opposite number: the source is the flake's, the output is a store | ||
| 300 | - # path, and the build is a sandbox with no network. It is the first thing here that builds | ||
| 301 | - # purely — the APK cannot, because Gradle fetches as it goes. | ||
| 302 | - # | ||
| 303 | - # Two stages, because the Dart does not exist until ClojureDart writes | ||
| 304 | - # it. `preBuild` runs the compiler over `flutter/src` and `common/` | ||
| 305 | - # with `--offline`, out of the caches `cljd-deps` fetched; everything | ||
| 306 | - # after that is an ordinary Flutter application as far as nixpkgs is | ||
| 307 | - # concerned. | ||
| 308 | - # | ||
| 309 | - # The caches are copied in rather than used where they lie. Maven, | ||
| 310 | - # tools.gitlibs and pub all expect to be able to write to their own | ||
| 311 | - # cache — a lock file, a resolved marker — and the store is read-only, | ||
| 312 | - # so pointing them at $out of a fixed-output derivation fails in three | ||
| 313 | - # different ways at three different depths. | ||
| 314 | - # | ||
| 315 | - # `src` is the whole tree and not `flutter/`: `flutter/deps.edn` puts | ||
| 316 | - # `../common` on the classpath, which is the entire point of that | ||
| 317 | - # directory, and a source root of `flutter/` would leave the screens | ||
| 318 | - # outside it. | ||
| 319 | - flutter-desktop-unwrapped = pkgs.flutter.buildFlutterApplication rec { | ||
| 320 | - pname = "frq-flutter"; | ||
| 321 | - version = "0.1.0"; | ||
| 322 | - | ||
| 323 | - src = lib.cleanSourceWith { | ||
| 324 | - src = ./.; | ||
| 325 | - # Two trees, and only two: `flutter/` is the app and `common/` | ||
| 326 | - # is the screens its deps.edn puts on the classpath. The root is | ||
| 327 | - # still the source root because of that `../common`, but letting | ||
| 328 | - # the *whole* root in means every file in the repo is an input — | ||
| 329 | - # so editing flake.nix or CLAUDE.md | ||
| 330 | - # invalidated the entire Dart compile and paid ten minutes for a | ||
| 331 | - # change the Flutter build cannot even see. | ||
| 332 | - # | ||
| 333 | - # Matched on the path relative to the root rather than on | ||
| 334 | - # basename: `src` as a basename would also exclude `flutter/src` | ||
| 335 | - # and `common/src`, which is everything that matters. | ||
| 336 | - filter = | ||
| 337 | - let root = toString ./.; in | ||
| 338 | - path: type: | ||
| 339 | - let | ||
| 340 | - rel = lib.removePrefix (root + "/") (toString path); | ||
| 341 | - inTree = d: rel == d || lib.hasPrefix (d + "/") rel; | ||
| 342 | - in | ||
| 343 | - (inTree "flutter" || inTree "common") | ||
| 344 | - # Build trees and caches, which are large, machine-specific | ||
| 345 | - # and would make every one of them a new store path. | ||
| 346 | - && !(builtins.elem (baseNameOf path) [ | ||
| 347 | - "build" ".home" ".clojuredart" ".cpcache" "cljd-out" | ||
| 348 | - ".dart_tool" "result" ".git" "buck-out" | ||
| 349 | - ]); | ||
| 350 | - }; | ||
| 351 | - sourceRoot = "source/flutter"; | ||
| 352 | - | ||
| 353 | - # Read at eval time, so the lock in git is the lock that is built. | ||
| 354 | - autoPubspecLock = ./flutter/pubspec.lock; | ||
| 355 | - | ||
| 356 | - # git, because tools.deps resolves the ClojureDart dependency through | ||
| 357 | - # tools.gitlibs even when every byte of it is already on disk — see | ||
| 358 | - # the _repos note in cljd-deps. | ||
| 359 | - nativeBuildInputs = [ pkgs.clojure pkgs.jdk17 pkgs.git ]; | ||
| 360 | - | ||
| 361 | - preBuild = '' | ||
| 362 | - export PUB_CACHE="$NIX_BUILD_TOP/pub-cache" | ||
| 363 | - export GITLIBS="$NIX_BUILD_TOP/gitlibs" | ||
| 364 | - cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/pub-cache "$PUB_CACHE" | ||
| 365 | - cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/gitlibs "$GITLIBS" | ||
| 366 | - cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/m2 "$NIX_BUILD_TOP/m2" | ||
| 367 | - mkdir -p .clojuredart | ||
| 368 | - cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/clojuredart/cache .clojuredart/cache | ||
| 369 | - chmod -R u+w "$PUB_CACHE" "$GITLIBS" "$NIX_BUILD_TOP/m2" .clojuredart | ||
| 370 | - | ||
| 371 | - # Resolve the analyzer project here rather than in cljd-deps, | ||
| 372 | - # which was not allowed to name the store. Offline, out of the | ||
| 373 | - # cache that derivation did fetch. ClojureDart only reaches for | ||
| 374 | - # the network when `bin/analyzer.dart` is missing, and it is not. | ||
| 375 | - for helper in .clojuredart/cache/*/cljd_helper; do | ||
| 376 | - ( cd "$helper" && flutter pub get --offline ) | ||
| 377 | - done | ||
| 378 | - | ||
| 379 | - # --offline is what keeps `pub get` out of a sandbox that has no | ||
| 380 | - # network; the analyzer project it would otherwise resolve is | ||
| 381 | - # already in .clojuredart, put there by cljd-deps. | ||
| 382 | - clojure -Sdeps "{:mvn/local-repo \"$NIX_BUILD_TOP/m2\"}" \ | ||
| 383 | - -M:cljd compile --offline | ||
| 384 | - ''; | ||
| 385 | - | ||
| 386 | - meta = { | ||
| 387 | - description = "frq's screens on Flutter's Linux target (no GL launcher)"; | ||
| 388 | - mainProgram = "frq"; | ||
| 389 | - platforms = systems; | ||
| 390 | - }; | ||
| 391 | - }; | ||
| 392 | - | ||
| 393 | - # The same shape as `frq` above: a launcher, and a package that is a | ||
| 394 | - # symlink to it. The reason is the same one `frqScript` gives — on | ||
| 395 | - # NixOS the store's Mesa is the system's and the window opens, and | ||
| 396 | - # anywhere else the real driver is the host's, so the process is | ||
| 397 | - # handed to nixGL. Without it the store build dies on a distrobox | ||
| 398 | - # Arch with "No provider of eglGetPlatformDisplayEXT found", which is | ||
| 399 | - # that failure wearing an EGL hat. | ||
| 400 | - # | ||
| 401 | - # A wrapper *around* the built application rather than a `postFixup` | ||
| 402 | - # inside it, because buildFlutterApplication's own dartFixupHook runs | ||
| 403 | - # after postFixup and rewrites `bin/frq` — so anything done to that | ||
| 404 | - # path from inside is undone on the way out. | ||
| 405 | - flutter-desktop = | ||
| 406 | - let | ||
| 407 | - unwrapped = | ||
| 408 | - self.packages.${pkgs.stdenv.hostPlatform.system}.flutter-desktop-unwrapped; | ||
| 409 | - | ||
| 410 | - # buildFlutterApplication's own wrapper appends a bare `/lib` to | ||
| 411 | - # LD_LIBRARY_PATH — the host's, not the bundle's. Off NixOS that | ||
| 412 | - # is a foreign library directory in front of nothing, and the app | ||
| 413 | - # dies in the loader before main: first | ||
| 414 | - # | ||
| 415 | - # /lib/libc.so.6: undefined symbol: __pointer_chk_guard | ||
| 416 | - # | ||
| 417 | - # and, once the store's glibc is put ahead of it, | ||
| 418 | - # | ||
| 419 | - # libc.so.6: version `GLIBC_2.43' not found | ||
| 420 | - # (required by /lib/libglib-2.0.so.0) | ||
| 421 | - # | ||
| 422 | - # which is the same bug wearing the other hat: the host's glib | ||
| 423 | - # against the store's glibc. Ordering cannot fix a mixture, so | ||
| 424 | - # the entry goes rather than moves. The wrapper is generated, so | ||
| 425 | - # this edits a copy and asserts the edit landed — a silent miss | ||
| 426 | - # here is a runtime failure on someone else's machine. | ||
| 427 | - fixed = pkgs.runCommand "frq-flutter-wrapper" { } '' | ||
| 428 | - mkdir -p "$out/bin" | ||
| 429 | - sed "s|'/lib'||g" ${unwrapped}/bin/frq > "$out/bin/frq" | ||
| 430 | - chmod +x "$out/bin/frq" | ||
| 431 | - if grep -q "'/lib'" "$out/bin/frq"; then | ||
| 432 | - echo "the /lib entry outlived the edit; look at the wrapper" >&2 | ||
| 433 | - exit 1 | ||
| 434 | - fi | ||
| 435 | - ''; | ||
| 436 | - | ||
| 437 | - script = pkgs.writeShellScript "frq" '' | ||
| 438 | - runner="" | ||
| 439 | - [ -e /run/current-system ] || runner="${nixGLFor pkgs}/bin/nixGLIntel" | ||
| 440 | - exec ''${runner} ${fixed}/bin/frq "$@" | ||
| 441 | - ''; | ||
| 442 | - in | ||
| 443 | - pkgs.runCommand "frq-flutter-0.1.0" | ||
| 444 | - { | ||
| 445 | - meta = { | ||
| 446 | - description = "frq's screens on Flutter's Linux target"; | ||
| 447 | - mainProgram = "frq"; | ||
| 448 | - platforms = systems; | ||
| 449 | - }; | ||
| 450 | - } | ||
| 451 | - '' | ||
| 452 | - mkdir -p "$out/bin" | ||
| 453 | - ln -s ${script} "$out/bin/frq" | ||
| 454 | - ''; | ||
| 455 | - }); | ||
| 456 | - | ||
| 457 | - devShells = forEachSystem (pkgs: | ||
| 458 | - let | ||
| 459 | - inherit (pkgs) lib; | ||
| 460 | - in | ||
| 461 | - { | ||
| 462 | - # Dart on its own, for the tests that need no Flutter: the FFI | ||
| 463 | - # binding to the Nim core runs on the plain VM, and making it wait | ||
| 464 | - # for a Flutter toolchain would throw away the reason it is fast. | ||
| 465 | - # | ||
| 466 | - # Flutter bundles a Dart, so this is a duplicate in one sense. It is | ||
| 467 | - # also thirty times smaller, and the point of the boundary is that | ||
| 468 | - # you can check it without the thing on the other side of it. | ||
| 469 | - dart = pkgs.mkShellNoCC { | ||
| 470 | - name = "frq-dart"; | ||
| 471 | - packages = [ pkgs.dart pkgs.just ]; | ||
| 472 | - | ||
| 473 | - # libfrqcore.so is linked against OpenSSL, and the process that | ||
| 474 | - # dlopens it has to be able to find one. Named here rather than | ||
| 475 | - # left to the host: a machine whose libssl is a different soname | ||
| 476 | - # fails at `frq_init` with a message about the wrong library. | ||
| 477 | - LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.openssl ]; | ||
| 478 | - | ||
| 479 | - FRQ_DART = "1"; | ||
| 480 | - }; | ||
| 481 | - | ||
| 482 | - # Nim, for `nim/` — the portable core as a native library. Just the | ||
| 483 | - # compiler: the core has no dependencies outside Nim's own standard | ||
| 484 | - # library, deliberately, because a dependency here is one that has | ||
| 485 | - # to cross-compile to every target the Dart side runs on. | ||
| 486 | - # | ||
| 487 | - # `nim c` shells out to a C compiler, so this is mkShell and not | ||
| 488 | - # mkShellNoCC: stdenv brings the one Nim will find. | ||
| 489 | - nim = pkgs.mkShell { | ||
| 490 | - name = "frq-nim"; | ||
| 491 | - packages = [ pkgs.nim pkgs.just ]; | ||
| 492 | - | ||
| 493 | - # OpenSSL, because `-d:ssl` in nim/nim.cfg makes std/net link | ||
| 494 | - # -lssl and -lcrypto: the IRC connection is TLS on :6697, which is | ||
| 495 | - # the only port freeq actually listens on. | ||
| 496 | - buildInputs = [ pkgs.openssl ]; | ||
| 497 | - | ||
| 498 | - # And on the loader path as well as the linker's. Nim resolves the | ||
| 499 | - # OpenSSL entry points through dynlib at run time, so without this | ||
| 500 | - # `newContext` finds nothing behind the symbol and dies with a | ||
| 501 | - # SIGSEGV that says nothing about SSL at all. | ||
| 502 | - LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.openssl ]; | ||
| 503 | - | ||
| 504 | - # The recipe's re-entry test, the way FRQ_FLUTTER_DESKTOP is the | ||
| 505 | - # desktop one's. | ||
| 506 | - FRQ_NIM = "1"; | ||
| 507 | - }; | ||
| 508 | - | ||
| 509 | - # The APK toolchain. It is not in a default shell any more because | ||
| 510 | - # there is no default shell: what used to be one belonged to the | ||
| 511 | - # libcosmic frontend, and a Flutter build asks for a toolchain by | ||
| 512 | - # name. | ||
| 513 | - # | ||
| 514 | - # Flutter brings its own Dart, Gradle and a JDK's worth of | ||
| 515 | - # closure: Flutter brings its own Dart, Gradle and a JDK's worth of | ||
| 516 | - # closure, and a desktop build has no use for any of it. | ||
| 517 | - # | ||
| 518 | - # `just apk` used to name these as `nix shell nixpkgs#clojure | ||
| 519 | - # nixpkgs#jdk17 nixpkgs#flutter`, which is the flake registry's | ||
| 520 | - # nixpkgs and not this flake's — so the Flutter under the APK | ||
| 521 | - # floated while everything else was locked. Same three packages, | ||
| 522 | - # from flake.lock now. | ||
| 523 | - # | ||
| 524 | - # JDK 17 and not newer on purpose: the Flutter template's Gradle | ||
| 525 | - # plugin pins a Gradle that rejects a JDK it was released before, | ||
| 526 | - # and the failure reads as an unsupported class file version rather | ||
| 527 | - # than as a version mismatch. | ||
| 528 | - flutter = pkgs.mkShellNoCC { | ||
| 529 | - name = "frq-flutter"; | ||
| 530 | - | ||
| 531 | - # git, because tools.deps resolves the ClojureDart dependency | ||
| 532 | - # through tools.gitlibs even when every byte of it is already in | ||
| 533 | - # the seeded cache — the same reason flutter-desktop-unwrapped | ||
| 534 | - # names it. The host's git has always been there to answer; naming | ||
| 535 | - # it means the shell does not depend on that. | ||
| 536 | - packages = [ pkgs.clojure pkgs.jdk17 pkgs.flutter pkgs.just pkgs.git ]; | ||
| 537 | - | ||
| 538 | - # Where the recipe copies from. Naming it here is also what makes | ||
| 539 | - # entering the shell build it, so the first `just apk` does not | ||
| 540 | - # stop for a few hundred megabytes of SDK with nothing said about | ||
| 541 | - # why. | ||
| 542 | - FRQ_ANDROID_SDK = | ||
| 543 | - "${androidSdkFor pkgs.stdenv.hostPlatform.system}/libexec/android-sdk"; | ||
| 544 | - | ||
| 545 | - # The Maven, gitlibs and pub caches the ClojureDart compile would | ||
| 546 | - # otherwise fetch, plus the analyzer project it writes under | ||
| 547 | - # .clojuredart. Here for both of FRQ_ANDROID_SDK's reasons: it is | ||
| 548 | - # where the recipe copies from, and naming it is what makes | ||
| 549 | - # entering the shell build it. | ||
| 550 | - # | ||
| 551 | - # The compile still runs online. These are a warm start and not a | ||
| 552 | - # pin — `--offline` would be, and would turn adding a line to | ||
| 553 | - # flutter/deps.edn into a re-hash of cljd-deps before anything | ||
| 554 | - # compiled again. The sandbox build takes that trade because it | ||
| 555 | - # has no network; the loop someone edits in should not. | ||
| 556 | - FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}"; | ||
| 557 | - }; | ||
| 558 | - | ||
| 559 | - # The other desktop GUI. Same ClojureDart half as the APK — one | ||
| 560 | - # `clojure -M:cljd compile`, one flutter/src — over Flutter's Linux | ||
| 561 | - # target instead of its Android one, so `flutter/linux/` is the | ||
| 562 | - # runner and CMake and Ninja are the build rather than Gradle. | ||
| 563 | - # | ||
| 564 | - # A separate shell from `flutter` rather than one that carries both, | ||
| 565 | - # because the halves are disjoint: this wants GTK and a C++ toolchain | ||
| 566 | - # and no JDK, and the APK wants a JDK and an SDK and no GTK. Sharing | ||
| 567 | - # them would mean every desktop build paying for a few hundred | ||
| 568 | - # megabytes of Android SDK it never opens, which is the same argument | ||
| 569 | - # that keeps Flutter out of the default shell. | ||
| 570 | - # | ||
| 571 | - # mkShell and not mkShellNoCC, unlike every other shell here: this is | ||
| 572 | - # the one that actually compiles C++. stdenv brings the compiler, and | ||
| 573 | - # gtk3 in buildInputs is what puts its .pc file where the runner's | ||
| 574 | - # `pkg_check_modules(GTK gtk+-3.0)` can find it. | ||
| 575 | - flutter-desktop = pkgs.mkShell { | ||
| 576 | - name = "frq-flutter-desktop"; | ||
| 577 | - | ||
| 578 | - # clojure and flutter are the APK shell's, and deliberately the | ||
| 579 | - # same two: the Dart that runs here is generated by the same | ||
| 580 | - # compiler from the same source, and a second Flutter version | ||
| 581 | - # under it would be a second set of engine artifacts and a second | ||
| 582 | - # answer to "does the phone build match the desktop one". | ||
| 583 | - nativeBuildInputs = [ | ||
| 584 | - pkgs.clojure | ||
| 585 | - pkgs.flutter | ||
| 586 | - pkgs.just | ||
| 587 | - pkgs.cmake | ||
| 588 | - pkgs.ninja | ||
| 589 | - pkgs.pkg-config | ||
| 590 | - ]; | ||
| 591 | - | ||
| 592 | - # gtk3 is the runner's own dependency; the rest are url_launcher's | ||
| 593 | - # Linux implementation, which is a GTK plugin compiled into the | ||
| 594 | - # bundle. path_provider needs nothing here — its Linux half is | ||
| 595 | - # pure Dart over the XDG directories. | ||
| 596 | - buildInputs = [ pkgs.gtk3 pkgs.glib ]; | ||
| 597 | - | ||
| 598 | - # Where libfrqcore.so's OpenSSL lives. A named variable and NOT | ||
| 599 | - # LD_LIBRARY_PATH, deliberately: this shell also runs Flutter | ||
| 600 | - # through nixGL, which does its own careful things to the loader | ||
| 601 | - # path, and a blanket LD_LIBRARY_PATH here is the sort of thing | ||
| 602 | - # that breaks GL on one machine and not another. The `nim-spike` | ||
| 603 | - # recipe prepends this for the app it launches and nothing else. | ||
| 604 | - FRQ_OPENSSL_LIB = lib.makeLibraryPath [ pkgs.openssl ]; | ||
| 605 | - | ||
| 606 | - # Flutter paints through GL, and off NixOS the driver that can do | ||
| 607 | - # that is the host's, not the store's. | ||
| 608 | - NIXGL = "${nixGLFor pkgs}/bin/nixGLIntel"; | ||
| 609 | - | ||
| 610 | - # The `flutter` shell's, deliberately the same one and for the | ||
| 611 | - # same reason clojure and flutter are: the ClojureDart half of | ||
| 612 | - # both builds is one compile over one deps.edn, so a second set of | ||
| 613 | - # caches would be a second answer to what it resolved against. | ||
| 614 | - # This recipe reads the variable directly — it is inside this | ||
| 615 | - # shell before it does any of the work — where `just apk` reaches | ||
| 616 | - # for the flake output itself. | ||
| 617 | - FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}"; | ||
| 618 | - | ||
| 619 | - # The recipe's re-entry test. Nothing else sets it, so `just flutter-desktop` | ||
| 620 | - # outside the shell re-enters and lands back on the same recipe — | ||
| 621 | - # no flag to forget, and no second code path for someone who runs | ||
| 622 | - # `nix develop .#flutter-desktop --command just flutter-desktop` | ||
| 623 | - # by hand. | ||
| 624 | - FRQ_FLUTTER_DESKTOP = "1"; | ||
| 625 | - }; | ||
| 626 | - | ||
| 627 | - # No `flutter-web` shell here any more. The web target was the one | ||
| 628 | - # that needed nothing of the host -- no JDK and no Android SDK as | ||
| 629 | - # the APK wants, no GTK and no C++ and no nixGL as the desktop one | ||
| 630 | - # does -- and a devShell whose only job is to hand over a Dart and | ||
| 631 | - # a JVM is a devShell that a pinned tarball can replace. It did: | ||
| 632 | - # `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by | ||
| 633 | - # sha256, `tools/build-web.sh` builds out of them, and | ||
| 634 | - # `.modal/flutter-web/` runs that same script on a plain Debian | ||
| 635 | - # image with no store to populate. | ||
| 636 | - # | ||
| 637 | - # The two shells above stay. What they supply is a host toolchain, | ||
| 638 | - # which is exactly what nix is better at than a tarball. | ||
| 639 | - }); | ||
| 640 | - | ||
| 641 | - }; | ||
| 642 | -} | ||
modified
flutter/README.md +24 -40 | @@ -76,39 +76,36 @@ all of them — `clojure -M:cljd compile` over `src/` and `../common` — and wh | ||
| 76 | 76 | differs is only what Flutter is asked to wrap it in. |
| 77 | 77 | |
| 78 | 78 | ```bash |
| 79 | -just apk # the debug APK | |
| 80 | -just apk install # and onto a connected device | |
| 81 | -just apk run # and launched | |
| 82 | -just apk log # logcat | |
| 79 | +just build apk # the debug APK | |
| 80 | +just run apk # onto a connected device, and launched | |
| 81 | +just run log # logcat | |
| 83 | 82 | |
| 84 | -just flutter-desktop # the debug Linux bundle | |
| 85 | -just flutter-desktop run # and the window | |
| 83 | +just build desktop # the debug Linux bundle | |
| 84 | +just run desktop # and the window | |
| 86 | 85 | |
| 87 | -just flutter-web # the web bundle | |
| 88 | -just flutter-web serve # and served on :8080 | |
| 86 | +just build web # the web bundle | |
| 87 | +just run web # and served on :8080 | |
| 89 | 88 | ``` |
| 90 | 89 | |
| 91 | 90 | ### The desktop one |
| 92 | 91 | |
| 93 | -`just flutter-desktop` is this tree under Flutter's Linux target — the same | |
| 92 | +`just build desktop` is this tree under Flutter's Linux target — the same | |
| 94 | 93 | screens out of `common/frq/screens/` as the APK, with `frq.hiccup` emitting |
| 95 | 94 | Flutter widgets. There used to be a second desktop GUI beside it, libcosmic |
| 96 | 95 | under jolt, walking the same hiccup through a different renderer; it is gone. |
| 97 | 96 | |
| 98 | -Its toolchain is `devShells.flutter-desktop`, which is the APK shell with the | |
| 99 | -Android half swapped out: clojure and Flutter are the same two packages at the | |
| 100 | -same pinned rev, and CMake, Ninja, pkg-config and GTK stand where the JDK and | |
| 97 | +Its toolchain is the same `.toolchain/` every other target uses — clojure and | |
| 98 | +Flutter, at the same pins — with CMake, Ninja, pkg-config and GTK coming from | |
| 99 | +the host where the JDK and | |
| 101 | 100 | the SDK do. Kept separate rather than merged into one shell because the halves |
| 102 | 101 | are disjoint — a desktop build has no use for a few hundred megabytes of |
| 103 | 102 | Android SDK. |
| 104 | 103 | |
| 105 | 104 | Still impure, for one of the two reasons the APK is: pub.dev resolution and |
| 106 | -Flutter's engine artifacts are network. What it does *not* need is the | |
| 107 | -writable-`ANDROID_HOME` dance, since nothing here writes into the store — so | |
| 108 | -there is no `flutter/.home` on this path. | |
| 105 | +Flutter's engine artifacts are network. What it does *not* need is the Android | |
| 106 | +SDK, so `just build desktop` never asks `tools/toolchain.sh` for one. | |
| 109 | 107 | |
| 110 | -nixGL off NixOS: Flutter paints through GL and the driver that can do that is | |
| 111 | -the host's. | |
| 108 | +GL is the host's, as is the driver that can do it. | |
| 112 | 109 | |
| 113 | 110 | `linux/` is the Flutter template's GTK runner, renamed — `frq` rather than |
| 114 | 111 | `cljd_flutter`, and `uk.nandi.frq` rather than `com.example.cljd_flutter`, so |
| @@ -132,28 +129,15 @@ Two things the desktop target changed in the Dart, both of them cases where | ||
| 132 | 129 | ### The APK |
| 133 | 130 | |
| 134 | 131 | Impure on purpose. Gradle resolves its own dependencies over the network and |
| 135 | -installs build-tools and a platform into `ANDROID_HOME` as it goes, so it can | |
| 136 | -neither run in a sandbox nor write to the store. What nix gives is the | |
| 137 | -toolchain — clojure, a JDK, Flutter, and an SDK composed by androidenv — and | |
| 138 | -the recipe copies that SDK to `flutter/.home` for Gradle to finish off. That | |
| 139 | -copy and everything Gradle leaves behind are gitignored. | |
| 140 | - | |
| 141 | -All of it is the flake's, which it did not used to be. The toolchain was | |
| 142 | -`nix shell nixpkgs#clojure nixpkgs#jdk17 nixpkgs#flutter` and the SDK was a | |
| 143 | -`nix build --impure --expr` around `builtins.getFlake | |
| 144 | -"github:NixOS/nixpkgs/nixos-unstable"` — two references to an *unlocked* | |
| 145 | -nixpkgs, so the Flutter that compiled the APK and the nixpkgs under everything | |
| 146 | -else could drift apart without flake.lock changing a line. They are | |
| 147 | -`devShells.<system>.flutter` and `packages.<system>.android-sdk` now, at the | |
| 148 | -pinned rev, and the recipe is `nix develop .#flutter --command` over | |
| 149 | -`nix build .#android-sdk`. | |
| 150 | - | |
| 151 | -The SDK needs `allowUnfree` and `android_sdk.accept_license`, which cannot be | |
| 152 | -set on a `legacyPackages` attribute after the fact — hence `androidPkgsFor` in | |
| 153 | -the flake, a second `import` of the same locked input rather than a second | |
| 154 | -nixpkgs. The Flutter toolchain is kept out of the default dev shell: it brings | |
| 155 | -its own Dart and a JDK's worth of closure, and a desktop build wants none of | |
| 156 | -it. | |
| 132 | +has sdkmanager install build-tools and a platform into `ANDROID_HOME` as it | |
| 133 | +goes, so it cannot run in a sandbox and the SDK it writes to has to be ours. | |
| 134 | +`just tools android` fetches Google's command-line tools by pinned sha256 into | |
| 135 | +`.toolchain/android-sdk` and accepts the licences; Gradle finishes the job from | |
| 136 | +there. All of it is gitignored. | |
| 137 | + | |
| 138 | +The platform and build-tools versions are deliberately not pinned here: they | |
| 139 | +come from whatever Flutter asks Gradle for, and pinning them in a second place | |
| 140 | +is how the two drift apart. | |
| 157 | 141 | |
| 158 | 142 | Two things the Flutter template wanted that are deliberately not here. There is |
| 159 | 143 | no `ndkVersion` in `android/app/build.gradle.kts`: setting it makes Gradle |
| @@ -249,7 +233,7 @@ move from. What remains is work the port never covered: | ||
| 249 | 233 | listen on localhost for a browser it does not own. That wants an app link or |
| 250 | 234 | a custom scheme, an intent filter, and a redirect URI the broker will accept |
| 251 | 235 | — a decision about freeq's broker, not a porting problem. The web build has |
| 252 | - its own answer in `frq.oauth.web`, and `just web-local` is why it only | |
| 236 | + its own answer in `frq.oauth.web`, and `just serve` is why it only | |
| 253 | 237 | completes on localhost. |
| 254 | 238 | 2. **Calls.** The signaling is IRC and is still in the screens; the media plane |
| 255 | 239 | it drove was `libjoltmoq` — Opus, H.264, V4L2, ALSA, MoQ over QUIC — under |
| @@ -76,39 +76,36 @@ all of them — `clojure -M:cljd compile` over `src/` and `../common` — and wh | |||
| 76 | differs is only what Flutter is asked to wrap it in. | 76 | differs is only what Flutter is asked to wrap it in. |
| 77 | 77 | ||
| 78 | ```bash | 78 | ```bash |
| 79 | -just apk # the debug APK | 79 | +just build apk # the debug APK |
| 80 | -just apk install # and onto a connected device | 80 | +just run apk # onto a connected device, and launched |
| 81 | -just apk run # and launched | 81 | +just run log # logcat |
| 82 | -just apk log # logcat | ||
| 83 | 82 | ||
| 84 | -just flutter-desktop # the debug Linux bundle | 83 | +just build desktop # the debug Linux bundle |
| 85 | -just flutter-desktop run # and the window | 84 | +just run desktop # and the window |
| 86 | 85 | ||
| 87 | -just flutter-web # the web bundle | 86 | +just build web # the web bundle |
| 88 | -just flutter-web serve # and served on :8080 | 87 | +just run web # and served on :8080 |
| 89 | ``` | 88 | ``` |
| 90 | 89 | ||
| 91 | ### The desktop one | 90 | ### The desktop one |
| 92 | 91 | ||
| 93 | -`just flutter-desktop` is this tree under Flutter's Linux target — the same | 92 | +`just build desktop` is this tree under Flutter's Linux target — the same |
| 94 | screens out of `common/frq/screens/` as the APK, with `frq.hiccup` emitting | 93 | screens out of `common/frq/screens/` as the APK, with `frq.hiccup` emitting |
| 95 | Flutter widgets. There used to be a second desktop GUI beside it, libcosmic | 94 | Flutter widgets. There used to be a second desktop GUI beside it, libcosmic |
| 96 | under jolt, walking the same hiccup through a different renderer; it is gone. | 95 | under jolt, walking the same hiccup through a different renderer; it is gone. |
| 97 | 96 | ||
| 98 | -Its toolchain is `devShells.flutter-desktop`, which is the APK shell with the | 97 | +Its toolchain is the same `.toolchain/` every other target uses — clojure and |
| 99 | -Android half swapped out: clojure and Flutter are the same two packages at the | 98 | +Flutter, at the same pins — with CMake, Ninja, pkg-config and GTK coming from |
| 100 | -same pinned rev, and CMake, Ninja, pkg-config and GTK stand where the JDK and | 99 | +the host where the JDK and |
| 101 | the SDK do. Kept separate rather than merged into one shell because the halves | 100 | the SDK do. Kept separate rather than merged into one shell because the halves |
| 102 | are disjoint — a desktop build has no use for a few hundred megabytes of | 101 | are disjoint — a desktop build has no use for a few hundred megabytes of |
| 103 | Android SDK. | 102 | Android SDK. |
| 104 | 103 | ||
| 105 | Still impure, for one of the two reasons the APK is: pub.dev resolution and | 104 | Still impure, for one of the two reasons the APK is: pub.dev resolution and |
| 106 | -Flutter's engine artifacts are network. What it does *not* need is the | 105 | +Flutter's engine artifacts are network. What it does *not* need is the Android |
| 107 | -writable-`ANDROID_HOME` dance, since nothing here writes into the store — so | 106 | +SDK, so `just build desktop` never asks `tools/toolchain.sh` for one. |
| 108 | -there is no `flutter/.home` on this path. | ||
| 109 | 107 | ||
| 110 | -nixGL off NixOS: Flutter paints through GL and the driver that can do that is | 108 | +GL is the host's, as is the driver that can do it. |
| 111 | -the host's. | ||
| 112 | 109 | ||
| 113 | `linux/` is the Flutter template's GTK runner, renamed — `frq` rather than | 110 | `linux/` is the Flutter template's GTK runner, renamed — `frq` rather than |
| 114 | `cljd_flutter`, and `uk.nandi.frq` rather than `com.example.cljd_flutter`, so | 111 | `cljd_flutter`, and `uk.nandi.frq` rather than `com.example.cljd_flutter`, so |
| @@ -132,28 +129,15 @@ Two things the desktop target changed in the Dart, both of them cases where | |||
| 132 | ### The APK | 129 | ### The APK |
| 133 | 130 | ||
| 134 | Impure on purpose. Gradle resolves its own dependencies over the network and | 131 | Impure on purpose. Gradle resolves its own dependencies over the network and |
| 135 | -installs build-tools and a platform into `ANDROID_HOME` as it goes, so it can | 132 | +has sdkmanager install build-tools and a platform into `ANDROID_HOME` as it |
| 136 | -neither run in a sandbox nor write to the store. What nix gives is the | 133 | +goes, so it cannot run in a sandbox and the SDK it writes to has to be ours. |
| 137 | -toolchain — clojure, a JDK, Flutter, and an SDK composed by androidenv — and | 134 | +`just tools android` fetches Google's command-line tools by pinned sha256 into |
| 138 | -the recipe copies that SDK to `flutter/.home` for Gradle to finish off. That | 135 | +`.toolchain/android-sdk` and accepts the licences; Gradle finishes the job from |
| 139 | -copy and everything Gradle leaves behind are gitignored. | 136 | +there. All of it is gitignored. |
| 140 | - | 137 | + |
| 141 | -All of it is the flake's, which it did not used to be. The toolchain was | 138 | +The platform and build-tools versions are deliberately not pinned here: they |
| 142 | -`nix shell nixpkgs#clojure nixpkgs#jdk17 nixpkgs#flutter` and the SDK was a | 139 | +come from whatever Flutter asks Gradle for, and pinning them in a second place |
| 143 | -`nix build --impure --expr` around `builtins.getFlake | 140 | +is how the two drift apart. |
| 144 | -"github:NixOS/nixpkgs/nixos-unstable"` — two references to an *unlocked* | ||
| 145 | -nixpkgs, so the Flutter that compiled the APK and the nixpkgs under everything | ||
| 146 | -else could drift apart without flake.lock changing a line. They are | ||
| 147 | -`devShells.<system>.flutter` and `packages.<system>.android-sdk` now, at the | ||
| 148 | -pinned rev, and the recipe is `nix develop .#flutter --command` over | ||
| 149 | -`nix build .#android-sdk`. | ||
| 150 | - | ||
| 151 | -The SDK needs `allowUnfree` and `android_sdk.accept_license`, which cannot be | ||
| 152 | -set on a `legacyPackages` attribute after the fact — hence `androidPkgsFor` in | ||
| 153 | -the flake, a second `import` of the same locked input rather than a second | ||
| 154 | -nixpkgs. The Flutter toolchain is kept out of the default dev shell: it brings | ||
| 155 | -its own Dart and a JDK's worth of closure, and a desktop build wants none of | ||
| 156 | -it. | ||
| 157 | 141 | ||
| 158 | Two things the Flutter template wanted that are deliberately not here. There is | 142 | Two things the Flutter template wanted that are deliberately not here. There is |
| 159 | no `ndkVersion` in `android/app/build.gradle.kts`: setting it makes Gradle | 143 | no `ndkVersion` in `android/app/build.gradle.kts`: setting it makes Gradle |
| @@ -249,7 +233,7 @@ move from. What remains is work the port never covered: | |||
| 249 | listen on localhost for a browser it does not own. That wants an app link or | 233 | listen on localhost for a browser it does not own. That wants an app link or |
| 250 | a custom scheme, an intent filter, and a redirect URI the broker will accept | 234 | a custom scheme, an intent filter, and a redirect URI the broker will accept |
| 251 | — a decision about freeq's broker, not a porting problem. The web build has | 235 | — a decision about freeq's broker, not a porting problem. The web build has |
| 252 | - its own answer in `frq.oauth.web`, and `just web-local` is why it only | 236 | + its own answer in `frq.oauth.web`, and `just serve` is why it only |
| 253 | completes on localhost. | 237 | completes on localhost. |
| 254 | 2. **Calls.** The signaling is IRC and is still in the screens; the media plane | 238 | 2. **Calls.** The signaling is IRC and is still in the screens; the media plane |
| 255 | it drove was `libjoltmoq` — Opus, H.264, V4L2, ALSA, MoQ over QUIC — under | 239 | it drove was `libjoltmoq` — Opus, H.264, V4L2, ALSA, MoQ over QUIC — under |
modified
justfile +203 -445 | @@ -1,384 +1,171 @@ | ||
| 1 | -# The work is here now, in recipe bodies, where it used to be in scripts/ — | |
| 2 | -# babashka scripts run through a scripts/bb that went looking for a babashka. | |
| 3 | -# That indirection bought one thing worth having, a shared way to reach nix on | |
| 4 | -# a host that keeps it in a container, and `nix` below is the whole of it. | |
| 1 | +# Six verbs over one tree. | |
| 5 | 2 | # |
| 6 | -# Every recipe that builds has the same shape: outside the dev shell, re-enter | |
| 7 | -# it and come back to this same recipe; inside, do the work. The re-entry test | |
| 8 | -# is an environment variable only the shell sets — no flag to forget, and no | |
| 9 | -# second code path for someone who runs `nix develop --command just ...` by | |
| 10 | -# hand. | |
| 3 | +# There is no nix here any more. `tools/toolchain.sh` fetches Flutter (which | |
| 4 | +# carries Dart), a JDK, the Clojure CLI and Nim as sha256-pinned tarballs into | |
| 5 | +# `.toolchain/`, and every recipe below runs inside the environment that | |
| 6 | +# script prints. One mechanism, and the same one the containers in `.modal/` | |
| 7 | +# use — which is what lets a plain Debian image build this. | |
| 8 | +# | |
| 9 | +# What the host still brings: a C compiler (Nim shells out to one), OpenSSL | |
| 10 | +# (nim.cfg is `-d:ssl`), git, curl, unzip, python3. For `run desktop` and the | |
| 11 | +# other Linux builds, GTK and the usual CMake/Ninja/pkg-config. | |
| 12 | +# | |
| 13 | +# just build TARGET apk desktop web lib ui app | |
| 14 | +# just run TARGET apk desktop web ui app | |
| 15 | +# just test SUITE all nim dart common live | |
| 16 | +# just modal CONTAINER dev web | |
| 17 | +# just serve [PORT] the Modal-built web bundle, on localhost | |
| 18 | +# just tools ... the toolchain itself | |
| 11 | 19 | |
| 12 | 20 | set shell := ["bash", "-euo", "pipefail", "-c"] |
| 13 | 21 | |
| 14 | -# Every recipe below is a `#!` script and passes its arguments on with "$@". | |
| 15 | -# Without this that is empty in one — just interpolates into a shebang recipe | |
| 16 | -# rather than handing it argv — and a recipe silently ignored its flags. | |
| 22 | +# Every recipe is a `#!` script. Without this, "$@" is empty in one of them — | |
| 23 | +# just interpolates into a shebang recipe rather than handing it argv. | |
| 17 | 24 | set positional-arguments |
| 18 | 25 | |
| 19 | -# nix is not on every host this runs on: on the machine these recipes were | |
| 20 | -# written for it lives in an Arch distrobox, at the same path — which is why | |
| 21 | -# the container is entered rather than the tree copied into it. See CLAUDE.md. | |
| 22 | -nix := `command -v nix >/dev/null 2>&1 && echo nix || echo "distrobox enter arch -- nix"` | |
| 23 | - | |
| 24 | -# --max-jobs 0 is what sends the work to the `builders` entry rather than | |
| 25 | -# compiling it here. Left to the default, nix prefers the local machine, and a | |
| 26 | -# cold Flutter toolchain is a lot of compiling on a laptop — for a derivation a | |
| 27 | -# remote builder has likely built already. FRQ_MAX_JOBS=auto is the way out on | |
| 28 | -# a machine with no builder configured. | |
| 29 | -jobs := env("FRQ_MAX_JOBS", "0") | |
| 26 | +root := justfile_directory() | |
| 27 | +tc := justfile_directory() / "tools/toolchain.sh" | |
| 30 | 28 | |
| 31 | 29 | default: |
| 32 | 30 | @just --list |
| 33 | 31 | |
| 34 | -# The APK: ClojureDart compiled to Dart, then Flutter's Gradle build. | |
| 35 | -# | |
| 36 | -# Impure on purpose, and worth saying why rather than leaving it to be | |
| 37 | -# discovered. Gradle resolves its own dependencies over the network and | |
| 38 | -# installs build-tools and a platform into ANDROID_HOME as it goes, so it | |
| 39 | -# cannot run in a sandbox and cannot write to the store. What nix gives here | |
| 40 | -# is the toolchain — clojure, a JDK, Flutter, and an SDK composed by | |
| 41 | -# androidenv — and the recipe copies that SDK somewhere writable | |
| 42 | -# (flutter/.home) for Gradle to finish off. That copy and everything Gradle | |
| 43 | -# leaves behind are gitignored. | |
| 44 | -# | |
| 45 | -# No ndkVersion in android/app/build.gradle.kts, for the same reason: the | |
| 46 | -# Flutter template sets it, setting it makes Gradle fetch that exact NDK, and | |
| 47 | -# there is no native code here to need one. | |
| 48 | -# | |
| 49 | -# just apk build the debug APK | |
| 50 | -# just apk install build it and put it on a connected device | |
| 51 | -# just apk run install and launch | |
| 52 | -# just apk log logcat, filtered to this app | |
| 53 | -apk action="build": | |
| 32 | +# The toolchain, directly: `just tools versions`, `just tools android`, | |
| 33 | +# `just tools exec -- flutter doctor`. | |
| 34 | +[doc('the toolchain itself: versions, android, exec -- CMD')] | |
| 35 | +tools *args: | |
| 36 | + #!/usr/bin/env bash | |
| 37 | + cd "{{root}}" | |
| 38 | + exec "{{tc}}" "$@" | |
| 39 | + | |
| 40 | +# Build a target. | |
| 41 | +# | |
| 42 | +# apk the Android app, via Gradle | |
| 43 | +# desktop the ClojureDart app on Flutter's Linux target | |
| 44 | +# web the same screens compiled to JavaScript, into build/web | |
| 45 | +# lib the Nim core as build/nim/libfrqcore.so | |
| 46 | +# ui Nim owning the state and the screens, painted by Flutter | |
| 47 | +# app the ClojureDart app with the Nim core as its transport only | |
| 48 | +[doc('build a target: apk desktop web lib ui app')] | |
| 49 | +build target="desktop": | |
| 54 | 50 | #!/usr/bin/env bash |
| 55 | 51 | set -euo pipefail |
| 56 | - cd "{{justfile_directory()}}/flutter" | |
| 57 | - | |
| 58 | - # The flake's, not an --impure --expr against whatever nixos-unstable is | |
| 59 | - # today: the licence config the SDK needs lives in `androidPkgsFor` now, | |
| 60 | - # so this is an ordinary output at the rev flake.lock pins. | |
| 61 | - sdk="$(nix build --no-link --print-out-paths \ | |
| 62 | - "{{justfile_directory()}}#android-sdk")/libexec/android-sdk" | |
| 63 | - | |
| 64 | - # adb keeps the key the phone has already trusted under the real HOME, and | |
| 65 | - # HOME moves below so Gradle can write into the SDK copy. Told where to | |
| 66 | - # look, adb keeps its identity; left to find $HOME/.android it generates a | |
| 67 | - # new one, the device stops recognising this machine, and the deploy ends | |
| 68 | - # in "no devices/emulators found" while `adb devices` in any other shell | |
| 69 | - # lists it perfectly well. | |
| 70 | - export ANDROID_USER_HOME="${ANDROID_USER_HOME:-$HOME/.android}" | |
| 71 | - | |
| 72 | - export HOME="$PWD/.home" | |
| 73 | - export ANDROID_HOME="$HOME/android-sdk" | |
| 74 | - export ANDROID_SDK_ROOT="$ANDROID_HOME" | |
| 75 | - mkdir -p "$HOME" | |
| 76 | - | |
| 77 | - # Gradle writes into ANDROID_HOME, so it is a copy rather than the store | |
| 78 | - # path. Made once and kept: re-copying would throw away the build-tools | |
| 79 | - # and platform Gradle installed into it on the last run. | |
| 80 | - if [ ! -d "$ANDROID_HOME" ]; then | |
| 81 | - cp -r "$sdk" "$ANDROID_HOME" | |
| 82 | - chmod -R u+w "$ANDROID_HOME" | |
| 83 | - fi | |
| 84 | - | |
| 85 | - # The compile's three caches, same flake-output-and-copy shape as the SDK | |
| 86 | - # and for the same reason: tools.deps and pub both write into theirs, and | |
| 87 | - # the store is read-only. What this buys is the "Resolving dependencies… | |
| 88 | - # Downloading packages…" that used to open every run. | |
| 89 | - deps="$(nix build --no-link --print-out-paths \ | |
| 90 | - "{{justfile_directory()}}#cljd-deps")" | |
| 91 | - | |
| 92 | - # Neither of these follows HOME. Both are read off the JVM's user.home, | |
| 93 | - # which comes from /etc/passwd rather than the environment — so moving | |
| 94 | - # HOME below is not enough to move them, and left alone they would be the | |
| 95 | - # real ~/.m2 and ~/.gitlibs, shared with every other project on the box. | |
| 96 | - export GITLIBS="$HOME/gitlibs" | |
| 97 | - m2="$HOME/m2" | |
| 98 | - export PUB_CACHE="$HOME/.pub-cache" | |
| 99 | - | |
| 100 | - # Seeded once each and then left alone, exactly as ANDROID_HOME is: after | |
| 101 | - # the first compile these hold whatever the working tree has asked for | |
| 102 | - # since, and re-copying would throw that away. | |
| 103 | - seed() { | |
| 104 | - [ -e "$2" ] && return 0 | |
| 105 | - mkdir -p "$(dirname "$2")" | |
| 106 | - cp -r "$deps/$1" "$2" | |
| 107 | - chmod -R u+w "$2" | |
| 108 | - } | |
| 109 | - seed m2 "$m2" | |
| 110 | - seed gitlibs "$GITLIBS" | |
| 111 | - seed pub-cache "$PUB_CACHE" | |
| 112 | - seed clojuredart/cache "$PWD/.clojuredart/cache" | |
| 113 | - | |
| 114 | - # Also the flake's. `nix shell nixpkgs#...` read the registry, which is a | |
| 115 | - # different and unlocked nixpkgs — the Flutter that built the APK could | |
| 116 | - # move under it without flake.lock changing a line. | |
| 117 | - flutter="nix develop {{justfile_directory()}}#flutter --command" | |
| 118 | - | |
| 119 | - # cljd-deps ships the analyzer project unresolved — a fixed-output | |
| 120 | - # derivation may not name the store, and a resolved pub project is | |
| 121 | - # nothing but store paths. So it is resolved here instead, offline, | |
| 122 | - # against the cache that derivation did fetch. ClojureDart reaches for the | |
| 123 | - # network only when bin/analyzer.dart is missing, and after this it is not. | |
| 124 | - for helper in .clojuredart/cache/*/cljd_helper; do | |
| 125 | - [ -d "$helper" ] || continue | |
| 126 | - [ -e "$helper/.dart_tool/package_config.json" ] && continue | |
| 127 | - ( cd "$helper" && $flutter flutter pub get --offline ) | |
| 128 | - done | |
| 129 | - | |
| 130 | - $flutter clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile | |
| 131 | - | |
| 132 | - # Rewritten every run: it carries absolute store paths, and the flutter | |
| 133 | - # one moves whenever nixpkgs does. | |
| 134 | - $flutter flutter config --android-sdk "$ANDROID_HOME" >/dev/null | |
| 135 | - | |
| 136 | - apk=build/app/outputs/flutter-apk/app-debug.apk | |
| 137 | - adb="${ADB:-$ANDROID_HOME/platform-tools/adb}" | |
| 138 | - | |
| 139 | - case "{{action}}" in | |
| 140 | - build) $flutter flutter build apk --debug ;; | |
| 141 | - install) $flutter flutter build apk --debug && "$adb" install -r "$apk" ;; | |
| 142 | - run) $flutter flutter build apk --debug && "$adb" install -r "$apk" \ | |
| 143 | - && "$adb" shell monkey -p uk.nandi.frq -c android.intent.category.LAUNCHER 1 ;; | |
| 144 | - log) "$adb" logcat -s flutter ;; | |
| 145 | - *) echo "usage: just apk [build|install|run|log]" >&2; exit 1 ;; | |
| 52 | + cd "{{root}}" | |
| 53 | + case "{{target}}" in | |
| 54 | + apk) just _flutter apk build ;; | |
| 55 | + desktop) just _flutter desktop build ;; | |
| 56 | + web) exec tools/build-web.sh build ;; | |
| 57 | + lib) just _nim-lib ;; | |
| 58 | + ui) just _flutter ui build ;; | |
| 59 | + app) just _flutter app build ;; | |
| 60 | + *) echo "usage: just build [apk|desktop|web|lib|ui|app]" >&2; exit 1 ;; | |
| 146 | 61 | esac |
| 147 | 62 | |
| 148 | -# What may appear in common/, checked. Needs nothing built: it reads the | |
| 149 | -# source, so it is the one check that runs anywhere, and CI runs exactly this. | |
| 150 | -check-common: | |
| 151 | - #!/usr/bin/env bash | |
| 152 | - python3 tools/check-common.py common | |
| 153 | - | |
| 154 | -# The desktop GUI: the same screens the APK paints, on Flutter's Linux target. | |
| 155 | -# | |
| 156 | -# This recipe and `just apk` are two targets over one tree, and the split is | |
| 157 | -# the one the APK already draws. Everything under `common/` — the screens, the | |
| 158 | -# cells, `frq.io` — is shared; what differs is who answers the host. So this | |
| 159 | -# is `just apk` with the Android half taken out: the same `clojure -M:cljd compile` over the same flutter/src, | |
| 160 | -# then Flutter's Linux target rather than its Android one. CMake and Ninja | |
| 161 | -# instead of Gradle, `flutter/linux/` as the runner, no SDK and no JDK. | |
| 63 | +# Build a target and start it. | |
| 162 | 64 | # |
| 163 | -# Still impure, for one of the two reasons `apk` is: pub.dev resolution and | |
| 164 | -# Flutter's own engine artifacts are network. What it does NOT need is the | |
| 165 | -# writable-ANDROID_HOME dance — nothing here writes into the store — so there | |
| 166 | -# is no `flutter/.home` on this path. | |
| 65 | +# apk install on a connected device and launch it | |
| 66 | +# web serve build/web on ARG (default 8080) | |
| 67 | +# the rest open the window | |
| 167 | 68 | # |
| 168 | -# nixGL because Flutter paints through GL, and off NixOS the driver is the | |
| 169 | -# host's. | |
| 170 | -# | |
| 171 | -# just flutter-desktop build the debug bundle | |
| 172 | -# just flutter-desktop run build it and open the window | |
| 173 | -flutter-desktop action="build": | |
| 69 | +# just run apk | |
| 70 | +# just run web 3000 | |
| 71 | +[doc('build a target and start it')] | |
| 72 | +run target="desktop" arg="": | |
| 174 | 73 | #!/usr/bin/env bash |
| 175 | 74 | set -euo pipefail |
| 176 | - cd "{{justfile_directory()}}" | |
| 177 | - if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then | |
| 178 | - exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \ | |
| 179 | - --command just flutter-desktop "$@" | |
| 180 | - fi | |
| 181 | - cd flutter | |
| 182 | - | |
| 183 | - # The same three caches `just apk` seeds, in the same place and out of the | |
| 184 | - # same flake output — one compiler, one set of dependencies, and no reason | |
| 185 | - # for the two frontends to keep a copy each. `.home/` is `apk`'s directory | |
| 186 | - # by name and this is the only thing put there from here, which is the | |
| 187 | - # point: whichever recipe runs first pays for the copy and the other finds | |
| 188 | - # it warm. | |
| 189 | - # | |
| 190 | - # No `nix build` here, unlike `apk`: this recipe is already inside the | |
| 191 | - # shell that names FRQ_CLJD_DEPS by the time it gets this far, and `apk` | |
| 192 | - # needs the path before it enters anything. | |
| 193 | - # | |
| 194 | - # m2 and gitlibs are set here for the reason they are set there — the JVM | |
| 195 | - # reads user.home out of /etc/passwd, so neither follows HOME and left | |
| 196 | - # alone they are the real ~/.m2 and ~/.gitlibs. | |
| 197 | - export PUB_CACHE="$PWD/.home/.pub-cache" | |
| 198 | - export GITLIBS="$PWD/.home/gitlibs" | |
| 199 | - m2="$PWD/.home/m2" | |
| 200 | - | |
| 201 | - seed() { | |
| 202 | - [ -e "$2" ] && return 0 | |
| 203 | - mkdir -p "$(dirname "$2")" | |
| 204 | - cp -r "$FRQ_CLJD_DEPS/$1" "$2" | |
| 205 | - chmod -R u+w "$2" | |
| 206 | - } | |
| 207 | - seed m2 "$m2" | |
| 208 | - seed gitlibs "$GITLIBS" | |
| 209 | - seed pub-cache "$PUB_CACHE" | |
| 210 | - seed clojuredart/cache "$PWD/.clojuredart/cache" | |
| 211 | - | |
| 212 | - # Resolved here rather than in cljd-deps, which was not allowed to name | |
| 213 | - # the store — see the same loop in `apk`. | |
| 214 | - for helper in .clojuredart/cache/*/cljd_helper; do | |
| 215 | - [ -d "$helper" ] || continue | |
| 216 | - [ -e "$helper/.dart_tool/package_config.json" ] && continue | |
| 217 | - ( cd "$helper" && flutter pub get --offline ) | |
| 218 | - done | |
| 219 | - | |
| 220 | - clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile | |
| 221 | - flutter build linux --debug | |
| 222 | - | |
| 223 | - # x64/arm64 is Flutter's own name for the host arch, not uname's. | |
| 224 | - case "$(uname -m)" in | |
| 225 | - x86_64) arch=x64 ;; | |
| 226 | - aarch64) arch=arm64 ;; | |
| 227 | - *) echo "unknown arch $(uname -m)" >&2; exit 1 ;; | |
| 228 | - esac | |
| 229 | - bundle="build/linux/$arch/debug/bundle" | |
| 230 | - | |
| 231 | - case "{{action}}" in | |
| 232 | - build) echo "built $PWD/$bundle/frq" ;; | |
| 233 | - run) | |
| 234 | - runner=() | |
| 235 | - [ -e /run/current-system ] || runner=("$NIXGL") | |
| 236 | - exec "${runner[@]}" "$bundle/frq" | |
| 237 | - ;; | |
| 238 | - *) echo "usage: just flutter-desktop [build|run]" >&2; exit 1 ;; | |
| 75 | + cd "{{root}}" | |
| 76 | + case "{{target}}" in | |
| 77 | + apk) just _flutter apk run ;; | |
| 78 | + desktop) just _flutter desktop run ;; | |
| 79 | + web) port="${2:-}"; exec tools/build-web.sh serve "${port:-8080}" ;; | |
| 80 | + ui) just _flutter ui run ;; | |
| 81 | + app) just _flutter app run ;; | |
| 82 | + log) just tools exec -- adb logcat -s flutter ;; | |
| 83 | + *) echo "usage: just run [apk|desktop|web|ui|app|log]" >&2; exit 1 ;; | |
| 239 | 84 | esac |
| 240 | 85 | |
| 241 | -# The third frontend: the same screens again, compiled to JavaScript. | |
| 242 | -# | |
| 243 | -# `flutter-desktop` with the Linux half taken out. One `clojure -M:cljd | |
| 244 | -# compile` over the same flutter/src and common/, then Flutter's web target | |
| 245 | -# instead of its Linux one — dart2js instead of CMake and Ninja, and a | |
| 246 | -# directory of static files instead of a bundle with an executable in it. | |
| 247 | -# | |
| 248 | -# And the one target with no nix in it. The other two need the host: a JDK | |
| 249 | -# and the Android SDK for `apk`, GTK and a C++ toolchain and nixGL for | |
| 250 | -# `flutter-desktop`. This one needs a Dart, a JVM and a browser, and the | |
| 251 | -# browser is not ours — so `tools/toolchain.sh` fetches the first two as | |
| 252 | -# pinned tarballs into `.toolchain/` and there is nothing left for a devShell | |
| 253 | -# to supply. That is what lets the container in `.modal/flutter-web/` drop | |
| 254 | -# its image build too: same script, same three pins, no store to populate. | |
| 255 | -# | |
| 256 | -# Impure for the reason the other two are: pub.dev resolution and Flutter's | |
| 257 | -# engine artifacts are network, and now the toolchain is as well — pinned by | |
| 258 | -# sha256, which is the reproducibility that was worth having out of the store. | |
| 259 | -# | |
| 260 | -# The entry point is `frq.main-web`, not `frq.main`: path_provider has no web | |
| 261 | -# implementation, so the `getApplicationSupportDirectory` that `frq.main` | |
| 262 | -# awaits throws MissingPluginException before any widget is built. The web | |
| 263 | -# entry installs `frq.io.web` — localStorage behind the same seam — and awaits | |
| 264 | -# nothing. `frq.net.dart` is still the socket half, so connecting will want a | |
| 265 | -# WebSocket before this does more than paint. | |
| 266 | -# | |
| 267 | -# One build and no `--debug` variant, because there is nothing to gain from | |
| 268 | -# one: dart2js at -O1 measured 52.5s against the release build's 49.8s on the | |
| 269 | -# same source change here, so a second, larger bundle would buy noise. See | |
| 270 | -# `tools/build-web.sh`, which writes the numbers down. | |
| 271 | -# | |
| 272 | -# just flutter-web build build/web | |
| 273 | -# just flutter-web serve build it and serve it on $PORT (8080) | |
| 274 | -# just flutter-web serve 3000 ...on another port | |
| 275 | -flutter-web action="build" port="8080": | |
| 86 | +# Test a suite. | |
| 87 | +# | |
| 88 | +# common what may appear in common/, read off the source. Needs no | |
| 89 | +# toolchain at all, which is why CI runs exactly this. | |
| 90 | +# nim the Nim core. No Flutter, no Dart, no SDK — a rule about the IRC | |
| 91 | +# wire format is checkable in a second. | |
| 92 | +# dart the Dart side of the FFI boundary, on the plain Dart VM. Passing | |
| 93 | +# `nim` and failing this one is a marshalling bug, which is why the | |
| 94 | +# two are separate suites. | |
| 95 | +# live the whole stack against a real freeq. Not in `all`: it wants a | |
| 96 | +# network and a running server. | |
| 97 | +# | |
| 98 | +# just test all of them | |
| 99 | +# just test nim tircparse one Nim file | |
| 100 | +[doc('run a suite: all common nim dart live')] | |
| 101 | +test suite="all" *args: | |
| 276 | 102 | #!/usr/bin/env bash |
| 277 | 103 | set -euo pipefail |
| 278 | - # A wrapper and nothing else. The build is a shell script because the | |
| 279 | - # container runs it too, and a container that had to install `just` to | |
| 280 | - # start would be one dependency away from the point. | |
| 281 | - exec "{{justfile_directory()}}/tools/build-web.sh" {{action}} {{port}} | |
| 104 | + cd "{{root}}" | |
| 105 | + shift | |
| 106 | + case "{{suite}}" in | |
| 107 | + all) just test common && just test nim && just test dart ;; | |
| 108 | + common) exec python3 tools/check-common.py common ;; | |
| 109 | + nim) just _nim-test "$@" ;; | |
| 110 | + dart) just _nim-lib | |
| 111 | + exec "{{tc}}" exec -- bash -c \ | |
| 112 | + 'cd dart/frq_core && dart pub get && dart test -r expanded' ;; | |
| 113 | + live) just _nim-lib | |
| 114 | + exec "{{tc}}" exec -- bash -c \ | |
| 115 | + 'cd dart/frq_core && dart pub get >/dev/null && dart run tool/live_ui.dart "$@"' _ "$@" ;; | |
| 116 | + *) echo "usage: just test [all|common|nim|dart|live]" >&2; exit 1 ;; | |
| 117 | + esac | |
| 282 | 118 | |
| 283 | -# The containers in `.modal/`, run on Modal rather than here. This machine | |
| 284 | -# evaluates and Modal builds — see CLAUDE.md, which says so rather more | |
| 285 | -# firmly — and these two recipes are the whole interface to that. | |
| 119 | +# The containers in `.modal/`, run on Modal rather than here: this machine | |
| 120 | +# evaluates and Modal builds. See CLAUDE.md, which says so rather more firmly. | |
| 286 | 121 | # |
| 287 | -# Named for where the work happens, the way `flutter-desktop` is named for | |
| 288 | -# what paints: there is no re-entry test here because nothing | |
| 289 | -# re-enters. There is no `nix` variable either, and that used to be because | |
| 290 | -# nix ran out there — now it is because neither of these containers has any | |
| 291 | -# nix in it at all. | |
| 122 | +# `--shell` leaves a sandbox running with the container's own image, volumes | |
| 123 | +# and environment, and prints the command to attach to it. It blocks — an | |
| 124 | +# ephemeral app takes its sandbox down when the entrypoint returns — so attach | |
| 125 | +# from a second terminal and Ctrl-C here when done. The sandbox bills until | |
| 126 | +# you do. | |
| 292 | 127 | # |
| 293 | -# just modal flutter-dev the incremental Flutter loop | |
| 294 | -# just modal flutter-web the web bundle | |
| 295 | -modal container="flutter-dev" *args: | |
| 128 | +# just modal dev | |
| 129 | +# just modal web --shell | |
| 130 | +[doc('run a .modal/ container on Modal')] | |
| 131 | +modal container="dev" *args: | |
| 296 | 132 | #!/usr/bin/env bash |
| 297 | 133 | set -euo pipefail |
| 298 | - cd "{{justfile_directory()}}" | |
| 299 | - shift | |
| 134 | + cd "{{root}}" | |
| 135 | + shift || true | |
| 300 | 136 | exec modal run ".modal/{{container}}/container.py" "$@" |
| 301 | 137 | |
| 302 | 138 | # The Modal-built web bundle, served from this machine on localhost. |
| 303 | 139 | # |
| 304 | -# Not a local build: `modal volume get` pulls what `just modal flutter-web` | |
| 305 | -# already compiled out of the devshell volume, so this needs no Flutter, no | |
| 306 | -# Dart and no nix — only python, which the flake shell has and so does the | |
| 307 | -# machine. | |
| 308 | -# | |
| 309 | -# It exists for one reason, and the reason is the auth broker rather than | |
| 310 | -# convenience. freeq's broker finishes an OAuth login by redirecting the | |
| 311 | -# browser to `return_to`, and it will only redirect to an origin on its | |
| 312 | -# allowlist: its own https hosts, and `http://localhost` or `http://127.0.0.1` | |
| 313 | -# on ANY port. A build served from anywhere else — the Modal URL included — | |
| 314 | -# gets `400 Invalid return_to URL` and can never complete a Bluesky sign-in, | |
| 315 | -# no matter what the client does. localhost is the one allowlisted origin we | |
| 316 | -# can serve from, so this is how Bluesky sign-in is tested. | |
| 317 | -# | |
| 318 | -# Guest and app-password sign-in need none of this; they work on the deployed | |
| 319 | -# URL, because neither goes near the broker. | |
| 320 | -# | |
| 321 | -# just web-local fetch and serve on :8080 | |
| 322 | -# just web-local 3000 another port; any port is allowlisted | |
| 323 | -web-local port="8080": | |
| 140 | +# Not a local build: `modal volume get` pulls down what `just modal | |
| 141 | +# web` already compiled, so this needs only python3. | |
| 142 | +# | |
| 143 | +# It exists for the auth broker rather than for convenience. freeq's broker | |
| 144 | +# finishes an OAuth login by redirecting to `return_to`, and only to an origin | |
| 145 | +# on its allowlist: its own https hosts, and http://localhost or | |
| 146 | +# http://127.0.0.1 on ANY port. Served from anywhere else — the Modal URL | |
| 147 | +# included — a Bluesky sign-in gets `400 Invalid return_to URL` and can never | |
| 148 | +# complete. Guest and app-password sign-in work on the deployed URL; neither | |
| 149 | +# goes near the broker. | |
| 150 | +[doc('serve the Modal-built web bundle on localhost')] | |
| 151 | +serve port="8080": | |
| 324 | 152 | #!/usr/bin/env bash |
| 325 | 153 | set -euo pipefail |
| 326 | - cd "{{justfile_directory()}}" | |
| 327 | - out="{{justfile_directory()}}/.web-local" | |
| 154 | + cd "{{root}}" | |
| 155 | + out="{{root}}/.web-local" | |
| 328 | 156 | mkdir -p "$out" |
| 329 | 157 | echo "fetching the Modal-built bundle…" |
| 330 | 158 | # --force: this is a mirror of the volume, and a stale file left behind |
| 331 | 159 | # would be served in preference to the one just built. |
| 332 | - modal volume get --force devshell frq-flutter-web/flutter/build/web "$out" | |
| 160 | + modal volume get --force devshell frq-web/flutter/build/web "$out" | |
| 333 | 161 | echo |
| 334 | 162 | echo " http://localhost:{{port}}" |
| 335 | 163 | echo |
| 336 | - echo "Bluesky sign-in works here and not on the Modal URL: the broker" | |
| 337 | - echo "allowlists localhost on any port. Put wss://irc.freeq.at/irc in the" | |
| 338 | - echo "Server field — a browser has no TCP." | |
| 164 | + echo "Bluesky sign-in works here and not on the Modal URL. Put" | |
| 165 | + echo "wss://irc.freeq.at/irc in the Server field — a browser has no TCP." | |
| 339 | 166 | exec python3 -m http.server {{port}} --bind 127.0.0.1 --directory "$out/web" |
| 340 | 167 | |
| 341 | -# A sandbox left running with the container's own image, volumes and | |
| 342 | -# environment, and the command to get into it. `modal shell --image` cannot | |
| 343 | -# be pointed at a published Modal image like arch-nix, so attaching to a | |
| 344 | -# running sandbox is the only way to get a shell that is the container. | |
| 345 | -# | |
| 346 | -# It blocks: an ephemeral app stops when its entrypoint returns and takes the | |
| 347 | -# sandbox with it. Attach from a second terminal, and Ctrl-C here when done — | |
| 348 | -# the sandbox bills until you do. | |
| 349 | -# | |
| 350 | -# just modal-shell flutter-dev, the usual one | |
| 351 | -# just modal-shell flutter-web the web bundle container | |
| 352 | -modal-shell container="flutter-dev": | |
| 353 | - #!/usr/bin/env bash | |
| 354 | - set -euo pipefail | |
| 355 | - cd "{{justfile_directory()}}" | |
| 356 | - exec modal run ".modal/{{container}}/container.py" --shell | |
| 357 | - | |
| 358 | -# The Nim core's test suite. | |
| 359 | -# | |
| 360 | -# It needs no Flutter, no Dart and no Android SDK — which is the point of | |
| 361 | -# having the logic here rather than under `common/`: a rule about the IRC wire | |
| 362 | -# format can be checked in a second, on any machine, without a toolchain that | |
| 363 | -# takes minutes to enter. | |
| 364 | -# | |
| 365 | -# just nim-test the whole suite | |
| 366 | -# just nim-test tircparse one file | |
| 367 | -nim-test file="": | |
| 368 | - #!/usr/bin/env bash | |
| 369 | - set -euo pipefail | |
| 370 | - cd "{{justfile_directory()}}" | |
| 371 | - if [ -z "${FRQ_NIM:-}" ]; then | |
| 372 | - exec {{nix}} develop .#nim --max-jobs {{jobs}} --command just nim-test "$@" | |
| 373 | - fi | |
| 374 | - cd nim | |
| 375 | - if [ -n "{{file}}" ]; then | |
| 376 | - exec nim c -r --hints:off --path:src "tests/{{file}}.nim" | |
| 377 | - fi | |
| 378 | - for t in tests/t*.nim; do | |
| 379 | - echo "== $t" | |
| 380 | - nim c -r --hints:off --path:src "$t" | |
| 381 | - done | |
| 168 | +# --- the work behind the verbs ------------------------------------------ | |
| 382 | 169 | |
| 383 | 170 | # The Nim core as a shared library, into build/nim. |
| 384 | 171 | # |
| @@ -387,117 +174,88 @@ nim-test file="": | ||
| 387 | 174 | # from another heap and dereferencing it segfaults. ORC's heap is shared. |
| 388 | 175 | # |
| 389 | 176 | # `-d:release` and not `-d:danger`: the bounds checks are what turn a |
| 390 | -# malformed line off a socket into an exception instead of a read past the end | |
| 391 | -# of a buffer, and this parses exactly that. | |
| 392 | -nim-lib: | |
| 177 | +# malformed line off a socket into an exception rather than a read past the | |
| 178 | +# end of a buffer, and this parses exactly that. | |
| 179 | +[private] | |
| 180 | +_nim-lib: | |
| 393 | 181 | #!/usr/bin/env bash |
| 394 | 182 | set -euo pipefail |
| 395 | - cd "{{justfile_directory()}}" | |
| 396 | - if [ -z "${FRQ_NIM:-}" ]; then | |
| 397 | - exec {{nix}} develop .#nim --max-jobs {{jobs}} --command just nim-lib | |
| 398 | - fi | |
| 399 | - out="{{justfile_directory()}}/build/nim" | |
| 183 | + cd "{{root}}" | |
| 184 | + out="{{root}}/build/nim" | |
| 400 | 185 | mkdir -p "$out" |
| 401 | - cd nim | |
| 402 | - nim c --app:lib --mm:orc -d:release --hints:off \ | |
| 403 | - --path:src --out:"$out/libfrqcore.so" src/frq_core.nim | |
| 404 | - echo "built $out/libfrqcore.so" | |
| 405 | - nm -D --defined-only "$out/libfrqcore.so" | grep ' T frq_' || true | |
| 406 | - | |
| 407 | -# The Dart side of the Nim boundary, on the plain Dart VM. | |
| 408 | -# | |
| 409 | -# No Flutter, no emulator, no ClojureDart — `dart/frq_core` is ordinary Dart | |
| 410 | -# over `dart:ffi` and is not a Flutter package, so the test that proves the | |
| 411 | -# marshalling runs in a second. Passing `just nim-test` and failing this one is a | |
| 412 | -# marshalling bug, which is the whole reason the two suites are separate. | |
| 413 | -# | |
| 414 | -# Builds the library first: the test dlopens a real .so and there is no point | |
| 415 | -# reporting that it could not find one. | |
| 416 | -dart-test: | |
| 186 | + exec "{{tc}}" exec -- bash -euo pipefail -c ' | |
| 187 | + cd nim | |
| 188 | + nim c --app:lib --mm:orc -d:release --hints:off \ | |
| 189 | + --path:src --out:"'"$out"'/libfrqcore.so" src/frq_core.nim | |
| 190 | + echo "built '"$out"'/libfrqcore.so" | |
| 191 | + nm -D --defined-only "'"$out"'/libfrqcore.so" | grep " T frq_" || true' | |
| 192 | + | |
| 193 | +[private] | |
| 194 | +_nim-test *args: | |
| 417 | 195 | #!/usr/bin/env bash |
| 418 | 196 | set -euo pipefail |
| 419 | - cd "{{justfile_directory()}}" | |
| 420 | - if [ -z "${FRQ_DART:-}" ]; then | |
| 421 | - # nim-lib before the re-entry, not after: it enters a shell of its own | |
| 422 | - # and doing it on the far side would build the library twice. | |
| 423 | - just nim-lib | |
| 424 | - exec {{nix}} develop .#dart --max-jobs {{jobs}} --command just dart-test | |
| 425 | - fi | |
| 426 | - cd dart/frq_core | |
| 427 | - dart pub get | |
| 428 | - dart test -r expanded | |
| 429 | - | |
| 430 | -# The whole Nim stack against a real freeq: socket, state and screens. | |
| 431 | -# | |
| 432 | -# Connects, waits for the room list, opens a room and prints what the tree | |
| 433 | -# actually contains — through the FFI, so it is the path the window uses. | |
| 434 | -# Not in any suite: it needs a network and a running freeq. | |
| 435 | -nim-live *args: | |
| 436 | - #!/usr/bin/env bash | |
| 437 | - set -euo pipefail | |
| 438 | - cd "{{justfile_directory()}}" | |
| 439 | - if [ -z "${FRQ_DART:-}" ]; then | |
| 440 | - just nim-lib | |
| 441 | - exec {{nix}} develop .#dart --max-jobs {{jobs}} --command just nim-live "$@" | |
| 442 | - fi | |
| 443 | - shift || true | |
| 444 | - cd dart/frq_core | |
| 445 | - dart pub get >/dev/null | |
| 446 | - exec dart run tool/live_ui.dart "$@" | |
| 447 | - | |
| 448 | -# frq with Nim owning the state and the screens, rendered by Flutter. | |
| 449 | -# | |
| 450 | -# No ClojureDart on this path. `lib/main_nim.dart` asks the Nim core for a | |
| 451 | -# widget tree and paints it; the screens are ports of `common/frq/screens/`. | |
| 452 | -# | |
| 453 | -# just nim-ui build it | |
| 454 | -# just nim-ui run open the window | |
| 455 | -nim-ui action="build": | |
| 456 | - #!/usr/bin/env bash | |
| 457 | - set -euo pipefail | |
| 458 | - cd "{{justfile_directory()}}" | |
| 459 | - if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then | |
| 460 | - just nim-lib | |
| 461 | - exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \ | |
| 462 | - --command just nim-ui "$@" | |
| 463 | - fi | |
| 464 | - cd flutter | |
| 465 | - flutter pub get | |
| 466 | - export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" | |
| 467 | - runner=() | |
| 468 | - [ -e /run/current-system ] || runner=("$NIXGL") | |
| 469 | - case "{{action}}" in | |
| 470 | - build) exec "${runner[@]}" flutter build linux --debug -t lib/main_nim.dart ;; | |
| 471 | - run) exec "${runner[@]}" flutter run -d linux -t lib/main_nim.dart ;; | |
| 472 | - *) echo "usage: just nim-ui [build|run]" >&2; exit 1 ;; | |
| 473 | - esac | |
| 474 | - | |
| 475 | -# The ClojureDart app, with the Nim core as its transport only. | |
| 476 | -# | |
| 477 | -# This is the wiring that matters: `frq.main-nim` is `frq.main` with one line | |
| 478 | -# changed — `frq.net.nim/install!` where it says `frq.net.dart/install!`. Every | |
| 479 | -# screen, every cell and every action is the one that was already there. Nim | |
| 480 | -# owns the socket, the TLS and the line framing, and nothing else. | |
| 481 | -# | |
| 482 | -# just nim-app build it | |
| 483 | -# just nim-app run open the window | |
| 484 | -nim-app action="build": | |
| 197 | + cd "{{root}}" | |
| 198 | + exec "{{tc}}" exec -- bash -euo pipefail -c ' | |
| 199 | + cd nim | |
| 200 | + if [ -n "${1:-}" ]; then | |
| 201 | + exec nim c -r --hints:off --path:src "tests/$1.nim" | |
| 202 | + fi | |
| 203 | + for t in tests/t*.nim; do | |
| 204 | + echo "== $t" | |
| 205 | + nim c -r --hints:off --path:src "$t" | |
| 206 | + done' _ "$@" | |
| 207 | + | |
| 208 | +# The three Flutter targets, which differ only in what they compile and which | |
| 209 | +# entry point they paint. | |
| 210 | +# | |
| 211 | +# apk ClojureDart, then Gradle. Impure on purpose: Gradle resolves its | |
| 212 | +# own dependencies over the network and has sdkmanager install a | |
| 213 | +# platform and build-tools into ANDROID_HOME as it goes, so the | |
| 214 | +# SDK has to be writable — which is what `just tools android` gets | |
| 215 | +# it. Everything it leaves behind is under `.toolchain/` and | |
| 216 | +# gitignored. | |
| 217 | +# desktop the same `clojure -M:cljd compile`, Flutter's Linux target. | |
| 218 | +# ui no ClojureDart at all: `lib/main_nim.dart` asks the Nim core for | |
| 219 | +# a widget tree and paints it. | |
| 220 | +# app `frq.main-nim` is `frq.main` with one line changed — | |
| 221 | +# `frq.net.nim/install!` where it said `frq.net.dart/install!`. | |
| 222 | +# Every screen, cell and action is the one that was already there. | |
| 223 | +[private] | |
| 224 | +_flutter target action: | |
| 485 | 225 | #!/usr/bin/env bash |
| 486 | 226 | set -euo pipefail |
| 487 | - cd "{{justfile_directory()}}" | |
| 488 | - if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then | |
| 489 | - just nim-lib | |
| 490 | - exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \ | |
| 491 | - --command just nim-app "$@" | |
| 492 | - fi | |
| 493 | - cd flutter | |
| 494 | - flutter pub get | |
| 495 | - export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" | |
| 496 | - clojure -M:cljd compile frq.main-nim | |
| 497 | - runner=() | |
| 498 | - [ -e /run/current-system ] || runner=("$NIXGL") | |
| 499 | - case "{{action}}" in | |
| 500 | - build) exec "${runner[@]}" flutter build linux --debug -t lib/main_nim_app.dart ;; | |
| 501 | - run) exec "${runner[@]}" flutter run -d linux -t lib/main_nim_app.dart ;; | |
| 502 | - *) echo "usage: just nim-app [build|run]" >&2; exit 1 ;; | |
| 227 | + cd "{{root}}" | |
| 228 | + case "{{target}}" in | |
| 229 | + apk) "{{tc}}" android ;; | |
| 230 | + ui|app) just _nim-lib ;; | |
| 503 | 231 | esac |
| 232 | + exec "{{tc}}" exec -- bash -euo pipefail -c ' | |
| 233 | + cd flutter | |
| 234 | + # Nim resolves OpenSSL through dynlib at run time; without the host | |
| 235 | + # library on the loader path `newContext` dies in a SIGSEGV that says | |
| 236 | + # nothing about SSL. | |
| 237 | + export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" | |
| 238 | + cljd() { clojure -Sdeps "{:mvn/local-repo \"$FRQ_M2\"}" -M:cljd compile "$@"; } | |
| 239 | + | |
| 240 | + case "$1:$2" in | |
| 241 | + apk:*) | |
| 242 | + cljd | |
| 243 | + # Rewritten every run: it carries absolute paths. | |
| 244 | + flutter config --android-sdk "$ANDROID_HOME" >/dev/null | |
| 245 | + flutter build apk --debug | |
| 246 | + apk=build/app/outputs/flutter-apk/app-debug.apk | |
| 247 | + [ "$2" = run ] || exit 0 | |
| 248 | + adb install -r "$apk" | |
| 249 | + exec adb shell monkey -p uk.nandi.frq \ | |
| 250 | + -c android.intent.category.LAUNCHER 1 ;; | |
| 251 | + desktop:build) cljd; exec flutter build linux --debug ;; | |
| 252 | + desktop:run) cljd; exec flutter run -d linux ;; | |
| 253 | + ui:build) flutter pub get | |
| 254 | + exec flutter build linux --debug -t lib/main_nim.dart ;; | |
| 255 | + ui:run) flutter pub get | |
| 256 | + exec flutter run -d linux -t lib/main_nim.dart ;; | |
| 257 | + app:build) flutter pub get; cljd frq.main-nim | |
| 258 | + exec flutter build linux --debug -t lib/main_nim_app.dart ;; | |
| 259 | + app:run) flutter pub get; cljd frq.main-nim | |
| 260 | + exec flutter run -d linux -t lib/main_nim_app.dart ;; | |
| 261 | + esac' _ "{{target}}" "{{action}}" | |
| @@ -1,384 +1,171 @@ | |||
| 1 | -# The work is here now, in recipe bodies, where it used to be in scripts/ — | 1 | +# Six verbs over one tree. |
| 2 | -# babashka scripts run through a scripts/bb that went looking for a babashka. | ||
| 3 | -# That indirection bought one thing worth having, a shared way to reach nix on | ||
| 4 | -# a host that keeps it in a container, and `nix` below is the whole of it. | ||
| 5 | # | 2 | # |
| 6 | -# Every recipe that builds has the same shape: outside the dev shell, re-enter | 3 | +# There is no nix here any more. `tools/toolchain.sh` fetches Flutter (which |
| 7 | -# it and come back to this same recipe; inside, do the work. The re-entry test | 4 | +# carries Dart), a JDK, the Clojure CLI and Nim as sha256-pinned tarballs into |
| 8 | -# is an environment variable only the shell sets — no flag to forget, and no | 5 | +# `.toolchain/`, and every recipe below runs inside the environment that |
| 9 | -# second code path for someone who runs `nix develop --command just ...` by | 6 | +# script prints. One mechanism, and the same one the containers in `.modal/` |
| 10 | -# hand. | 7 | +# use — which is what lets a plain Debian image build this. |
| 8 | +# | ||
| 9 | +# What the host still brings: a C compiler (Nim shells out to one), OpenSSL | ||
| 10 | +# (nim.cfg is `-d:ssl`), git, curl, unzip, python3. For `run desktop` and the | ||
| 11 | +# other Linux builds, GTK and the usual CMake/Ninja/pkg-config. | ||
| 12 | +# | ||
| 13 | +# just build TARGET apk desktop web lib ui app | ||
| 14 | +# just run TARGET apk desktop web ui app | ||
| 15 | +# just test SUITE all nim dart common live | ||
| 16 | +# just modal CONTAINER dev web | ||
| 17 | +# just serve [PORT] the Modal-built web bundle, on localhost | ||
| 18 | +# just tools ... the toolchain itself | ||
| 11 | 19 | ||
| 12 | set shell := ["bash", "-euo", "pipefail", "-c"] | 20 | set shell := ["bash", "-euo", "pipefail", "-c"] |
| 13 | 21 | ||
| 14 | -# Every recipe below is a `#!` script and passes its arguments on with "$@". | 22 | +# Every recipe is a `#!` script. Without this, "$@" is empty in one of them — |
| 15 | -# Without this that is empty in one — just interpolates into a shebang recipe | 23 | +# just interpolates into a shebang recipe rather than handing it argv. |
| 16 | -# rather than handing it argv — and a recipe silently ignored its flags. | ||
| 17 | set positional-arguments | 24 | set positional-arguments |
| 18 | 25 | ||
| 19 | -# nix is not on every host this runs on: on the machine these recipes were | 26 | +root := justfile_directory() |
| 20 | -# written for it lives in an Arch distrobox, at the same path — which is why | 27 | +tc := justfile_directory() / "tools/toolchain.sh" |
| 21 | -# the container is entered rather than the tree copied into it. See CLAUDE.md. | ||
| 22 | -nix := `command -v nix >/dev/null 2>&1 && echo nix || echo "distrobox enter arch -- nix"` | ||
| 23 | - | ||
| 24 | -# --max-jobs 0 is what sends the work to the `builders` entry rather than | ||
| 25 | -# compiling it here. Left to the default, nix prefers the local machine, and a | ||
| 26 | -# cold Flutter toolchain is a lot of compiling on a laptop — for a derivation a | ||
| 27 | -# remote builder has likely built already. FRQ_MAX_JOBS=auto is the way out on | ||
| 28 | -# a machine with no builder configured. | ||
| 29 | -jobs := env("FRQ_MAX_JOBS", "0") | ||
| 30 | 28 | ||
| 31 | default: | 29 | default: |
| 32 | @just --list | 30 | @just --list |
| 33 | 31 | ||
| 34 | -# The APK: ClojureDart compiled to Dart, then Flutter's Gradle build. | 32 | +# The toolchain, directly: `just tools versions`, `just tools android`, |
| 35 | -# | 33 | +# `just tools exec -- flutter doctor`. |
| 36 | -# Impure on purpose, and worth saying why rather than leaving it to be | 34 | +[doc('the toolchain itself: versions, android, exec -- CMD')] |
| 37 | -# discovered. Gradle resolves its own dependencies over the network and | 35 | +tools *args: |
| 38 | -# installs build-tools and a platform into ANDROID_HOME as it goes, so it | 36 | + #!/usr/bin/env bash |
| 39 | -# cannot run in a sandbox and cannot write to the store. What nix gives here | 37 | + cd "{{root}}" |
| 40 | -# is the toolchain — clojure, a JDK, Flutter, and an SDK composed by | 38 | + exec "{{tc}}" "$@" |
| 41 | -# androidenv — and the recipe copies that SDK somewhere writable | 39 | + |
| 42 | -# (flutter/.home) for Gradle to finish off. That copy and everything Gradle | 40 | +# Build a target. |
| 43 | -# leaves behind are gitignored. | 41 | +# |
| 44 | -# | 42 | +# apk the Android app, via Gradle |
| 45 | -# No ndkVersion in android/app/build.gradle.kts, for the same reason: the | 43 | +# desktop the ClojureDart app on Flutter's Linux target |
| 46 | -# Flutter template sets it, setting it makes Gradle fetch that exact NDK, and | 44 | +# web the same screens compiled to JavaScript, into build/web |
| 47 | -# there is no native code here to need one. | 45 | +# lib the Nim core as build/nim/libfrqcore.so |
| 48 | -# | 46 | +# ui Nim owning the state and the screens, painted by Flutter |
| 49 | -# just apk build the debug APK | 47 | +# app the ClojureDart app with the Nim core as its transport only |
| 50 | -# just apk install build it and put it on a connected device | 48 | +[doc('build a target: apk desktop web lib ui app')] |
| 51 | -# just apk run install and launch | 49 | +build target="desktop": |
| 52 | -# just apk log logcat, filtered to this app | ||
| 53 | -apk action="build": | ||
| 54 | #!/usr/bin/env bash | 50 | #!/usr/bin/env bash |
| 55 | set -euo pipefail | 51 | set -euo pipefail |
| 56 | - cd "{{justfile_directory()}}/flutter" | 52 | + cd "{{root}}" |
| 57 | - | 53 | + case "{{target}}" in |
| 58 | - # The flake's, not an --impure --expr against whatever nixos-unstable is | 54 | + apk) just _flutter apk build ;; |
| 59 | - # today: the licence config the SDK needs lives in `androidPkgsFor` now, | 55 | + desktop) just _flutter desktop build ;; |
| 60 | - # so this is an ordinary output at the rev flake.lock pins. | 56 | + web) exec tools/build-web.sh build ;; |
| 61 | - sdk="$(nix build --no-link --print-out-paths \ | 57 | + lib) just _nim-lib ;; |
| 62 | - "{{justfile_directory()}}#android-sdk")/libexec/android-sdk" | 58 | + ui) just _flutter ui build ;; |
| 63 | - | 59 | + app) just _flutter app build ;; |
| 64 | - # adb keeps the key the phone has already trusted under the real HOME, and | 60 | + *) echo "usage: just build [apk|desktop|web|lib|ui|app]" >&2; exit 1 ;; |
| 65 | - # HOME moves below so Gradle can write into the SDK copy. Told where to | ||
| 66 | - # look, adb keeps its identity; left to find $HOME/.android it generates a | ||
| 67 | - # new one, the device stops recognising this machine, and the deploy ends | ||
| 68 | - # in "no devices/emulators found" while `adb devices` in any other shell | ||
| 69 | - # lists it perfectly well. | ||
| 70 | - export ANDROID_USER_HOME="${ANDROID_USER_HOME:-$HOME/.android}" | ||
| 71 | - | ||
| 72 | - export HOME="$PWD/.home" | ||
| 73 | - export ANDROID_HOME="$HOME/android-sdk" | ||
| 74 | - export ANDROID_SDK_ROOT="$ANDROID_HOME" | ||
| 75 | - mkdir -p "$HOME" | ||
| 76 | - | ||
| 77 | - # Gradle writes into ANDROID_HOME, so it is a copy rather than the store | ||
| 78 | - # path. Made once and kept: re-copying would throw away the build-tools | ||
| 79 | - # and platform Gradle installed into it on the last run. | ||
| 80 | - if [ ! -d "$ANDROID_HOME" ]; then | ||
| 81 | - cp -r "$sdk" "$ANDROID_HOME" | ||
| 82 | - chmod -R u+w "$ANDROID_HOME" | ||
| 83 | - fi | ||
| 84 | - | ||
| 85 | - # The compile's three caches, same flake-output-and-copy shape as the SDK | ||
| 86 | - # and for the same reason: tools.deps and pub both write into theirs, and | ||
| 87 | - # the store is read-only. What this buys is the "Resolving dependencies… | ||
| 88 | - # Downloading packages…" that used to open every run. | ||
| 89 | - deps="$(nix build --no-link --print-out-paths \ | ||
| 90 | - "{{justfile_directory()}}#cljd-deps")" | ||
| 91 | - | ||
| 92 | - # Neither of these follows HOME. Both are read off the JVM's user.home, | ||
| 93 | - # which comes from /etc/passwd rather than the environment — so moving | ||
| 94 | - # HOME below is not enough to move them, and left alone they would be the | ||
| 95 | - # real ~/.m2 and ~/.gitlibs, shared with every other project on the box. | ||
| 96 | - export GITLIBS="$HOME/gitlibs" | ||
| 97 | - m2="$HOME/m2" | ||
| 98 | - export PUB_CACHE="$HOME/.pub-cache" | ||
| 99 | - | ||
| 100 | - # Seeded once each and then left alone, exactly as ANDROID_HOME is: after | ||
| 101 | - # the first compile these hold whatever the working tree has asked for | ||
| 102 | - # since, and re-copying would throw that away. | ||
| 103 | - seed() { | ||
| 104 | - [ -e "$2" ] && return 0 | ||
| 105 | - mkdir -p "$(dirname "$2")" | ||
| 106 | - cp -r "$deps/$1" "$2" | ||
| 107 | - chmod -R u+w "$2" | ||
| 108 | - } | ||
| 109 | - seed m2 "$m2" | ||
| 110 | - seed gitlibs "$GITLIBS" | ||
| 111 | - seed pub-cache "$PUB_CACHE" | ||
| 112 | - seed clojuredart/cache "$PWD/.clojuredart/cache" | ||
| 113 | - | ||
| 114 | - # Also the flake's. `nix shell nixpkgs#...` read the registry, which is a | ||
| 115 | - # different and unlocked nixpkgs — the Flutter that built the APK could | ||
| 116 | - # move under it without flake.lock changing a line. | ||
| 117 | - flutter="nix develop {{justfile_directory()}}#flutter --command" | ||
| 118 | - | ||
| 119 | - # cljd-deps ships the analyzer project unresolved — a fixed-output | ||
| 120 | - # derivation may not name the store, and a resolved pub project is | ||
| 121 | - # nothing but store paths. So it is resolved here instead, offline, | ||
| 122 | - # against the cache that derivation did fetch. ClojureDart reaches for the | ||
| 123 | - # network only when bin/analyzer.dart is missing, and after this it is not. | ||
| 124 | - for helper in .clojuredart/cache/*/cljd_helper; do | ||
| 125 | - [ -d "$helper" ] || continue | ||
| 126 | - [ -e "$helper/.dart_tool/package_config.json" ] && continue | ||
| 127 | - ( cd "$helper" && $flutter flutter pub get --offline ) | ||
| 128 | - done | ||
| 129 | - | ||
| 130 | - $flutter clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile | ||
| 131 | - | ||
| 132 | - # Rewritten every run: it carries absolute store paths, and the flutter | ||
| 133 | - # one moves whenever nixpkgs does. | ||
| 134 | - $flutter flutter config --android-sdk "$ANDROID_HOME" >/dev/null | ||
| 135 | - | ||
| 136 | - apk=build/app/outputs/flutter-apk/app-debug.apk | ||
| 137 | - adb="${ADB:-$ANDROID_HOME/platform-tools/adb}" | ||
| 138 | - | ||
| 139 | - case "{{action}}" in | ||
| 140 | - build) $flutter flutter build apk --debug ;; | ||
| 141 | - install) $flutter flutter build apk --debug && "$adb" install -r "$apk" ;; | ||
| 142 | - run) $flutter flutter build apk --debug && "$adb" install -r "$apk" \ | ||
| 143 | - && "$adb" shell monkey -p uk.nandi.frq -c android.intent.category.LAUNCHER 1 ;; | ||
| 144 | - log) "$adb" logcat -s flutter ;; | ||
| 145 | - *) echo "usage: just apk [build|install|run|log]" >&2; exit 1 ;; | ||
| 146 | esac | 61 | esac |
| 147 | 62 | ||
| 148 | -# What may appear in common/, checked. Needs nothing built: it reads the | 63 | +# Build a target and start it. |
| 149 | -# source, so it is the one check that runs anywhere, and CI runs exactly this. | ||
| 150 | -check-common: | ||
| 151 | - #!/usr/bin/env bash | ||
| 152 | - python3 tools/check-common.py common | ||
| 153 | - | ||
| 154 | -# The desktop GUI: the same screens the APK paints, on Flutter's Linux target. | ||
| 155 | -# | ||
| 156 | -# This recipe and `just apk` are two targets over one tree, and the split is | ||
| 157 | -# the one the APK already draws. Everything under `common/` — the screens, the | ||
| 158 | -# cells, `frq.io` — is shared; what differs is who answers the host. So this | ||
| 159 | -# is `just apk` with the Android half taken out: the same `clojure -M:cljd compile` over the same flutter/src, | ||
| 160 | -# then Flutter's Linux target rather than its Android one. CMake and Ninja | ||
| 161 | -# instead of Gradle, `flutter/linux/` as the runner, no SDK and no JDK. | ||
| 162 | # | 64 | # |
| 163 | -# Still impure, for one of the two reasons `apk` is: pub.dev resolution and | 65 | +# apk install on a connected device and launch it |
| 164 | -# Flutter's own engine artifacts are network. What it does NOT need is the | 66 | +# web serve build/web on ARG (default 8080) |
| 165 | -# writable-ANDROID_HOME dance — nothing here writes into the store — so there | 67 | +# the rest open the window |
| 166 | -# is no `flutter/.home` on this path. | ||
| 167 | # | 68 | # |
| 168 | -# nixGL because Flutter paints through GL, and off NixOS the driver is the | 69 | +# just run apk |
| 169 | -# host's. | 70 | +# just run web 3000 |
| 170 | -# | 71 | +[doc('build a target and start it')] |
| 171 | -# just flutter-desktop build the debug bundle | 72 | +run target="desktop" arg="": |
| 172 | -# just flutter-desktop run build it and open the window | ||
| 173 | -flutter-desktop action="build": | ||
| 174 | #!/usr/bin/env bash | 73 | #!/usr/bin/env bash |
| 175 | set -euo pipefail | 74 | set -euo pipefail |
| 176 | - cd "{{justfile_directory()}}" | 75 | + cd "{{root}}" |
| 177 | - if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then | 76 | + case "{{target}}" in |
| 178 | - exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \ | 77 | + apk) just _flutter apk run ;; |
| 179 | - --command just flutter-desktop "$@" | 78 | + desktop) just _flutter desktop run ;; |
| 180 | - fi | 79 | + web) port="${2:-}"; exec tools/build-web.sh serve "${port:-8080}" ;; |
| 181 | - cd flutter | 80 | + ui) just _flutter ui run ;; |
| 182 | - | 81 | + app) just _flutter app run ;; |
| 183 | - # The same three caches `just apk` seeds, in the same place and out of the | 82 | + log) just tools exec -- adb logcat -s flutter ;; |
| 184 | - # same flake output — one compiler, one set of dependencies, and no reason | 83 | + *) echo "usage: just run [apk|desktop|web|ui|app|log]" >&2; exit 1 ;; |
| 185 | - # for the two frontends to keep a copy each. `.home/` is `apk`'s directory | ||
| 186 | - # by name and this is the only thing put there from here, which is the | ||
| 187 | - # point: whichever recipe runs first pays for the copy and the other finds | ||
| 188 | - # it warm. | ||
| 189 | - # | ||
| 190 | - # No `nix build` here, unlike `apk`: this recipe is already inside the | ||
| 191 | - # shell that names FRQ_CLJD_DEPS by the time it gets this far, and `apk` | ||
| 192 | - # needs the path before it enters anything. | ||
| 193 | - # | ||
| 194 | - # m2 and gitlibs are set here for the reason they are set there — the JVM | ||
| 195 | - # reads user.home out of /etc/passwd, so neither follows HOME and left | ||
| 196 | - # alone they are the real ~/.m2 and ~/.gitlibs. | ||
| 197 | - export PUB_CACHE="$PWD/.home/.pub-cache" | ||
| 198 | - export GITLIBS="$PWD/.home/gitlibs" | ||
| 199 | - m2="$PWD/.home/m2" | ||
| 200 | - | ||
| 201 | - seed() { | ||
| 202 | - [ -e "$2" ] && return 0 | ||
| 203 | - mkdir -p "$(dirname "$2")" | ||
| 204 | - cp -r "$FRQ_CLJD_DEPS/$1" "$2" | ||
| 205 | - chmod -R u+w "$2" | ||
| 206 | - } | ||
| 207 | - seed m2 "$m2" | ||
| 208 | - seed gitlibs "$GITLIBS" | ||
| 209 | - seed pub-cache "$PUB_CACHE" | ||
| 210 | - seed clojuredart/cache "$PWD/.clojuredart/cache" | ||
| 211 | - | ||
| 212 | - # Resolved here rather than in cljd-deps, which was not allowed to name | ||
| 213 | - # the store — see the same loop in `apk`. | ||
| 214 | - for helper in .clojuredart/cache/*/cljd_helper; do | ||
| 215 | - [ -d "$helper" ] || continue | ||
| 216 | - [ -e "$helper/.dart_tool/package_config.json" ] && continue | ||
| 217 | - ( cd "$helper" && flutter pub get --offline ) | ||
| 218 | - done | ||
| 219 | - | ||
| 220 | - clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile | ||
| 221 | - flutter build linux --debug | ||
| 222 | - | ||
| 223 | - # x64/arm64 is Flutter's own name for the host arch, not uname's. | ||
| 224 | - case "$(uname -m)" in | ||
| 225 | - x86_64) arch=x64 ;; | ||
| 226 | - aarch64) arch=arm64 ;; | ||
| 227 | - *) echo "unknown arch $(uname -m)" >&2; exit 1 ;; | ||
| 228 | - esac | ||
| 229 | - bundle="build/linux/$arch/debug/bundle" | ||
| 230 | - | ||
| 231 | - case "{{action}}" in | ||
| 232 | - build) echo "built $PWD/$bundle/frq" ;; | ||
| 233 | - run) | ||
| 234 | - runner=() | ||
| 235 | - [ -e /run/current-system ] || runner=("$NIXGL") | ||
| 236 | - exec "${runner[@]}" "$bundle/frq" | ||
| 237 | - ;; | ||
| 238 | - *) echo "usage: just flutter-desktop [build|run]" >&2; exit 1 ;; | ||
| 239 | esac | 84 | esac |
| 240 | 85 | ||
| 241 | -# The third frontend: the same screens again, compiled to JavaScript. | 86 | +# Test a suite. |
| 242 | -# | 87 | +# |
| 243 | -# `flutter-desktop` with the Linux half taken out. One `clojure -M:cljd | 88 | +# common what may appear in common/, read off the source. Needs no |
| 244 | -# compile` over the same flutter/src and common/, then Flutter's web target | 89 | +# toolchain at all, which is why CI runs exactly this. |
| 245 | -# instead of its Linux one — dart2js instead of CMake and Ninja, and a | 90 | +# nim the Nim core. No Flutter, no Dart, no SDK — a rule about the IRC |
| 246 | -# directory of static files instead of a bundle with an executable in it. | 91 | +# wire format is checkable in a second. |
| 247 | -# | 92 | +# dart the Dart side of the FFI boundary, on the plain Dart VM. Passing |
| 248 | -# And the one target with no nix in it. The other two need the host: a JDK | 93 | +# `nim` and failing this one is a marshalling bug, which is why the |
| 249 | -# and the Android SDK for `apk`, GTK and a C++ toolchain and nixGL for | 94 | +# two are separate suites. |
| 250 | -# `flutter-desktop`. This one needs a Dart, a JVM and a browser, and the | 95 | +# live the whole stack against a real freeq. Not in `all`: it wants a |
| 251 | -# browser is not ours — so `tools/toolchain.sh` fetches the first two as | 96 | +# network and a running server. |
| 252 | -# pinned tarballs into `.toolchain/` and there is nothing left for a devShell | 97 | +# |
| 253 | -# to supply. That is what lets the container in `.modal/flutter-web/` drop | 98 | +# just test all of them |
| 254 | -# its image build too: same script, same three pins, no store to populate. | 99 | +# just test nim tircparse one Nim file |
| 255 | -# | 100 | +[doc('run a suite: all common nim dart live')] |
| 256 | -# Impure for the reason the other two are: pub.dev resolution and Flutter's | 101 | +test suite="all" *args: |
| 257 | -# engine artifacts are network, and now the toolchain is as well — pinned by | ||
| 258 | -# sha256, which is the reproducibility that was worth having out of the store. | ||
| 259 | -# | ||
| 260 | -# The entry point is `frq.main-web`, not `frq.main`: path_provider has no web | ||
| 261 | -# implementation, so the `getApplicationSupportDirectory` that `frq.main` | ||
| 262 | -# awaits throws MissingPluginException before any widget is built. The web | ||
| 263 | -# entry installs `frq.io.web` — localStorage behind the same seam — and awaits | ||
| 264 | -# nothing. `frq.net.dart` is still the socket half, so connecting will want a | ||
| 265 | -# WebSocket before this does more than paint. | ||
| 266 | -# | ||
| 267 | -# One build and no `--debug` variant, because there is nothing to gain from | ||
| 268 | -# one: dart2js at -O1 measured 52.5s against the release build's 49.8s on the | ||
| 269 | -# same source change here, so a second, larger bundle would buy noise. See | ||
| 270 | -# `tools/build-web.sh`, which writes the numbers down. | ||
| 271 | -# | ||
| 272 | -# just flutter-web build build/web | ||
| 273 | -# just flutter-web serve build it and serve it on $PORT (8080) | ||
| 274 | -# just flutter-web serve 3000 ...on another port | ||
| 275 | -flutter-web action="build" port="8080": | ||
| 276 | #!/usr/bin/env bash | 102 | #!/usr/bin/env bash |
| 277 | set -euo pipefail | 103 | set -euo pipefail |
| 278 | - # A wrapper and nothing else. The build is a shell script because the | 104 | + cd "{{root}}" |
| 279 | - # container runs it too, and a container that had to install `just` to | 105 | + shift |
| 280 | - # start would be one dependency away from the point. | 106 | + case "{{suite}}" in |
| 281 | - exec "{{justfile_directory()}}/tools/build-web.sh" {{action}} {{port}} | 107 | + all) just test common && just test nim && just test dart ;; |
| 108 | + common) exec python3 tools/check-common.py common ;; | ||
| 109 | + nim) just _nim-test "$@" ;; | ||
| 110 | + dart) just _nim-lib | ||
| 111 | + exec "{{tc}}" exec -- bash -c \ | ||
| 112 | + 'cd dart/frq_core && dart pub get && dart test -r expanded' ;; | ||
| 113 | + live) just _nim-lib | ||
| 114 | + exec "{{tc}}" exec -- bash -c \ | ||
| 115 | + 'cd dart/frq_core && dart pub get >/dev/null && dart run tool/live_ui.dart "$@"' _ "$@" ;; | ||
| 116 | + *) echo "usage: just test [all|common|nim|dart|live]" >&2; exit 1 ;; | ||
| 117 | + esac | ||
| 282 | 118 | ||
| 283 | -# The containers in `.modal/`, run on Modal rather than here. This machine | 119 | +# The containers in `.modal/`, run on Modal rather than here: this machine |
| 284 | -# evaluates and Modal builds — see CLAUDE.md, which says so rather more | 120 | +# evaluates and Modal builds. See CLAUDE.md, which says so rather more firmly. |
| 285 | -# firmly — and these two recipes are the whole interface to that. | ||
| 286 | # | 121 | # |
| 287 | -# Named for where the work happens, the way `flutter-desktop` is named for | 122 | +# `--shell` leaves a sandbox running with the container's own image, volumes |
| 288 | -# what paints: there is no re-entry test here because nothing | 123 | +# and environment, and prints the command to attach to it. It blocks — an |
| 289 | -# re-enters. There is no `nix` variable either, and that used to be because | 124 | +# ephemeral app takes its sandbox down when the entrypoint returns — so attach |
| 290 | -# nix ran out there — now it is because neither of these containers has any | 125 | +# from a second terminal and Ctrl-C here when done. The sandbox bills until |
| 291 | -# nix in it at all. | 126 | +# you do. |
| 292 | # | 127 | # |
| 293 | -# just modal flutter-dev the incremental Flutter loop | 128 | +# just modal dev |
| 294 | -# just modal flutter-web the web bundle | 129 | +# just modal web --shell |
| 295 | -modal container="flutter-dev" *args: | 130 | +[doc('run a .modal/ container on Modal')] |
| 131 | +modal container="dev" *args: | ||
| 296 | #!/usr/bin/env bash | 132 | #!/usr/bin/env bash |
| 297 | set -euo pipefail | 133 | set -euo pipefail |
| 298 | - cd "{{justfile_directory()}}" | 134 | + cd "{{root}}" |
| 299 | - shift | 135 | + shift || true |
| 300 | exec modal run ".modal/{{container}}/container.py" "$@" | 136 | exec modal run ".modal/{{container}}/container.py" "$@" |
| 301 | 137 | ||
| 302 | # The Modal-built web bundle, served from this machine on localhost. | 138 | # The Modal-built web bundle, served from this machine on localhost. |
| 303 | # | 139 | # |
| 304 | -# Not a local build: `modal volume get` pulls what `just modal flutter-web` | 140 | +# Not a local build: `modal volume get` pulls down what `just modal |
| 305 | -# already compiled out of the devshell volume, so this needs no Flutter, no | 141 | +# web` already compiled, so this needs only python3. |
| 306 | -# Dart and no nix — only python, which the flake shell has and so does the | 142 | +# |
| 307 | -# machine. | 143 | +# It exists for the auth broker rather than for convenience. freeq's broker |
| 308 | -# | 144 | +# finishes an OAuth login by redirecting to `return_to`, and only to an origin |
| 309 | -# It exists for one reason, and the reason is the auth broker rather than | 145 | +# on its allowlist: its own https hosts, and http://localhost or |
| 310 | -# convenience. freeq's broker finishes an OAuth login by redirecting the | 146 | +# http://127.0.0.1 on ANY port. Served from anywhere else — the Modal URL |
| 311 | -# browser to `return_to`, and it will only redirect to an origin on its | 147 | +# included — a Bluesky sign-in gets `400 Invalid return_to URL` and can never |
| 312 | -# allowlist: its own https hosts, and `http://localhost` or `http://127.0.0.1` | 148 | +# complete. Guest and app-password sign-in work on the deployed URL; neither |
| 313 | -# on ANY port. A build served from anywhere else — the Modal URL included — | 149 | +# goes near the broker. |
| 314 | -# gets `400 Invalid return_to URL` and can never complete a Bluesky sign-in, | 150 | +[doc('serve the Modal-built web bundle on localhost')] |
| 315 | -# no matter what the client does. localhost is the one allowlisted origin we | 151 | +serve port="8080": |
| 316 | -# can serve from, so this is how Bluesky sign-in is tested. | ||
| 317 | -# | ||
| 318 | -# Guest and app-password sign-in need none of this; they work on the deployed | ||
| 319 | -# URL, because neither goes near the broker. | ||
| 320 | -# | ||
| 321 | -# just web-local fetch and serve on :8080 | ||
| 322 | -# just web-local 3000 another port; any port is allowlisted | ||
| 323 | -web-local port="8080": | ||
| 324 | #!/usr/bin/env bash | 152 | #!/usr/bin/env bash |
| 325 | set -euo pipefail | 153 | set -euo pipefail |
| 326 | - cd "{{justfile_directory()}}" | 154 | + cd "{{root}}" |
| 327 | - out="{{justfile_directory()}}/.web-local" | 155 | + out="{{root}}/.web-local" |
| 328 | mkdir -p "$out" | 156 | mkdir -p "$out" |
| 329 | echo "fetching the Modal-built bundle…" | 157 | echo "fetching the Modal-built bundle…" |
| 330 | # --force: this is a mirror of the volume, and a stale file left behind | 158 | # --force: this is a mirror of the volume, and a stale file left behind |
| 331 | # would be served in preference to the one just built. | 159 | # would be served in preference to the one just built. |
| 332 | - modal volume get --force devshell frq-flutter-web/flutter/build/web "$out" | 160 | + modal volume get --force devshell frq-web/flutter/build/web "$out" |
| 333 | echo | 161 | echo |
| 334 | echo " http://localhost:{{port}}" | 162 | echo " http://localhost:{{port}}" |
| 335 | echo | 163 | echo |
| 336 | - echo "Bluesky sign-in works here and not on the Modal URL: the broker" | 164 | + echo "Bluesky sign-in works here and not on the Modal URL. Put" |
| 337 | - echo "allowlists localhost on any port. Put wss://irc.freeq.at/irc in the" | 165 | + echo "wss://irc.freeq.at/irc in the Server field — a browser has no TCP." |
| 338 | - echo "Server field — a browser has no TCP." | ||
| 339 | exec python3 -m http.server {{port}} --bind 127.0.0.1 --directory "$out/web" | 166 | exec python3 -m http.server {{port}} --bind 127.0.0.1 --directory "$out/web" |
| 340 | 167 | ||
| 341 | -# A sandbox left running with the container's own image, volumes and | 168 | +# --- the work behind the verbs ------------------------------------------ |
| 342 | -# environment, and the command to get into it. `modal shell --image` cannot | ||
| 343 | -# be pointed at a published Modal image like arch-nix, so attaching to a | ||
| 344 | -# running sandbox is the only way to get a shell that is the container. | ||
| 345 | -# | ||
| 346 | -# It blocks: an ephemeral app stops when its entrypoint returns and takes the | ||
| 347 | -# sandbox with it. Attach from a second terminal, and Ctrl-C here when done — | ||
| 348 | -# the sandbox bills until you do. | ||
| 349 | -# | ||
| 350 | -# just modal-shell flutter-dev, the usual one | ||
| 351 | -# just modal-shell flutter-web the web bundle container | ||
| 352 | -modal-shell container="flutter-dev": | ||
| 353 | - #!/usr/bin/env bash | ||
| 354 | - set -euo pipefail | ||
| 355 | - cd "{{justfile_directory()}}" | ||
| 356 | - exec modal run ".modal/{{container}}/container.py" --shell | ||
| 357 | - | ||
| 358 | -# The Nim core's test suite. | ||
| 359 | -# | ||
| 360 | -# It needs no Flutter, no Dart and no Android SDK — which is the point of | ||
| 361 | -# having the logic here rather than under `common/`: a rule about the IRC wire | ||
| 362 | -# format can be checked in a second, on any machine, without a toolchain that | ||
| 363 | -# takes minutes to enter. | ||
| 364 | -# | ||
| 365 | -# just nim-test the whole suite | ||
| 366 | -# just nim-test tircparse one file | ||
| 367 | -nim-test file="": | ||
| 368 | - #!/usr/bin/env bash | ||
| 369 | - set -euo pipefail | ||
| 370 | - cd "{{justfile_directory()}}" | ||
| 371 | - if [ -z "${FRQ_NIM:-}" ]; then | ||
| 372 | - exec {{nix}} develop .#nim --max-jobs {{jobs}} --command just nim-test "$@" | ||
| 373 | - fi | ||
| 374 | - cd nim | ||
| 375 | - if [ -n "{{file}}" ]; then | ||
| 376 | - exec nim c -r --hints:off --path:src "tests/{{file}}.nim" | ||
| 377 | - fi | ||
| 378 | - for t in tests/t*.nim; do | ||
| 379 | - echo "== $t" | ||
| 380 | - nim c -r --hints:off --path:src "$t" | ||
| 381 | - done | ||
| 382 | 169 | ||
| 383 | # The Nim core as a shared library, into build/nim. | 170 | # The Nim core as a shared library, into build/nim. |
| 384 | # | 171 | # |
| @@ -387,117 +174,88 @@ nim-test file="": | |||
| 387 | # from another heap and dereferencing it segfaults. ORC's heap is shared. | 174 | # from another heap and dereferencing it segfaults. ORC's heap is shared. |
| 388 | # | 175 | # |
| 389 | # `-d:release` and not `-d:danger`: the bounds checks are what turn a | 176 | # `-d:release` and not `-d:danger`: the bounds checks are what turn a |
| 390 | -# malformed line off a socket into an exception instead of a read past the end | 177 | +# malformed line off a socket into an exception rather than a read past the |
| 391 | -# of a buffer, and this parses exactly that. | 178 | +# end of a buffer, and this parses exactly that. |
| 392 | -nim-lib: | 179 | +[private] |
| 180 | +_nim-lib: | ||
| 393 | #!/usr/bin/env bash | 181 | #!/usr/bin/env bash |
| 394 | set -euo pipefail | 182 | set -euo pipefail |
| 395 | - cd "{{justfile_directory()}}" | 183 | + cd "{{root}}" |
| 396 | - if [ -z "${FRQ_NIM:-}" ]; then | 184 | + out="{{root}}/build/nim" |
| 397 | - exec {{nix}} develop .#nim --max-jobs {{jobs}} --command just nim-lib | ||
| 398 | - fi | ||
| 399 | - out="{{justfile_directory()}}/build/nim" | ||
| 400 | mkdir -p "$out" | 185 | mkdir -p "$out" |
| 401 | - cd nim | 186 | + exec "{{tc}}" exec -- bash -euo pipefail -c ' |
| 402 | - nim c --app:lib --mm:orc -d:release --hints:off \ | 187 | + cd nim |
| 403 | - --path:src --out:"$out/libfrqcore.so" src/frq_core.nim | 188 | + nim c --app:lib --mm:orc -d:release --hints:off \ |
| 404 | - echo "built $out/libfrqcore.so" | 189 | + --path:src --out:"'"$out"'/libfrqcore.so" src/frq_core.nim |
| 405 | - nm -D --defined-only "$out/libfrqcore.so" | grep ' T frq_' || true | 190 | + echo "built '"$out"'/libfrqcore.so" |
| 406 | - | 191 | + nm -D --defined-only "'"$out"'/libfrqcore.so" | grep " T frq_" || true' |
| 407 | -# The Dart side of the Nim boundary, on the plain Dart VM. | 192 | + |
| 408 | -# | 193 | +[private] |
| 409 | -# No Flutter, no emulator, no ClojureDart — `dart/frq_core` is ordinary Dart | 194 | +_nim-test *args: |
| 410 | -# over `dart:ffi` and is not a Flutter package, so the test that proves the | ||
| 411 | -# marshalling runs in a second. Passing `just nim-test` and failing this one is a | ||
| 412 | -# marshalling bug, which is the whole reason the two suites are separate. | ||
| 413 | -# | ||
| 414 | -# Builds the library first: the test dlopens a real .so and there is no point | ||
| 415 | -# reporting that it could not find one. | ||
| 416 | -dart-test: | ||
| 417 | #!/usr/bin/env bash | 195 | #!/usr/bin/env bash |
| 418 | set -euo pipefail | 196 | set -euo pipefail |
| 419 | - cd "{{justfile_directory()}}" | 197 | + cd "{{root}}" |
| 420 | - if [ -z "${FRQ_DART:-}" ]; then | 198 | + exec "{{tc}}" exec -- bash -euo pipefail -c ' |
| 421 | - # nim-lib before the re-entry, not after: it enters a shell of its own | 199 | + cd nim |
| 422 | - # and doing it on the far side would build the library twice. | 200 | + if [ -n "${1:-}" ]; then |
| 423 | - just nim-lib | 201 | + exec nim c -r --hints:off --path:src "tests/$1.nim" |
| 424 | - exec {{nix}} develop .#dart --max-jobs {{jobs}} --command just dart-test | 202 | + fi |
| 425 | - fi | 203 | + for t in tests/t*.nim; do |
| 426 | - cd dart/frq_core | 204 | + echo "== $t" |
| 427 | - dart pub get | 205 | + nim c -r --hints:off --path:src "$t" |
| 428 | - dart test -r expanded | 206 | + done' _ "$@" |
| 429 | - | 207 | + |
| 430 | -# The whole Nim stack against a real freeq: socket, state and screens. | 208 | +# The three Flutter targets, which differ only in what they compile and which |
| 431 | -# | 209 | +# entry point they paint. |
| 432 | -# Connects, waits for the room list, opens a room and prints what the tree | 210 | +# |
| 433 | -# actually contains — through the FFI, so it is the path the window uses. | 211 | +# apk ClojureDart, then Gradle. Impure on purpose: Gradle resolves its |
| 434 | -# Not in any suite: it needs a network and a running freeq. | 212 | +# own dependencies over the network and has sdkmanager install a |
| 435 | -nim-live *args: | 213 | +# platform and build-tools into ANDROID_HOME as it goes, so the |
| 436 | - #!/usr/bin/env bash | 214 | +# SDK has to be writable — which is what `just tools android` gets |
| 437 | - set -euo pipefail | 215 | +# it. Everything it leaves behind is under `.toolchain/` and |
| 438 | - cd "{{justfile_directory()}}" | 216 | +# gitignored. |
| 439 | - if [ -z "${FRQ_DART:-}" ]; then | 217 | +# desktop the same `clojure -M:cljd compile`, Flutter's Linux target. |
| 440 | - just nim-lib | 218 | +# ui no ClojureDart at all: `lib/main_nim.dart` asks the Nim core for |
| 441 | - exec {{nix}} develop .#dart --max-jobs {{jobs}} --command just nim-live "$@" | 219 | +# a widget tree and paints it. |
| 442 | - fi | 220 | +# app `frq.main-nim` is `frq.main` with one line changed — |
| 443 | - shift || true | 221 | +# `frq.net.nim/install!` where it said `frq.net.dart/install!`. |
| 444 | - cd dart/frq_core | 222 | +# Every screen, cell and action is the one that was already there. |
| 445 | - dart pub get >/dev/null | 223 | +[private] |
| 446 | - exec dart run tool/live_ui.dart "$@" | 224 | +_flutter target action: |
| 447 | - | ||
| 448 | -# frq with Nim owning the state and the screens, rendered by Flutter. | ||
| 449 | -# | ||
| 450 | -# No ClojureDart on this path. `lib/main_nim.dart` asks the Nim core for a | ||
| 451 | -# widget tree and paints it; the screens are ports of `common/frq/screens/`. | ||
| 452 | -# | ||
| 453 | -# just nim-ui build it | ||
| 454 | -# just nim-ui run open the window | ||
| 455 | -nim-ui action="build": | ||
| 456 | - #!/usr/bin/env bash | ||
| 457 | - set -euo pipefail | ||
| 458 | - cd "{{justfile_directory()}}" | ||
| 459 | - if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then | ||
| 460 | - just nim-lib | ||
| 461 | - exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \ | ||
| 462 | - --command just nim-ui "$@" | ||
| 463 | - fi | ||
| 464 | - cd flutter | ||
| 465 | - flutter pub get | ||
| 466 | - export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" | ||
| 467 | - runner=() | ||
| 468 | - [ -e /run/current-system ] || runner=("$NIXGL") | ||
| 469 | - case "{{action}}" in | ||
| 470 | - build) exec "${runner[@]}" flutter build linux --debug -t lib/main_nim.dart ;; | ||
| 471 | - run) exec "${runner[@]}" flutter run -d linux -t lib/main_nim.dart ;; | ||
| 472 | - *) echo "usage: just nim-ui [build|run]" >&2; exit 1 ;; | ||
| 473 | - esac | ||
| 474 | - | ||
| 475 | -# The ClojureDart app, with the Nim core as its transport only. | ||
| 476 | -# | ||
| 477 | -# This is the wiring that matters: `frq.main-nim` is `frq.main` with one line | ||
| 478 | -# changed — `frq.net.nim/install!` where it says `frq.net.dart/install!`. Every | ||
| 479 | -# screen, every cell and every action is the one that was already there. Nim | ||
| 480 | -# owns the socket, the TLS and the line framing, and nothing else. | ||
| 481 | -# | ||
| 482 | -# just nim-app build it | ||
| 483 | -# just nim-app run open the window | ||
| 484 | -nim-app action="build": | ||
| 485 | #!/usr/bin/env bash | 225 | #!/usr/bin/env bash |
| 486 | set -euo pipefail | 226 | set -euo pipefail |
| 487 | - cd "{{justfile_directory()}}" | 227 | + cd "{{root}}" |
| 488 | - if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then | 228 | + case "{{target}}" in |
| 489 | - just nim-lib | 229 | + apk) "{{tc}}" android ;; |
| 490 | - exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \ | 230 | + ui|app) just _nim-lib ;; |
| 491 | - --command just nim-app "$@" | ||
| 492 | - fi | ||
| 493 | - cd flutter | ||
| 494 | - flutter pub get | ||
| 495 | - export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" | ||
| 496 | - clojure -M:cljd compile frq.main-nim | ||
| 497 | - runner=() | ||
| 498 | - [ -e /run/current-system ] || runner=("$NIXGL") | ||
| 499 | - case "{{action}}" in | ||
| 500 | - build) exec "${runner[@]}" flutter build linux --debug -t lib/main_nim_app.dart ;; | ||
| 501 | - run) exec "${runner[@]}" flutter run -d linux -t lib/main_nim_app.dart ;; | ||
| 502 | - *) echo "usage: just nim-app [build|run]" >&2; exit 1 ;; | ||
| 503 | esac | 231 | esac |
| 232 | + exec "{{tc}}" exec -- bash -euo pipefail -c ' | ||
| 233 | + cd flutter | ||
| 234 | + # Nim resolves OpenSSL through dynlib at run time; without the host | ||
| 235 | + # library on the loader path `newContext` dies in a SIGSEGV that says | ||
| 236 | + # nothing about SSL. | ||
| 237 | + export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" | ||
| 238 | + cljd() { clojure -Sdeps "{:mvn/local-repo \"$FRQ_M2\"}" -M:cljd compile "$@"; } | ||
| 239 | + | ||
| 240 | + case "$1:$2" in | ||
| 241 | + apk:*) | ||
| 242 | + cljd | ||
| 243 | + # Rewritten every run: it carries absolute paths. | ||
| 244 | + flutter config --android-sdk "$ANDROID_HOME" >/dev/null | ||
| 245 | + flutter build apk --debug | ||
| 246 | + apk=build/app/outputs/flutter-apk/app-debug.apk | ||
| 247 | + [ "$2" = run ] || exit 0 | ||
| 248 | + adb install -r "$apk" | ||
| 249 | + exec adb shell monkey -p uk.nandi.frq \ | ||
| 250 | + -c android.intent.category.LAUNCHER 1 ;; | ||
| 251 | + desktop:build) cljd; exec flutter build linux --debug ;; | ||
| 252 | + desktop:run) cljd; exec flutter run -d linux ;; | ||
| 253 | + ui:build) flutter pub get | ||
| 254 | + exec flutter build linux --debug -t lib/main_nim.dart ;; | ||
| 255 | + ui:run) flutter pub get | ||
| 256 | + exec flutter run -d linux -t lib/main_nim.dart ;; | ||
| 257 | + app:build) flutter pub get; cljd frq.main-nim | ||
| 258 | + exec flutter build linux --debug -t lib/main_nim_app.dart ;; | ||
| 259 | + app:run) flutter pub get; cljd frq.main-nim | ||
| 260 | + exec flutter run -d linux -t lib/main_nim_app.dart ;; | ||
| 261 | + esac' _ "{{target}}" "{{action}}" | ||
modified
nim/README.md +20 -19 | @@ -21,7 +21,7 @@ is proven against it. Nothing is deleted on faith. | ||
| 21 | 21 | ``` |
| 22 | 22 | src/frq_core.nim the C ABI: every exported symbol, and nothing else |
| 23 | 23 | src/frq/ircparse.nim the IRC wire format |
| 24 | -tests/ one per module, run by `just nim-test` | |
| 24 | +tests/ one per module, run by `just test nim` | |
| 25 | 25 | ``` |
| 26 | 26 | |
| 27 | 27 | `src/frq_core.nim` is the only file that knows about C. Everything under |
| @@ -57,7 +57,7 @@ deleted as each module lands: they are the web's implementation, not dead code. | ||
| 57 | 57 | |
| 58 | 58 | ## What is wired up |
| 59 | 59 | |
| 60 | -`just nim-app run` is the real client with the Nim core as its transport. Every | |
| 60 | +`just run app` is the real client with the Nim core as its transport. Every | |
| 61 | 61 | screen, cell and action is the one that was already there; `frq.main-nim` is |
| 62 | 62 | `frq.main` with one line changed. |
| 63 | 63 | |
| @@ -88,29 +88,30 @@ replies, no images — and the path forward from it was rewriting every screen | ||
| 88 | 88 | in Nim and losing all of that. It is at 1d62d1a if it is ever wanted. |
| 89 | 89 | |
| 90 | 90 | ```bash |
| 91 | -just nim-app run # the real client, Nim transport | |
| 92 | -just nim-test # the Nim suite | |
| 93 | -just dart-test # the Dart side of the boundary | |
| 94 | -just nim-lib # libfrqcore.so into build/nim | |
| 91 | +just run app # the real client, Nim transport | |
| 92 | +just test nim # the Nim suite | |
| 93 | +just test dart # the Dart side of the boundary | |
| 94 | +just build lib # libfrqcore.so into build/nim | |
| 95 | 95 | |
| 96 | -FRQ_TRACE=1 just nim-app run # every line in and out, both languages | |
| 96 | +FRQ_TRACE=1 just run app # every line in and out, both languages | |
| 97 | 97 | ``` |
| 98 | 98 | |
| 99 | -The GUI needs OpenSSL on its loader path, which the `flutter-desktop` shell | |
| 100 | -provides as `FRQ_OPENSSL_LIB` and the `nim-app` recipe prepends for the app | |
| 101 | -alone. Not set as `LD_LIBRARY_PATH` in the shell itself: that shell also runs | |
| 102 | -Flutter through nixGL, which does its own careful things to the loader path. | |
| 99 | +The GUI needs OpenSSL on its loader path: Nim resolves the entry points | |
| 100 | +through dynlib at run time, and without the library there `newContext` dies in | |
| 101 | +a SIGSEGV that says nothing about SSL. The recipe prepends `FRQ_OPENSSL_LIB` | |
| 102 | +for the app alone, so a host whose libssl is somewhere unusual has one variable | |
| 103 | +to set rather than an `LD_LIBRARY_PATH` to inherit. | |
| 103 | 104 | |
| 104 | 105 | ## Status |
| 105 | 106 | |
| 106 | -`frq/ircparse.nim` is ported and tested — 29 cases, `just nim-test` — and the | |
| 107 | +`frq/ircparse.nim` is ported and tested — 29 cases, `just test nim` — and the | |
| 107 | 108 | ABI is exercised from C through `dlopen`, including the allocation contract |
| 108 | 109 | under a hundred thousand parse/free cycles. That half is real. |
| 109 | 110 | |
| 110 | 111 | The Dart binding is real too, and is `dart/frq_core` — **plain Dart, not |
| 111 | 112 | ClojureDart**. It calls the library over `dart:ffi` and is covered by 20 tests |
| 112 | 113 | on the Dart VM, including the UTF-8 round trip and ten thousand calls against |
| 113 | -the ownership rules. `just dart-test` runs the pair of them in about a second. | |
| 114 | +the ownership rules. `just test dart` runs the pair of them in about a second. | |
| 114 | 115 | |
| 115 | 116 | The binding was ClojureDart for one commit and should not have been: the Nim |
| 116 | 117 | core exists to have less Clojure in the tree, and `lookupFunction` takes two |
| @@ -123,8 +124,7 @@ the shipping app is unchanged and `frq.main-nim` is a second entry point | ||
| 123 | 124 | beside it. `common/frq/irc/parse.cljc` is still what does the parsing on every |
| 124 | 125 | target, including this one. That step is its |
| 125 | 126 | own piece of work — the Flutter app takes the package as a path dependency |
| 126 | -(which means a `pubspec.lock` regeneration and widening the nix build's source | |
| 127 | -root), the library has to reach each target (`jniLibs` for the APK, beside the | |
| 127 | +(which means a `pubspec.lock` regeneration), the library has to reach each target (`jniLibs` for the APK, beside the | |
| 128 | 128 | executable for the desktop bundle), and only then can a call site choose Nim |
| 129 | 129 | on native and the ClojureDart original on the web. |
| 130 | 130 | |
| @@ -135,9 +135,10 @@ Modules still in `common/` and not yet here: `rooms`, `msgsig`, `crypto`, | ||
| 135 | 135 | ## Building |
| 136 | 136 | |
| 137 | 137 | ```bash |
| 138 | -just nim-test # the Nim test suite | |
| 139 | -just nim-lib # libfrqcore.so into build/nim | |
| 138 | +just test nim # the Nim test suite | |
| 139 | +just build lib # libfrqcore.so into build/nim | |
| 140 | 140 | ``` |
| 141 | 141 | |
| 142 | -Both want `nix develop .#nim`, and re-enter it themselves if they are not | |
| 143 | -already inside. | |
| 142 | +Both run out of `.toolchain/`, which `tools/toolchain.sh` fills on first | |
| 143 | +use. What they want from the host is a C compiler -- `nim c` shells out to one | |
| 144 | +-- and OpenSSL. | |
| @@ -21,7 +21,7 @@ is proven against it. Nothing is deleted on faith. | |||
| 21 | ``` | 21 | ``` |
| 22 | src/frq_core.nim the C ABI: every exported symbol, and nothing else | 22 | src/frq_core.nim the C ABI: every exported symbol, and nothing else |
| 23 | src/frq/ircparse.nim the IRC wire format | 23 | src/frq/ircparse.nim the IRC wire format |
| 24 | -tests/ one per module, run by `just nim-test` | 24 | +tests/ one per module, run by `just test nim` |
| 25 | ``` | 25 | ``` |
| 26 | 26 | ||
| 27 | `src/frq_core.nim` is the only file that knows about C. Everything under | 27 | `src/frq_core.nim` is the only file that knows about C. Everything under |
| @@ -57,7 +57,7 @@ deleted as each module lands: they are the web's implementation, not dead code. | |||
| 57 | 57 | ||
| 58 | ## What is wired up | 58 | ## What is wired up |
| 59 | 59 | ||
| 60 | -`just nim-app run` is the real client with the Nim core as its transport. Every | 60 | +`just run app` is the real client with the Nim core as its transport. Every |
| 61 | screen, cell and action is the one that was already there; `frq.main-nim` is | 61 | screen, cell and action is the one that was already there; `frq.main-nim` is |
| 62 | `frq.main` with one line changed. | 62 | `frq.main` with one line changed. |
| 63 | 63 | ||
| @@ -88,29 +88,30 @@ replies, no images — and the path forward from it was rewriting every screen | |||
| 88 | in Nim and losing all of that. It is at 1d62d1a if it is ever wanted. | 88 | in Nim and losing all of that. It is at 1d62d1a if it is ever wanted. |
| 89 | 89 | ||
| 90 | ```bash | 90 | ```bash |
| 91 | -just nim-app run # the real client, Nim transport | 91 | +just run app # the real client, Nim transport |
| 92 | -just nim-test # the Nim suite | 92 | +just test nim # the Nim suite |
| 93 | -just dart-test # the Dart side of the boundary | 93 | +just test dart # the Dart side of the boundary |
| 94 | -just nim-lib # libfrqcore.so into build/nim | 94 | +just build lib # libfrqcore.so into build/nim |
| 95 | 95 | ||
| 96 | -FRQ_TRACE=1 just nim-app run # every line in and out, both languages | 96 | +FRQ_TRACE=1 just run app # every line in and out, both languages |
| 97 | ``` | 97 | ``` |
| 98 | 98 | ||
| 99 | -The GUI needs OpenSSL on its loader path, which the `flutter-desktop` shell | 99 | +The GUI needs OpenSSL on its loader path: Nim resolves the entry points |
| 100 | -provides as `FRQ_OPENSSL_LIB` and the `nim-app` recipe prepends for the app | 100 | +through dynlib at run time, and without the library there `newContext` dies in |
| 101 | -alone. Not set as `LD_LIBRARY_PATH` in the shell itself: that shell also runs | 101 | +a SIGSEGV that says nothing about SSL. The recipe prepends `FRQ_OPENSSL_LIB` |
| 102 | -Flutter through nixGL, which does its own careful things to the loader path. | 102 | +for the app alone, so a host whose libssl is somewhere unusual has one variable |
| 103 | +to set rather than an `LD_LIBRARY_PATH` to inherit. | ||
| 103 | 104 | ||
| 104 | ## Status | 105 | ## Status |
| 105 | 106 | ||
| 106 | -`frq/ircparse.nim` is ported and tested — 29 cases, `just nim-test` — and the | 107 | +`frq/ircparse.nim` is ported and tested — 29 cases, `just test nim` — and the |
| 107 | ABI is exercised from C through `dlopen`, including the allocation contract | 108 | ABI is exercised from C through `dlopen`, including the allocation contract |
| 108 | under a hundred thousand parse/free cycles. That half is real. | 109 | under a hundred thousand parse/free cycles. That half is real. |
| 109 | 110 | ||
| 110 | The Dart binding is real too, and is `dart/frq_core` — **plain Dart, not | 111 | The Dart binding is real too, and is `dart/frq_core` — **plain Dart, not |
| 111 | ClojureDart**. It calls the library over `dart:ffi` and is covered by 20 tests | 112 | ClojureDart**. It calls the library over `dart:ffi` and is covered by 20 tests |
| 112 | on the Dart VM, including the UTF-8 round trip and ten thousand calls against | 113 | on the Dart VM, including the UTF-8 round trip and ten thousand calls against |
| 113 | -the ownership rules. `just dart-test` runs the pair of them in about a second. | 114 | +the ownership rules. `just test dart` runs the pair of them in about a second. |
| 114 | 115 | ||
| 115 | The binding was ClojureDart for one commit and should not have been: the Nim | 116 | The binding was ClojureDart for one commit and should not have been: the Nim |
| 116 | core exists to have less Clojure in the tree, and `lookupFunction` takes two | 117 | core exists to have less Clojure in the tree, and `lookupFunction` takes two |
| @@ -123,8 +124,7 @@ the shipping app is unchanged and `frq.main-nim` is a second entry point | |||
| 123 | beside it. `common/frq/irc/parse.cljc` is still what does the parsing on every | 124 | beside it. `common/frq/irc/parse.cljc` is still what does the parsing on every |
| 124 | target, including this one. That step is its | 125 | target, including this one. That step is its |
| 125 | own piece of work — the Flutter app takes the package as a path dependency | 126 | own piece of work — the Flutter app takes the package as a path dependency |
| 126 | -(which means a `pubspec.lock` regeneration and widening the nix build's source | 127 | +(which means a `pubspec.lock` regeneration), the library has to reach each target (`jniLibs` for the APK, beside the |
| 127 | -root), the library has to reach each target (`jniLibs` for the APK, beside the | ||
| 128 | executable for the desktop bundle), and only then can a call site choose Nim | 128 | executable for the desktop bundle), and only then can a call site choose Nim |
| 129 | on native and the ClojureDart original on the web. | 129 | on native and the ClojureDart original on the web. |
| 130 | 130 | ||
| @@ -135,9 +135,10 @@ Modules still in `common/` and not yet here: `rooms`, `msgsig`, `crypto`, | |||
| 135 | ## Building | 135 | ## Building |
| 136 | 136 | ||
| 137 | ```bash | 137 | ```bash |
| 138 | -just nim-test # the Nim test suite | 138 | +just test nim # the Nim test suite |
| 139 | -just nim-lib # libfrqcore.so into build/nim | 139 | +just build lib # libfrqcore.so into build/nim |
| 140 | ``` | 140 | ``` |
| 141 | 141 | ||
| 142 | -Both want `nix develop .#nim`, and re-enter it themselves if they are not | 142 | +Both run out of `.toolchain/`, which `tools/toolchain.sh` fills on first |
| 143 | -already inside. | 143 | +use. What they want from the host is a C compiler -- `nim c` shells out to one |
| 144 | +-- and OpenSSL. | ||
modified
tools/build-web.sh +1 -1 | @@ -6,7 +6,7 @@ | ||
| 6 | 6 | # one target that needs nothing from the host — no JDK of the machine's, no |
| 7 | 7 | # GTK, no Android SDK, no nix. `tools/toolchain.sh` fetches the three tarballs |
| 8 | 8 | # it does need, and everything below runs out of `.toolchain/`. The container |
| 9 | -# in `.modal/flutter-web/` runs this same file; `just flutter-web` is a | |
| 9 | +# in `.modal/web/` runs this same file; `just build web` is a | |
| 10 | 10 | # wrapper around it. |
| 11 | 11 | # |
| 12 | 12 | # tools/build-web.sh build build/web |
| @@ -6,7 +6,7 @@ | |||
| 6 | # one target that needs nothing from the host — no JDK of the machine's, no | 6 | # one target that needs nothing from the host — no JDK of the machine's, no |
| 7 | # GTK, no Android SDK, no nix. `tools/toolchain.sh` fetches the three tarballs | 7 | # GTK, no Android SDK, no nix. `tools/toolchain.sh` fetches the three tarballs |
| 8 | # it does need, and everything below runs out of `.toolchain/`. The container | 8 | # it does need, and everything below runs out of `.toolchain/`. The container |
| 9 | -# in `.modal/flutter-web/` runs this same file; `just flutter-web` is a | 9 | +# in `.modal/web/` runs this same file; `just build web` is a |
| 10 | # wrapper around it. | 10 | # wrapper around it. |
| 11 | # | 11 | # |
| 12 | # tools/build-web.sh build build/web | 12 | # tools/build-web.sh build build/web |
modified
tools/toolchain.sh +78 -9 | @@ -1,22 +1,27 @@ | ||
| 1 | 1 | #!/usr/bin/env bash |
| 2 | 2 | # The toolchain the web build needs, fetched by hand. |
| 3 | 3 | # |
| 4 | -# Three tarballs — Flutter (which carries Dart), a JDK, and the Clojure CLI — | |
| 4 | +# Archives — Flutter (which carries Dart), a JDK, the Clojure CLI and Nim — | |
| 5 | 5 | # pinned by version and by sha256, unpacked into `.toolchain/`, and put on a |
| 6 | 6 | # PATH. That is the whole of it. No nix, no image, no devShell: a checkout |
| 7 | -# plus this script is a machine that can build the web bundle, and the same | |
| 8 | -# script is what the Modal container runs. | |
| 7 | +# plus this script is a machine that can build any target here, and the same | |
| 8 | +# script is what the Modal containers run. | |
| 9 | +# | |
| 10 | +# The Android SDK is the one thing not fetched by default, because only | |
| 11 | +# `just build apk` wants it and it is large: `tools/toolchain.sh android` | |
| 12 | +# fetches Google's command-line tools and lets sdkmanager finish the job. | |
| 9 | 13 | # |
| 10 | 14 | # Why not DotSlash, which the rest of the repo uses for its native libraries: |
| 11 | 15 | # DotSlash hands out an *immutable* cached artifact, and Flutter is not one. |
| 12 | 16 | # `flutter build web` downloads its engine artifacts into `bin/cache/` inside |
| 13 | 17 | # its own SDK directory the first time it runs, so the SDK has to be writable |
| 14 | -# — which is the same thing `just apk` learned when it had to copy the | |
| 15 | -# store's Android SDK out to `flutter/.home` before Gradle would touch it. | |
| 18 | +# — which is the same reason the Android SDK below is fetched into a | |
| 19 | +# directory of ours rather than used read-only from anywhere. | |
| 16 | 20 | # A pinned URL and a checked hash give the reproducibility DotSlash is for; |
| 17 | 21 | # the writability is what it cannot give. |
| 18 | 22 | # |
| 19 | 23 | # tools/toolchain.sh fetch whatever is missing |
| 24 | +# tools/toolchain.sh android ...and the Android SDK as well | |
| 20 | 25 | # eval "$(tools/toolchain.sh env)" ...and put it on this shell's PATH |
| 21 | 26 | # tools/toolchain.sh exec -- flutter --version |
| 22 | 27 | # |
| @@ -52,13 +57,34 @@ CLOJURE_VERSION="1.12.6.1673" | ||
| 52 | 57 | CLOJURE_URL="https://github.com/clojure/brew-install/releases/download/${CLOJURE_VERSION}/clojure-tools-${CLOJURE_VERSION}.tar.gz" |
| 53 | 58 | CLOJURE_SHA="fe9194858e75d5af13c2e2aff92d710674d5bc5105f2b42f90a7d94d82ec023c" |
| 54 | 59 | |
| 60 | +# Nim, for `nim/` — the portable core. Upstream's prebuilt linux_x64 tarball, | |
| 61 | +# so there is no bootstrap compile here. The core deliberately has no | |
| 62 | +# dependencies outside Nim's standard library, so the compiler is all of it. | |
| 63 | +# | |
| 64 | +# Two things Nim wants from the host rather than from here: a C compiler, | |
| 65 | +# because `nim c` shells out to one, and OpenSSL, because `-d:ssl` in | |
| 66 | +# nim/nim.cfg makes std/net resolve -lssl and -lcrypto through dynlib at run | |
| 67 | +# time. A missing libssl is a SIGSEGV in `newContext` that says nothing about | |
| 68 | +# SSL, which is why require_host_tools checks for cc up front. | |
| 69 | +NIM_VERSION="2.2.4" | |
| 70 | +NIM_URL="https://nim-lang.org/download/nim-${NIM_VERSION}-linux_x64.tar.xz" | |
| 71 | +NIM_SHA="791802138aaf19c8579232c50b4998ce2ae2928b791127ce5b4ef3c7af53fb46" | |
| 72 | + | |
| 73 | +# Google's command-line tools, which is the smallest thing that can install an | |
| 74 | +# Android SDK. The platform and build-tools are not pinned here: the versions | |
| 75 | +# come from whatever Flutter asks Gradle for, and sdkmanager fetches them into | |
| 76 | +# the same writable directory on first use. `install_android` below. | |
| 77 | +ANDROID_TOOLS_VERSION="11076708" | |
| 78 | +ANDROID_TOOLS_URL="https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_TOOLS_VERSION}_latest.zip" | |
| 79 | +ANDROID_TOOLS_SHA="2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258" | |
| 80 | + | |
| 55 | 81 | # What the host still has to bring. Small, boring, and on every machine and |
| 56 | 82 | # in every base image that is not deliberately empty — but Flutter shells out |
| 57 | 83 | # to `git` on its own SDK and to `unzip` on its downloads, so a missing one |
| 58 | 84 | # fails somewhere far from here with a much worse message than this. |
| 59 | 85 | require_host_tools() { |
| 60 | 86 | local missing=() |
| 61 | - for t in curl tar git unzip; do | |
| 87 | + for t in curl tar git unzip cc; do | |
| 62 | 88 | command -v "$t" >/dev/null 2>&1 || missing+=("$t") |
| 63 | 89 | done |
| 64 | 90 | if [ ${#missing[@]} -gt 0 ]; then |
| @@ -102,6 +128,38 @@ install_clojure() { | ||
| 102 | 128 | chmod +x "$dest/bin/clojure" "$dest/bin/clj" |
| 103 | 129 | } |
| 104 | 130 | |
| 131 | +# The Android SDK, which is a zip rather than a tarball and has to land in a | |
| 132 | +# layout sdkmanager recognises: cmdline-tools/latest/, with the tools' own | |
| 133 | +# top-level directory renamed. Everything after that — platform-tools, the | |
| 134 | +# platform, the build-tools — Gradle asks sdkmanager for as it goes, which is | |
| 135 | +# why this directory is ours and writable rather than a read-only artifact. | |
| 136 | +# | |
| 137 | +# No ndkVersion in android/app/build.gradle.kts, and nothing here installs an | |
| 138 | +# NDK: there is no native code in the app to need one. | |
| 139 | +install_android() { | |
| 140 | + local dest="$TC/android-sdk" | |
| 141 | + local stamp="$dest/.cmdline-tools.sha256" | |
| 142 | + if [ ! -d "$dest/cmdline-tools/latest" ] \ | |
| 143 | + || [ "$(cat "$stamp" 2>/dev/null || true)" != "$ANDROID_TOOLS_SHA" ]; then | |
| 144 | + echo "toolchain: fetching the Android command-line tools" >&2 | |
| 145 | + local dl="$TC/.download.android" | |
| 146 | + rm -rf "$dest/cmdline-tools" "$dl" | |
| 147 | + mkdir -p "$dest/cmdline-tools" | |
| 148 | + curl -fsSL --retry 3 -o "$dl" "$ANDROID_TOOLS_URL" | |
| 149 | + echo "$ANDROID_TOOLS_SHA $dl" | sha256sum -c - >/dev/null | |
| 150 | + unzip -q "$dl" -d "$dest/cmdline-tools" | |
| 151 | + rm -f "$dl" | |
| 152 | + mv "$dest/cmdline-tools/cmdline-tools" "$dest/cmdline-tools/latest" | |
| 153 | + echo "$ANDROID_TOOLS_SHA" > "$stamp" | |
| 154 | + fi | |
| 155 | + # Gradle will not install anything into an SDK whose licences are | |
| 156 | + # unaccepted, and it fails late and obscurely when they are not. | |
| 157 | + if [ ! -d "$dest/licenses" ]; then | |
| 158 | + JAVA_HOME="$TC/jdk" ANDROID_HOME="$dest" \ | |
| 159 | + yes | "$dest/cmdline-tools/latest/bin/sdkmanager" --licenses >/dev/null | |
| 160 | + fi | |
| 161 | +} | |
| 162 | + | |
| 105 | 163 | install_all() { |
| 106 | 164 | require_host_tools |
| 107 | 165 | mkdir -p "$TC" |
| @@ -109,6 +167,7 @@ install_all() { | ||
| 109 | 167 | install_archive jdk "$JDK_URL" "$JDK_SHA" 1 |
| 110 | 168 | install_archive clojure "$CLOJURE_URL" "$CLOJURE_SHA" 1 |
| 111 | 169 | install_clojure |
| 170 | + install_archive nim "$NIM_URL" "$NIM_SHA" 1 | |
| 112 | 171 | } |
| 113 | 172 | |
| 114 | 173 | # The environment, as shell. Everything that would otherwise land in a home |
| @@ -116,7 +175,7 @@ install_all() { | ||
| 116 | 175 | # cache, the git dependencies tools.deps clones, the local maven repo. One |
| 117 | 176 | # directory to keep on a volume, one directory to delete when it goes wrong. |
| 118 | 177 | # |
| 119 | -# GITLIBS and the maven repo are set for the reason `just apk` sets them — | |
| 178 | +# GITLIBS and the maven repo are named here because | |
| 120 | 179 | # the JVM reads user.home out of /etc/passwd, so neither of them follows HOME. |
| 121 | 180 | print_env() { |
| 122 | 181 | cat <<ENV |
| @@ -137,12 +196,20 @@ export JAVA_HOME="$TC/jdk" | ||
| 137 | 196 | export PUB_CACHE="$TC/pub-cache" |
| 138 | 197 | export GITLIBS="$TC/gitlibs" |
| 139 | 198 | export FRQ_M2="$TC/m2" |
| 140 | -export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:\$PATH" | |
| 199 | +export ANDROID_HOME="$TC/android-sdk" | |
| 200 | +export ANDROID_SDK_ROOT="$TC/android-sdk" | |
| 201 | +# adb keeps the key the phone has already trusted here. Left to its default | |
| 202 | +# it would follow HOME, and a build that moved HOME would hand the device a | |
| 203 | +# new identity -- after which the deploy ends in "no devices/emulators found" | |
| 204 | +# while \`adb devices\` in any other shell lists the phone perfectly well. | |
| 205 | +export ANDROID_USER_HOME="\${ANDROID_USER_HOME:-\$HOME/.android}" | |
| 206 | +export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:$TC/nim/bin:$TC/android-sdk/platform-tools:\$PATH" | |
| 141 | 207 | ENV |
| 142 | 208 | } |
| 143 | 209 | |
| 144 | 210 | case "${1:-install}" in |
| 145 | 211 | install) install_all ;; |
| 212 | + android) install_all; install_android ;; | |
| 146 | 213 | env) install_all; print_env ;; |
| 147 | 214 | exec) |
| 148 | 215 | install_all |
| @@ -155,6 +222,8 @@ case "${1:-install}" in | ||
| 155 | 222 | echo "flutter $FLUTTER_VERSION" |
| 156 | 223 | echo "jdk $JDK_VERSION" |
| 157 | 224 | echo "clojure $CLOJURE_VERSION" |
| 225 | + echo "nim $NIM_VERSION" | |
| 226 | + echo "android-tools $ANDROID_TOOLS_VERSION" | |
| 158 | 227 | ;; |
| 159 | - *) echo "usage: toolchain.sh [install|env|exec -- cmd...|versions]" >&2; exit 1 ;; | |
| 228 | + *) echo "usage: toolchain.sh [install|android|env|exec -- cmd...|versions]" >&2; exit 1 ;; | |
| 160 | 229 | esac |
| @@ -1,22 +1,27 @@ | |||
| 1 | #!/usr/bin/env bash | 1 | #!/usr/bin/env bash |
| 2 | # The toolchain the web build needs, fetched by hand. | 2 | # The toolchain the web build needs, fetched by hand. |
| 3 | # | 3 | # |
| 4 | -# Three tarballs — Flutter (which carries Dart), a JDK, and the Clojure CLI — | 4 | +# Archives — Flutter (which carries Dart), a JDK, the Clojure CLI and Nim — |
| 5 | # pinned by version and by sha256, unpacked into `.toolchain/`, and put on a | 5 | # pinned by version and by sha256, unpacked into `.toolchain/`, and put on a |
| 6 | # PATH. That is the whole of it. No nix, no image, no devShell: a checkout | 6 | # PATH. That is the whole of it. No nix, no image, no devShell: a checkout |
| 7 | -# plus this script is a machine that can build the web bundle, and the same | 7 | +# plus this script is a machine that can build any target here, and the same |
| 8 | -# script is what the Modal container runs. | 8 | +# script is what the Modal containers run. |
| 9 | +# | ||
| 10 | +# The Android SDK is the one thing not fetched by default, because only | ||
| 11 | +# `just build apk` wants it and it is large: `tools/toolchain.sh android` | ||
| 12 | +# fetches Google's command-line tools and lets sdkmanager finish the job. | ||
| 9 | # | 13 | # |
| 10 | # Why not DotSlash, which the rest of the repo uses for its native libraries: | 14 | # Why not DotSlash, which the rest of the repo uses for its native libraries: |
| 11 | # DotSlash hands out an *immutable* cached artifact, and Flutter is not one. | 15 | # DotSlash hands out an *immutable* cached artifact, and Flutter is not one. |
| 12 | # `flutter build web` downloads its engine artifacts into `bin/cache/` inside | 16 | # `flutter build web` downloads its engine artifacts into `bin/cache/` inside |
| 13 | # its own SDK directory the first time it runs, so the SDK has to be writable | 17 | # its own SDK directory the first time it runs, so the SDK has to be writable |
| 14 | -# — which is the same thing `just apk` learned when it had to copy the | 18 | +# — which is the same reason the Android SDK below is fetched into a |
| 15 | -# store's Android SDK out to `flutter/.home` before Gradle would touch it. | 19 | +# directory of ours rather than used read-only from anywhere. |
| 16 | # A pinned URL and a checked hash give the reproducibility DotSlash is for; | 20 | # A pinned URL and a checked hash give the reproducibility DotSlash is for; |
| 17 | # the writability is what it cannot give. | 21 | # the writability is what it cannot give. |
| 18 | # | 22 | # |
| 19 | # tools/toolchain.sh fetch whatever is missing | 23 | # tools/toolchain.sh fetch whatever is missing |
| 24 | +# tools/toolchain.sh android ...and the Android SDK as well | ||
| 20 | # eval "$(tools/toolchain.sh env)" ...and put it on this shell's PATH | 25 | # eval "$(tools/toolchain.sh env)" ...and put it on this shell's PATH |
| 21 | # tools/toolchain.sh exec -- flutter --version | 26 | # tools/toolchain.sh exec -- flutter --version |
| 22 | # | 27 | # |
| @@ -52,13 +57,34 @@ CLOJURE_VERSION="1.12.6.1673" | |||
| 52 | CLOJURE_URL="https://github.com/clojure/brew-install/releases/download/${CLOJURE_VERSION}/clojure-tools-${CLOJURE_VERSION}.tar.gz" | 57 | CLOJURE_URL="https://github.com/clojure/brew-install/releases/download/${CLOJURE_VERSION}/clojure-tools-${CLOJURE_VERSION}.tar.gz" |
| 53 | CLOJURE_SHA="fe9194858e75d5af13c2e2aff92d710674d5bc5105f2b42f90a7d94d82ec023c" | 58 | CLOJURE_SHA="fe9194858e75d5af13c2e2aff92d710674d5bc5105f2b42f90a7d94d82ec023c" |
| 54 | 59 | ||
| 60 | +# Nim, for `nim/` — the portable core. Upstream's prebuilt linux_x64 tarball, | ||
| 61 | +# so there is no bootstrap compile here. The core deliberately has no | ||
| 62 | +# dependencies outside Nim's standard library, so the compiler is all of it. | ||
| 63 | +# | ||
| 64 | +# Two things Nim wants from the host rather than from here: a C compiler, | ||
| 65 | +# because `nim c` shells out to one, and OpenSSL, because `-d:ssl` in | ||
| 66 | +# nim/nim.cfg makes std/net resolve -lssl and -lcrypto through dynlib at run | ||
| 67 | +# time. A missing libssl is a SIGSEGV in `newContext` that says nothing about | ||
| 68 | +# SSL, which is why require_host_tools checks for cc up front. | ||
| 69 | +NIM_VERSION="2.2.4" | ||
| 70 | +NIM_URL="https://nim-lang.org/download/nim-${NIM_VERSION}-linux_x64.tar.xz" | ||
| 71 | +NIM_SHA="791802138aaf19c8579232c50b4998ce2ae2928b791127ce5b4ef3c7af53fb46" | ||
| 72 | + | ||
| 73 | +# Google's command-line tools, which is the smallest thing that can install an | ||
| 74 | +# Android SDK. The platform and build-tools are not pinned here: the versions | ||
| 75 | +# come from whatever Flutter asks Gradle for, and sdkmanager fetches them into | ||
| 76 | +# the same writable directory on first use. `install_android` below. | ||
| 77 | +ANDROID_TOOLS_VERSION="11076708" | ||
| 78 | +ANDROID_TOOLS_URL="https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_TOOLS_VERSION}_latest.zip" | ||
| 79 | +ANDROID_TOOLS_SHA="2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258" | ||
| 80 | + | ||
| 55 | # What the host still has to bring. Small, boring, and on every machine and | 81 | # What the host still has to bring. Small, boring, and on every machine and |
| 56 | # in every base image that is not deliberately empty — but Flutter shells out | 82 | # in every base image that is not deliberately empty — but Flutter shells out |
| 57 | # to `git` on its own SDK and to `unzip` on its downloads, so a missing one | 83 | # to `git` on its own SDK and to `unzip` on its downloads, so a missing one |
| 58 | # fails somewhere far from here with a much worse message than this. | 84 | # fails somewhere far from here with a much worse message than this. |
| 59 | require_host_tools() { | 85 | require_host_tools() { |
| 60 | local missing=() | 86 | local missing=() |
| 61 | - for t in curl tar git unzip; do | 87 | + for t in curl tar git unzip cc; do |
| 62 | command -v "$t" >/dev/null 2>&1 || missing+=("$t") | 88 | command -v "$t" >/dev/null 2>&1 || missing+=("$t") |
| 63 | done | 89 | done |
| 64 | if [ ${#missing[@]} -gt 0 ]; then | 90 | if [ ${#missing[@]} -gt 0 ]; then |
| @@ -102,6 +128,38 @@ install_clojure() { | |||
| 102 | chmod +x "$dest/bin/clojure" "$dest/bin/clj" | 128 | chmod +x "$dest/bin/clojure" "$dest/bin/clj" |
| 103 | } | 129 | } |
| 104 | 130 | ||
| 131 | +# The Android SDK, which is a zip rather than a tarball and has to land in a | ||
| 132 | +# layout sdkmanager recognises: cmdline-tools/latest/, with the tools' own | ||
| 133 | +# top-level directory renamed. Everything after that — platform-tools, the | ||
| 134 | +# platform, the build-tools — Gradle asks sdkmanager for as it goes, which is | ||
| 135 | +# why this directory is ours and writable rather than a read-only artifact. | ||
| 136 | +# | ||
| 137 | +# No ndkVersion in android/app/build.gradle.kts, and nothing here installs an | ||
| 138 | +# NDK: there is no native code in the app to need one. | ||
| 139 | +install_android() { | ||
| 140 | + local dest="$TC/android-sdk" | ||
| 141 | + local stamp="$dest/.cmdline-tools.sha256" | ||
| 142 | + if [ ! -d "$dest/cmdline-tools/latest" ] \ | ||
| 143 | + || [ "$(cat "$stamp" 2>/dev/null || true)" != "$ANDROID_TOOLS_SHA" ]; then | ||
| 144 | + echo "toolchain: fetching the Android command-line tools" >&2 | ||
| 145 | + local dl="$TC/.download.android" | ||
| 146 | + rm -rf "$dest/cmdline-tools" "$dl" | ||
| 147 | + mkdir -p "$dest/cmdline-tools" | ||
| 148 | + curl -fsSL --retry 3 -o "$dl" "$ANDROID_TOOLS_URL" | ||
| 149 | + echo "$ANDROID_TOOLS_SHA $dl" | sha256sum -c - >/dev/null | ||
| 150 | + unzip -q "$dl" -d "$dest/cmdline-tools" | ||
| 151 | + rm -f "$dl" | ||
| 152 | + mv "$dest/cmdline-tools/cmdline-tools" "$dest/cmdline-tools/latest" | ||
| 153 | + echo "$ANDROID_TOOLS_SHA" > "$stamp" | ||
| 154 | + fi | ||
| 155 | + # Gradle will not install anything into an SDK whose licences are | ||
| 156 | + # unaccepted, and it fails late and obscurely when they are not. | ||
| 157 | + if [ ! -d "$dest/licenses" ]; then | ||
| 158 | + JAVA_HOME="$TC/jdk" ANDROID_HOME="$dest" \ | ||
| 159 | + yes | "$dest/cmdline-tools/latest/bin/sdkmanager" --licenses >/dev/null | ||
| 160 | + fi | ||
| 161 | +} | ||
| 162 | + | ||
| 105 | install_all() { | 163 | install_all() { |
| 106 | require_host_tools | 164 | require_host_tools |
| 107 | mkdir -p "$TC" | 165 | mkdir -p "$TC" |
| @@ -109,6 +167,7 @@ install_all() { | |||
| 109 | install_archive jdk "$JDK_URL" "$JDK_SHA" 1 | 167 | install_archive jdk "$JDK_URL" "$JDK_SHA" 1 |
| 110 | install_archive clojure "$CLOJURE_URL" "$CLOJURE_SHA" 1 | 168 | install_archive clojure "$CLOJURE_URL" "$CLOJURE_SHA" 1 |
| 111 | install_clojure | 169 | install_clojure |
| 170 | + install_archive nim "$NIM_URL" "$NIM_SHA" 1 | ||
| 112 | } | 171 | } |
| 113 | 172 | ||
| 114 | # The environment, as shell. Everything that would otherwise land in a home | 173 | # The environment, as shell. Everything that would otherwise land in a home |
| @@ -116,7 +175,7 @@ install_all() { | |||
| 116 | # cache, the git dependencies tools.deps clones, the local maven repo. One | 175 | # cache, the git dependencies tools.deps clones, the local maven repo. One |
| 117 | # directory to keep on a volume, one directory to delete when it goes wrong. | 176 | # directory to keep on a volume, one directory to delete when it goes wrong. |
| 118 | # | 177 | # |
| 119 | -# GITLIBS and the maven repo are set for the reason `just apk` sets them — | 178 | +# GITLIBS and the maven repo are named here because |
| 120 | # the JVM reads user.home out of /etc/passwd, so neither of them follows HOME. | 179 | # the JVM reads user.home out of /etc/passwd, so neither of them follows HOME. |
| 121 | print_env() { | 180 | print_env() { |
| 122 | cat <<ENV | 181 | cat <<ENV |
| @@ -137,12 +196,20 @@ export JAVA_HOME="$TC/jdk" | |||
| 137 | export PUB_CACHE="$TC/pub-cache" | 196 | export PUB_CACHE="$TC/pub-cache" |
| 138 | export GITLIBS="$TC/gitlibs" | 197 | export GITLIBS="$TC/gitlibs" |
| 139 | export FRQ_M2="$TC/m2" | 198 | export FRQ_M2="$TC/m2" |
| 140 | -export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:\$PATH" | 199 | +export ANDROID_HOME="$TC/android-sdk" |
| 200 | +export ANDROID_SDK_ROOT="$TC/android-sdk" | ||
| 201 | +# adb keeps the key the phone has already trusted here. Left to its default | ||
| 202 | +# it would follow HOME, and a build that moved HOME would hand the device a | ||
| 203 | +# new identity -- after which the deploy ends in "no devices/emulators found" | ||
| 204 | +# while \`adb devices\` in any other shell lists the phone perfectly well. | ||
| 205 | +export ANDROID_USER_HOME="\${ANDROID_USER_HOME:-\$HOME/.android}" | ||
| 206 | +export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:$TC/nim/bin:$TC/android-sdk/platform-tools:\$PATH" | ||
| 141 | ENV | 207 | ENV |
| 142 | } | 208 | } |
| 143 | 209 | ||
| 144 | case "${1:-install}" in | 210 | case "${1:-install}" in |
| 145 | install) install_all ;; | 211 | install) install_all ;; |
| 212 | + android) install_all; install_android ;; | ||
| 146 | env) install_all; print_env ;; | 213 | env) install_all; print_env ;; |
| 147 | exec) | 214 | exec) |
| 148 | install_all | 215 | install_all |
| @@ -155,6 +222,8 @@ case "${1:-install}" in | |||
| 155 | echo "flutter $FLUTTER_VERSION" | 222 | echo "flutter $FLUTTER_VERSION" |
| 156 | echo "jdk $JDK_VERSION" | 223 | echo "jdk $JDK_VERSION" |
| 157 | echo "clojure $CLOJURE_VERSION" | 224 | echo "clojure $CLOJURE_VERSION" |
| 225 | + echo "nim $NIM_VERSION" | ||
| 226 | + echo "android-tools $ANDROID_TOOLS_VERSION" | ||
| 158 | ;; | 227 | ;; |
| 159 | - *) echo "usage: toolchain.sh [install|env|exec -- cmd...|versions]" >&2; exit 1 ;; | 228 | + *) echo "usage: toolchain.sh [install|android|env|exec -- cmd...|versions]" >&2; exit 1 ;; |
| 160 | esac | 229 | esac |