nandi/frqpublic Fork 0
2e24e64
Commits
Clone
git clone https://git.rickub.com/nandi/frq.git
git clone ssh://git@rickub.com/nandi/frq.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Six verbs, and the last of the nix

The justfile was thirteen recipes, each carrying its own copy of the same
re-entry dance: test an environment variable, `nix develop` back into the
same recipe if it is unset, do the work if it is set. Now it is six verbs
— build, run, test, modal, serve, tools — and the three near-identical
Flutter recipes are one private `_flutter` that differs only in which
entry point it paints.

The toolchain is `tools/toolchain.sh` for everything, not just the web
build: Nim joins Flutter, the JDK and the Clojure CLI as a pinned tarball,
and `toolchain.sh android` fetches Google's command-line tools on demand
so Gradle has an SDK it is allowed to write to. That is what the APK
needed a writable copy of the store for, and it is the last thing the
flake was doing that a tarball could not.

So flake.nix and flake.lock are gone, and `.modal/dev` — which baked
`nix print-dev-env` into its image and entered `nix develop` at run time —
is rebuilt on the same debian image and the same script `.modal/web`
already used. `_loader.py` loses its `[nix]` support with them: the
ptyshim, the devShell warming layer, the substituter plumbing, the
`nix develop` wrapper. 139 lines out, 25 in.

The containers lose the `flutter-` prefix too, this being a Flutter-only
project: `just modal dev`, `just modal web`. Their directories on the
devshell volume are renamed to match, so the first run after this starts
from a cold toolchain once. The deployed app in `.modal/web/serve.py`
keeps its name — it is the live URL, and freeq's auth broker allowlists
that origin.

Verified from a cold .toolchain: test common/nim/dart pass, build desktop
and build ui both compile, and both container specs construct their
images through the pruned loader.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nandi committed 2026-09-18T23:25:04-07:00 Browse files
2e24e64 parent: e505ded
modified .gitignore +3 -2
@@ -11,7 +11,8 @@
1111 /nim/tests/t*
1212 !/nim/tests/t*.nim
1313
14-# The pinned Flutter/JDK/Clojure tarballs `tools/toolchain.sh` fetches, plus
14+# The pinned Flutter/JDK/Clojure/Nim tarballs and the Android SDK that
15+# `tools/toolchain.sh` fetches, plus
1516 # the pub cache, gitlibs and maven repo it keeps beside them. A gigabyte of
1617 # SDK, reproducible from the hashes in that script.
1718 /.toolchain/
@@ -31,5 +32,5 @@
3132 # Python bytecode, from the loader the .modal/ containers import.
3233 __pycache__/
3334
34-# The mirror `just web-local` keeps of the Modal-built web bundle.
35+# The mirror `just serve` keeps of the Modal-built web bundle.
3536 .web-local/
@@ -11,7 +11,8 @@
11 /nim/tests/t*11 /nim/tests/t*
12 !/nim/tests/t*.nim12 !/nim/tests/t*.nim
13 13
14-# The pinned Flutter/JDK/Clojure tarballs `tools/toolchain.sh` fetches, plus14+# The pinned Flutter/JDK/Clojure/Nim tarballs and the Android SDK that
15+# `tools/toolchain.sh` fetches, plus
15 # the pub cache, gitlibs and maven repo it keeps beside them. A gigabyte of16 # the pub cache, gitlibs and maven repo it keeps beside them. A gigabyte of
16 # SDK, reproducible from the hashes in that script.17 # SDK, reproducible from the hashes in that script.
17 /.toolchain/18 /.toolchain/
@@ -31,5 +32,5 @@
31 # Python bytecode, from the loader the .modal/ containers import.32 # Python bytecode, from the loader the .modal/ containers import.
32 __pycache__/33 __pycache__/
33 34
34-# The mirror `just web-local` keeps of the Modal-built web bundle.35+# The mirror `just serve` keeps of the Modal-built web bundle.
35 .web-local/36 .web-local/
modified .modal/_loader.py +25 -139
@@ -2,23 +2,19 @@
22
33 Every container -- under `containers/` here, `.modal/` in a repo that merely
44 builds itself on Modal -- is a directory with a `container.toml` and
5-a stub `container.py`. See `spec.md` for the keys; this file is what reads
6-them. Nothing here is Modal-specific configuration in its own right -- each
5+a stub `container.py`. The keys are documented by the comments below, which
6+is the whole of the spec. Nothing here is Modal-specific configuration in its
7+own right -- each
78 spec key maps onto a documented Modal argument, and the mapping is meant to
89 stay boring enough to read straight through.
910 """
1011
1112 import os
12-import shlex
1313 import subprocess
1414 import tomllib
1515
1616 import modal
1717
18-REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
19-PTYSHIM_C = os.path.join(REPO, "ptyshim.c")
20-SHIM_SO = "/opt/ptyshim.so"
21-
2218
2319 class SpecError(Exception):
2420 """The container.toml says something that cannot be built."""
@@ -44,34 +40,16 @@ class Container:
4440 self.command = run.get("command", "")
4541 self.env = dict(run.get("env", {}))
4642
47- # Substituters reach nix through NIX_CONFIG rather than nix.conf, and
48- # at run time rather than build time. Written into the image's nix.conf
49- # instead, nix touches the path while the image is being built and
50- # creates it -- and Modal will not mount a volume over a non-empty
51- # directory, so the container that wanted the cache cannot start. As
52- # env it covers every nix command that runs in the container, the ones
53- # typed by hand in a shell included, and leaves the mount point empty.
54- if subs := list(spec.get("nix", {}).get("substituters", [])):
55- line = "extra-substituters = " + " ".join(subs)
56- self.env["NIX_CONFIG"] = (
57- f"{self.env['NIX_CONFIG']}\n{line}" if "NIX_CONFIG" in self.env
58- else line
59- )
60-
6143 # "function" (the default) or "sandbox". Sandboxes can run on a real
6244 # VM, which Functions cannot -- see ../README.md.
6345 self.runtime = spec.get("container", {}).get("runtime", "function")
6446
65- nix = spec.get("nix", {})
66- self.use_flake = bool(nix.get("flake", False))
67- self.use_shim = bool(nix.get("shim", False))
68-
6947 # name -> mount path. Modal Volumes, mounted while the container runs
7048 # and NOT while its image is built: a volume mount is not part of the
7149 # resulting image, so anything written to one during a build step is
7250 # gone by the time the container starts. Persist across runs is the
73- # whole point -- a nix store to substitute from, a cargo target
74- # directory, a dataset too big to bake in.
51+ # whole point -- a toolchain too big to fetch every run, a build
52+ # tree an incremental compile reuses, a dataset too big to bake in.
7553 self.volume_spec = dict(spec.get("volumes", {}))
7654
7755 # Ports to tunnel out of a Sandbox, from [network] ports. Encrypted,
@@ -86,7 +64,7 @@ class Container:
8664
8765 # Modal re-imports this module inside the container, so everything
8866 # below runs twice: once here, once out there. Out there the local
89- # tree does not exist -- no flake.nix, no ptyshim.c, no repo -- and
67+ # tree does not exist -- no repo to copy from -- and
9068 # the image is already built, so validating and rebuilding it would
9169 # only fail. The App still has to exist for the decorators to bind.
9270 if self.is_remote:
@@ -124,17 +102,6 @@ class Container:
124102 )
125103 if bool(c.get("base")) == bool(c.get("registry")):
126104 raise SpecError("set exactly one of [container] base or registry")
127- if self.use_flake:
128- if not os.path.exists(os.path.join(self.dir, "flake.nix")):
129- raise SpecError("[nix] flake = true but there is no flake.nix")
130- if not self.use_shim:
131- # Warming the shell builds nix-shell-env, which is the gVisor
132- # pty bug. Failing here beats failing ten minutes into a build.
133- raise SpecError(
134- "[nix] flake = true needs shim = true -- see ../README.md"
135- )
136- if self.use_shim and not os.path.exists(PTYSHIM_C):
137- raise SpecError(f"[nix] shim = true but {PTYSHIM_C} is missing")
138105 if self.ports and self.runtime != "sandbox":
139106 raise SpecError(
140107 "[network] ports needs [container] runtime = \"sandbox\" --"
@@ -146,9 +113,9 @@ class Container:
146113 f"[volumes] {name} must be an absolute path, not {mount!r}"
147114 )
148115 # Mounting over one of these hides what the image already has
149- # there -- /nix in particular, where the empty volume would shadow
150- # the store the base image spent its build populating.
151- if mount.rstrip("/") in ("", "/nix", "/nix/store", "/usr", "/etc"):
116+ # there: an empty volume would shadow what the image's own build
117+ # put in its place.
118+ if mount.rstrip("/") in ("", "/usr", "/etc", "/bin", "/lib"):
152119 raise SpecError(
153120 f"[volumes] {name} may not mount over {mount} --"
154121 " it would hide what the image has there"
@@ -161,33 +128,6 @@ class Container:
161128
162129 # -- image -----------------------------------------------------------
163130
164- @property
165- def nix(self) -> str:
166- """The `nix` command as the container runs it.
167-
168- A sandbox on a real VM has a working pty, so the shim buys nothing
169- there and is left off even when the image was built with it.
170- """
171- if self.use_shim and not self.vm_at_runtime:
172- return f"LD_PRELOAD={SHIM_SO} nix"
173- return "nix"
174-
175- @property
176- def vm_at_runtime(self) -> bool:
177- """True when this container actually runs on a VM rather than gVisor."""
178- return (self.runtime == "sandbox"
179- and bool(self.experimental_options.get("vm_runtime")))
180-
181- @property
182- def build_nix(self) -> str:
183- """The `nix` command for BUILD steps, which always run under gVisor.
184-
185- Image builds are Functions underneath, so a sandbox container still
186- needs the shim while its image is being built -- only its run time
187- gets the VM.
188- """
189- return f"LD_PRELOAD={SHIM_SO} nix" if self.use_shim else "nix"
190-
191131 def _build_image(self) -> modal.Image:
192132 c = self.spec["container"]
193133 build = self.spec.get("build", {})
@@ -197,45 +137,6 @@ class Container:
197137 else:
198138 image = modal.Image.from_registry(c["registry"])
199139
200- # `nix profile install` puts things in ~/.nix-profile/bin, which is on
201- # nobody's PATH here -- so the install succeeds and the very next line
202- # says `command not found`. Set on the image rather than in a .bashrc,
203- # because a container's command runs under `sh -c` and reads neither.
204- # Spelled out rather than prefixed onto $PATH: an image env is a value,
205- # not a shell expression, so "$PATH" here would be four literal
206- # characters.
207- image = image.env({
208- "PATH": "/root/.nix-profile/bin:/usr/local/sbin:/usr/local/bin"
209- ":/usr/sbin:/usr/bin:/sbin:/bin",
210- })
211-
212- if self.use_shim:
213- image = image.add_local_file(
214- PTYSHIM_C, "/opt/ptyshim.c", copy=True
215- ).run_commands(
216- f"gcc -shared -fPIC -O2 -o {SHIM_SO} /opt/ptyshim.c -ldl"
217- )
218-
219- # Warm the devShell BEFORE the source is copied in. Everything the
220- # shell needs is binary-cached, so the download happens once -- but
221- # only if this layer survives. Copy the source first and any edit to
222- # any file invalidates the warm, and the whole closure is fetched
223- # again on every build. Only flake.nix and flake.lock go in here, so
224- # the layer is invalidated by a dependency change and nothing else.
225- if self.use_flake:
226- for f in ("flake.nix", "flake.lock"):
227- path = os.path.join(self.dir, f)
228- if os.path.exists(path):
229- image = image.add_local_file(
230- path, f"{self.workdir}/{f}", copy=True
231- )
232- image = image.run_commands(
233- f"cd {self.workdir} && {self.build_nix} develop"
234- " --accept-flake-config --command true",
235- f'echo "store paths after warming:'
236- f' $({self.build_nix} path-info --all | wc -l)"',
237- )
238-
239140 # copy=True throughout: later run_commands need these files present.
240141 # `context` is what include paths are relative to, and it may sit above
241142 # the container directory -- a container that builds the repo it lives
@@ -244,19 +145,13 @@ class Container:
244145
245146 # Image building and program building, kept apart.
246147 #
247- # `[build] commands` run AFTER the source copy, so any edit anywhere in
248- # the tree invalidates them -- and for a container whose commands warm
249- # a devShell, that means minutes of nix on every iteration of a
250- # one-line change. `warm` + `setup` are the same two steps moved in
251- # front of the source: `warm` names only the files the flake actually
252- # evaluates (its own, and whatever the devShell's derivations read),
253- # and `setup` runs against those alone. Editing `flutter/src` then
254- # invalidates nothing above the final copy, and the toolchain layer is
255- # reused until a dependency moves.
256- #
257- # Volumes are mounted here for `commands`' reason: a step that can
258- # reach the binary cache never has to build, which is the one thing
259- # gVisor will not do.
148+ # `[build] commands` run AFTER the source copy, so any edit anywhere
149+ # in the tree invalidates them. `warm` + `setup` are the same step
150+ # moved in front of the source: `warm` names only the files the step
151+ # actually reads, and `setup` runs against those alone, so editing
152+ # `flutter/src` invalidates nothing above the final copy. A container
153+ # whose setup is an `apt-get` needs no `warm` at all -- it reads
154+ # nothing out of the tree, so nothing in the tree can invalidate it.
260155 for rel in build.get("warm", []):
261156 src = os.path.normpath(os.path.join(context, rel))
262157 dest = f"{self.workdir}/{rel}"
@@ -281,21 +176,17 @@ class Container:
281176 else:
282177 image = image.add_local_file(src, dest, copy=True)
283178
284- # A repo copied in brings its `.git` along, and in a worktree that is a
285- # *file* holding `gitdir: <path on the machine that copied it>`. Nix
286- # believes it and goes looking for a checkout that is not there --
287- # `nix develop` and `nix build .#x` both die before evaluating
288- # anything. Nothing in a container wants the git metadata, so it goes.
179+ # A repo copied in brings its `.git` along, and in a worktree that is
180+ # a *file* holding `gitdir: <path on the machine that copied it>` --
181+ # which points at nothing out here, so any tool that follows it fails
182+ # in a way that has nothing to do with what it was asked to do.
183+ # Nothing in a container wants the git metadata, so it goes.
289184 image = image.run_commands(f"rm -rf {self.workdir}/.git")
290185
291186 if commands := build.get("commands", []):
292- # Volumes mounted for the build too, not just the run. A build step
293- # that wants nix to *build* something is the one thing gVisor will
294- # not do -- image builds are Functions underneath, and a derivation
295- # there dies on `unexpected EOF reading a line`. Substituting is
296- # fine, so a step that can reach the cache never has to build, and
297- # the shim stays retired. The mount is not part of the resulting
298- # image; only what the step writes outside it is.
187+ # Volumes mounted for the build too, not just the run, so a step
188+ # can read a cache the last run filled. The mount is not part of
189+ # the resulting image; only what the step writes outside it is.
299190 image = image.run_commands(*commands, volumes=self.volumes)
300191
301192 # container.py does `from _loader import Container`, and Modal mounts
@@ -386,15 +277,10 @@ class Container:
386277 return kwargs
387278
388279 def shell_command(self, override: str = "") -> str:
389- """The full shell line the container runs, devShell wrapper included."""
280+ """The full shell line the container runs."""
390281 command = override or self.command
391282 if not command:
392283 raise SpecError("container.toml has no [run] command")
393- if self.use_flake:
394- return (
395- f"cd {self.workdir} && {self.nix} develop --accept-flake-config"
396- f" --command sh -c {shlex.quote(command)}"
397- )
398284 return f"cd {self.workdir} && {command}"
399285
400286 def run_sandbox(self, override: str = "") -> str:
@@ -2,23 +2,19 @@
2 2
3 Every container -- under `containers/` here, `.modal/` in a repo that merely3 Every container -- under `containers/` here, `.modal/` in a repo that merely
4 builds itself on Modal -- is a directory with a `container.toml` and4 builds itself on Modal -- is a directory with a `container.toml` and
5-a stub `container.py`. See `spec.md` for the keys; this file is what reads5+a stub `container.py`. The keys are documented by the comments below, which
6-them. Nothing here is Modal-specific configuration in its own right -- each6+is the whole of the spec. Nothing here is Modal-specific configuration in its
7+own right -- each
7 spec key maps onto a documented Modal argument, and the mapping is meant to8 spec key maps onto a documented Modal argument, and the mapping is meant to
8 stay boring enough to read straight through.9 stay boring enough to read straight through.
9 """10 """
10 11
11 import os12 import os
12-import shlex
13 import subprocess13 import subprocess
14 import tomllib14 import tomllib
15 15
16 import modal16 import modal
17 17
18-REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
19-PTYSHIM_C = os.path.join(REPO, "ptyshim.c")
20-SHIM_SO = "/opt/ptyshim.so"
21-
22 18
23 class SpecError(Exception):19 class SpecError(Exception):
24 """The container.toml says something that cannot be built."""20 """The container.toml says something that cannot be built."""
@@ -44,34 +40,16 @@ class Container:
44 self.command = run.get("command", "")40 self.command = run.get("command", "")
45 self.env = dict(run.get("env", {}))41 self.env = dict(run.get("env", {}))
46 42
47- # Substituters reach nix through NIX_CONFIG rather than nix.conf, and
48- # at run time rather than build time. Written into the image's nix.conf
49- # instead, nix touches the path while the image is being built and
50- # creates it -- and Modal will not mount a volume over a non-empty
51- # directory, so the container that wanted the cache cannot start. As
52- # env it covers every nix command that runs in the container, the ones
53- # typed by hand in a shell included, and leaves the mount point empty.
54- if subs := list(spec.get("nix", {}).get("substituters", [])):
55- line = "extra-substituters = " + " ".join(subs)
56- self.env["NIX_CONFIG"] = (
57- f"{self.env['NIX_CONFIG']}\n{line}" if "NIX_CONFIG" in self.env
58- else line
59- )
60-
61 # "function" (the default) or "sandbox". Sandboxes can run on a real43 # "function" (the default) or "sandbox". Sandboxes can run on a real
62 # VM, which Functions cannot -- see ../README.md.44 # VM, which Functions cannot -- see ../README.md.
63 self.runtime = spec.get("container", {}).get("runtime", "function")45 self.runtime = spec.get("container", {}).get("runtime", "function")
64 46
65- nix = spec.get("nix", {})
66- self.use_flake = bool(nix.get("flake", False))
67- self.use_shim = bool(nix.get("shim", False))
68-
69 # name -> mount path. Modal Volumes, mounted while the container runs47 # name -> mount path. Modal Volumes, mounted while the container runs
70 # and NOT while its image is built: a volume mount is not part of the48 # and NOT while its image is built: a volume mount is not part of the
71 # resulting image, so anything written to one during a build step is49 # resulting image, so anything written to one during a build step is
72 # gone by the time the container starts. Persist across runs is the50 # gone by the time the container starts. Persist across runs is the
73- # whole point -- a nix store to substitute from, a cargo target51+ # whole point -- a toolchain too big to fetch every run, a build
74- # directory, a dataset too big to bake in.52+ # tree an incremental compile reuses, a dataset too big to bake in.
75 self.volume_spec = dict(spec.get("volumes", {}))53 self.volume_spec = dict(spec.get("volumes", {}))
76 54
77 # Ports to tunnel out of a Sandbox, from [network] ports. Encrypted,55 # Ports to tunnel out of a Sandbox, from [network] ports. Encrypted,
@@ -86,7 +64,7 @@ class Container:
86 64
87 # Modal re-imports this module inside the container, so everything65 # Modal re-imports this module inside the container, so everything
88 # below runs twice: once here, once out there. Out there the local66 # below runs twice: once here, once out there. Out there the local
89- # tree does not exist -- no flake.nix, no ptyshim.c, no repo -- and67+ # tree does not exist -- no repo to copy from -- and
90 # the image is already built, so validating and rebuilding it would68 # the image is already built, so validating and rebuilding it would
91 # only fail. The App still has to exist for the decorators to bind.69 # only fail. The App still has to exist for the decorators to bind.
92 if self.is_remote:70 if self.is_remote:
@@ -124,17 +102,6 @@ class Container:
124 )102 )
125 if bool(c.get("base")) == bool(c.get("registry")):103 if bool(c.get("base")) == bool(c.get("registry")):
126 raise SpecError("set exactly one of [container] base or registry")104 raise SpecError("set exactly one of [container] base or registry")
127- if self.use_flake:
128- if not os.path.exists(os.path.join(self.dir, "flake.nix")):
129- raise SpecError("[nix] flake = true but there is no flake.nix")
130- if not self.use_shim:
131- # Warming the shell builds nix-shell-env, which is the gVisor
132- # pty bug. Failing here beats failing ten minutes into a build.
133- raise SpecError(
134- "[nix] flake = true needs shim = true -- see ../README.md"
135- )
136- if self.use_shim and not os.path.exists(PTYSHIM_C):
137- raise SpecError(f"[nix] shim = true but {PTYSHIM_C} is missing")
138 if self.ports and self.runtime != "sandbox":105 if self.ports and self.runtime != "sandbox":
139 raise SpecError(106 raise SpecError(
140 "[network] ports needs [container] runtime = \"sandbox\" --"107 "[network] ports needs [container] runtime = \"sandbox\" --"
@@ -146,9 +113,9 @@ class Container:
146 f"[volumes] {name} must be an absolute path, not {mount!r}"113 f"[volumes] {name} must be an absolute path, not {mount!r}"
147 )114 )
148 # Mounting over one of these hides what the image already has115 # Mounting over one of these hides what the image already has
149- # there -- /nix in particular, where the empty volume would shadow116+ # there: an empty volume would shadow what the image's own build
150- # the store the base image spent its build populating.117+ # put in its place.
151- if mount.rstrip("/") in ("", "/nix", "/nix/store", "/usr", "/etc"):118+ if mount.rstrip("/") in ("", "/usr", "/etc", "/bin", "/lib"):
152 raise SpecError(119 raise SpecError(
153 f"[volumes] {name} may not mount over {mount} --"120 f"[volumes] {name} may not mount over {mount} --"
154 " it would hide what the image has there"121 " it would hide what the image has there"
@@ -161,33 +128,6 @@ class Container:
161 128
162 # -- image -----------------------------------------------------------129 # -- image -----------------------------------------------------------
163 130
164- @property
165- def nix(self) -> str:
166- """The `nix` command as the container runs it.
167-
168- A sandbox on a real VM has a working pty, so the shim buys nothing
169- there and is left off even when the image was built with it.
170- """
171- if self.use_shim and not self.vm_at_runtime:
172- return f"LD_PRELOAD={SHIM_SO} nix"
173- return "nix"
174-
175- @property
176- def vm_at_runtime(self) -> bool:
177- """True when this container actually runs on a VM rather than gVisor."""
178- return (self.runtime == "sandbox"
179- and bool(self.experimental_options.get("vm_runtime")))
180-
181- @property
182- def build_nix(self) -> str:
183- """The `nix` command for BUILD steps, which always run under gVisor.
184-
185- Image builds are Functions underneath, so a sandbox container still
186- needs the shim while its image is being built -- only its run time
187- gets the VM.
188- """
189- return f"LD_PRELOAD={SHIM_SO} nix" if self.use_shim else "nix"
190-
191 def _build_image(self) -> modal.Image:131 def _build_image(self) -> modal.Image:
192 c = self.spec["container"]132 c = self.spec["container"]
193 build = self.spec.get("build", {})133 build = self.spec.get("build", {})
@@ -197,45 +137,6 @@ class Container:
197 else:137 else:
198 image = modal.Image.from_registry(c["registry"])138 image = modal.Image.from_registry(c["registry"])
199 139
200- # `nix profile install` puts things in ~/.nix-profile/bin, which is on
201- # nobody's PATH here -- so the install succeeds and the very next line
202- # says `command not found`. Set on the image rather than in a .bashrc,
203- # because a container's command runs under `sh -c` and reads neither.
204- # Spelled out rather than prefixed onto $PATH: an image env is a value,
205- # not a shell expression, so "$PATH" here would be four literal
206- # characters.
207- image = image.env({
208- "PATH": "/root/.nix-profile/bin:/usr/local/sbin:/usr/local/bin"
209- ":/usr/sbin:/usr/bin:/sbin:/bin",
210- })
211-
212- if self.use_shim:
213- image = image.add_local_file(
214- PTYSHIM_C, "/opt/ptyshim.c", copy=True
215- ).run_commands(
216- f"gcc -shared -fPIC -O2 -o {SHIM_SO} /opt/ptyshim.c -ldl"
217- )
218-
219- # Warm the devShell BEFORE the source is copied in. Everything the
220- # shell needs is binary-cached, so the download happens once -- but
221- # only if this layer survives. Copy the source first and any edit to
222- # any file invalidates the warm, and the whole closure is fetched
223- # again on every build. Only flake.nix and flake.lock go in here, so
224- # the layer is invalidated by a dependency change and nothing else.
225- if self.use_flake:
226- for f in ("flake.nix", "flake.lock"):
227- path = os.path.join(self.dir, f)
228- if os.path.exists(path):
229- image = image.add_local_file(
230- path, f"{self.workdir}/{f}", copy=True
231- )
232- image = image.run_commands(
233- f"cd {self.workdir} && {self.build_nix} develop"
234- " --accept-flake-config --command true",
235- f'echo "store paths after warming:'
236- f' $({self.build_nix} path-info --all | wc -l)"',
237- )
238-
239 # copy=True throughout: later run_commands need these files present.140 # copy=True throughout: later run_commands need these files present.
240 # `context` is what include paths are relative to, and it may sit above141 # `context` is what include paths are relative to, and it may sit above
241 # the container directory -- a container that builds the repo it lives142 # the container directory -- a container that builds the repo it lives
@@ -244,19 +145,13 @@ class Container:
244 145
245 # Image building and program building, kept apart.146 # Image building and program building, kept apart.
246 #147 #
247- # `[build] commands` run AFTER the source copy, so any edit anywhere in148+ # `[build] commands` run AFTER the source copy, so any edit anywhere
248- # the tree invalidates them -- and for a container whose commands warm149+ # in the tree invalidates them. `warm` + `setup` are the same step
249- # a devShell, that means minutes of nix on every iteration of a150+ # moved in front of the source: `warm` names only the files the step
250- # one-line change. `warm` + `setup` are the same two steps moved in151+ # actually reads, and `setup` runs against those alone, so editing
251- # front of the source: `warm` names only the files the flake actually152+ # `flutter/src` invalidates nothing above the final copy. A container
252- # evaluates (its own, and whatever the devShell's derivations read),153+ # whose setup is an `apt-get` needs no `warm` at all -- it reads
253- # and `setup` runs against those alone. Editing `flutter/src` then154+ # nothing out of the tree, so nothing in the tree can invalidate it.
254- # invalidates nothing above the final copy, and the toolchain layer is
255- # reused until a dependency moves.
256- #
257- # Volumes are mounted here for `commands`' reason: a step that can
258- # reach the binary cache never has to build, which is the one thing
259- # gVisor will not do.
260 for rel in build.get("warm", []):155 for rel in build.get("warm", []):
261 src = os.path.normpath(os.path.join(context, rel))156 src = os.path.normpath(os.path.join(context, rel))
262 dest = f"{self.workdir}/{rel}"157 dest = f"{self.workdir}/{rel}"
@@ -281,21 +176,17 @@ class Container:
281 else:176 else:
282 image = image.add_local_file(src, dest, copy=True)177 image = image.add_local_file(src, dest, copy=True)
283 178
284- # A repo copied in brings its `.git` along, and in a worktree that is a179+ # A repo copied in brings its `.git` along, and in a worktree that is
285- # *file* holding `gitdir: <path on the machine that copied it>`. Nix180+ # a *file* holding `gitdir: <path on the machine that copied it>` --
286- # believes it and goes looking for a checkout that is not there --181+ # which points at nothing out here, so any tool that follows it fails
287- # `nix develop` and `nix build .#x` both die before evaluating182+ # in a way that has nothing to do with what it was asked to do.
288- # anything. Nothing in a container wants the git metadata, so it goes.183+ # Nothing in a container wants the git metadata, so it goes.
289 image = image.run_commands(f"rm -rf {self.workdir}/.git")184 image = image.run_commands(f"rm -rf {self.workdir}/.git")
290 185
291 if commands := build.get("commands", []):186 if commands := build.get("commands", []):
292- # Volumes mounted for the build too, not just the run. A build step187+ # Volumes mounted for the build too, not just the run, so a step
293- # that wants nix to *build* something is the one thing gVisor will188+ # can read a cache the last run filled. The mount is not part of
294- # not do -- image builds are Functions underneath, and a derivation189+ # the resulting image; only what the step writes outside it is.
295- # there dies on `unexpected EOF reading a line`. Substituting is
296- # fine, so a step that can reach the cache never has to build, and
297- # the shim stays retired. The mount is not part of the resulting
298- # image; only what the step writes outside it is.
299 image = image.run_commands(*commands, volumes=self.volumes)190 image = image.run_commands(*commands, volumes=self.volumes)
300 191
301 # container.py does `from _loader import Container`, and Modal mounts192 # container.py does `from _loader import Container`, and Modal mounts
@@ -386,15 +277,10 @@ class Container:
386 return kwargs277 return kwargs
387 278
388 def shell_command(self, override: str = "") -> str:279 def shell_command(self, override: str = "") -> str:
389- """The full shell line the container runs, devShell wrapper included."""280+ """The full shell line the container runs."""
390 command = override or self.command281 command = override or self.command
391 if not command:282 if not command:
392 raise SpecError("container.toml has no [run] command")283 raise SpecError("container.toml has no [run] command")
393- if self.use_flake:
394- return (
395- f"cd {self.workdir} && {self.nix} develop --accept-flake-config"
396- f" --command sh -c {shlex.quote(command)}"
397- )
398 return f"cd {self.workdir} && {command}"284 return f"cd {self.workdir} && {command}"
399 285
400 def run_sandbox(self, override: str = "") -> str:286 def run_sandbox(self, override: str = "") -> str:
added .modal/dev/README.md +17 -0
new file mode 100644
@@ -0,0 +1,17 @@
1+# `dev`
2+
3+ scripts/deploy dev
4+ modal run .modal/dev/container.py
5+
6+Defined by `container.toml`; `../_loader.py` is what reads it, and
7+its comments are the spec.
8+Built on `debian:13-slim`.
9+
10+Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command
11+is the sandbox's own process, so it dies when the command exits --
12+no idle window and nothing to tear down. Note the VM restrictions:
13+no GPU, and memory is exactly what `[resources] memory` asks for.
14+
15+No nix anywhere: the toolchain is `tools/toolchain.sh`, fetched by
16+pinned sha256 onto the `devshell` volume, so nothing is substituted
17+at build time and nothing is evaluated at start.
new file mode 100644
@@ -0,0 +1,17 @@
1+# `dev`
2+
3+ scripts/deploy dev
4+ modal run .modal/dev/container.py
5+
6+Defined by `container.toml`; `../_loader.py` is what reads it, and
7+its comments are the spec.
8+Built on `debian:13-slim`.
9+
10+Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command
11+is the sandbox's own process, so it dies when the command exits --
12+no idle window and nothing to tear down. Note the VM restrictions:
13+no GPU, and memory is exactly what `[resources] memory` asks for.
14+
15+No nix anywhere: the toolchain is `tools/toolchain.sh`, fetched by
16+pinned sha256 onto the `devshell` volume, so nothing is substituted
17+at build time and nothing is evaluated at start.
renamed .modal/dev/container.py +5 -4
similarity index 81%
rename from .modal/flutter-web/container.py
rename to .modal/dev/container.py
@@ -26,10 +26,11 @@ def main(command: str = "", shell: bool = False):
2626 c.run_sandbox(command)
2727 return
2828
29- # `modal shell --image` only takes registry references, so it cannot be
30- # pointed at a published Modal image like arch-nix. Attaching to a running
31- # Sandbox can, and that Sandbox is this container: same image, same
32- # volumes, same env.
29+ # `modal shell --image` takes a registry reference and nothing else, so
30+ # it cannot be pointed at the image this container actually runs -- the
31+ # one with its setup steps and its source copy in it. Attaching to a
32+ # running Sandbox can, and that Sandbox is this container: same image,
33+ # same volumes, same env.
3334 sb = c.open_sandbox()
3435 print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}")
3536 print(f" attach: modal shell {sb.object_id} (from another terminal)")
similarity index 81%
rename from .modal/flutter-web/container.py
rename to .modal/dev/container.py
@@ -26,10 +26,11 @@ def main(command: str = "", shell: bool = False):
26 c.run_sandbox(command)26 c.run_sandbox(command)
27 return27 return
28 28
29- # `modal shell --image` only takes registry references, so it cannot be29+ # `modal shell --image` takes a registry reference and nothing else, so
30- # pointed at a published Modal image like arch-nix. Attaching to a running30+ # it cannot be pointed at the image this container actually runs -- the
31- # Sandbox can, and that Sandbox is this container: same image, same31+ # one with its setup steps and its source copy in it. Attaching to a
32- # volumes, same env.32+ # running Sandbox can, and that Sandbox is this container: same image,
33+ # same volumes, same env.
33 sb = c.open_sandbox()34 sb = c.open_sandbox()
34 print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}")35 print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}")
35 print(f" attach: modal shell {sb.object_id} (from another terminal)")36 print(f" attach: modal shell {sb.object_id} (from another terminal)")
added .modal/dev/container.toml +140 -0
new file mode 100644
@@ -0,0 +1,140 @@
1+[container]
2+name = "frq-dev"
3+description = "the Flutter desktop build, incremental, from a pinned toolchain"
4+# `debian:13-slim` and not `arch-nix`: there is no nix in this container any
5+# more, and no flake left in the tree for it to enter. The build is
6+# `just build desktop`, which gets Flutter, a JDK, the Clojure CLI and Nim out
7+# of `tools/toolchain.sh` by pinned sha256 -- so what the image owes it is the
8+# C and GTK half Flutter's Linux target links against, and nothing else.
9+registry = "debian:13-slim"
10+# A Sandbox, not a Function: runs on a real VM, and the command is
11+# the sandbox's own process so it dies when the command does.
12+runtime = "sandbox"
13+
14+[build]
15+# The container lives inside the repo it builds, so the copy is rooted two
16+# levels up and `.` is the whole tree.
17+context = "../.."
18+include = ["."]
19+# The whole image build, and it is one apt line. What used to be here -- a nix
20+# store to populate and a devShell to print into /etc/devshell.sh, so that a
21+# shell attached to this container *was* the devShell -- is gone with the nix
22+# it was for. The toolchain script now plays that part, and it lives on the
23+# volume rather than in a layer.
24+#
25+# The first half is what the toolchain itself needs: git, because Flutter
26+# shells out to it against its own SDK checkout and refuses to run without
27+# one; unzip and xz-utils for the tarballs; ca-certificates so curl can verify
28+# what it fetches; rsync for the sync below.
29+#
30+# The second half is Flutter's Linux target: CMake, Ninja and pkg-config drive
31+# the build, GTK 3 is what the runner links, and a C toolchain compiles both
32+# that and whatever `nim c` is asked for. libssl is Nim's: nim.cfg is
33+# `-d:ssl`, and std/net resolves -lssl and -lcrypto through dynlib at run time.
34+setup = [
35+ "apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git rsync tar unzip xz-utils && rm -rf /var/lib/apt/lists/*",
36+ "apt-get update && apt-get install -y --no-install-recommends build-essential clang cmake ninja-build pkg-config libgtk-3-dev liblzma-dev libssl-dev && rm -rf /var/lib/apt/lists/*",
37+]
38+# The build state a local checkout carries: 395MB of a 441MB repo, uploaded on
39+# every start and wanted by nothing out there. Flutter builds into a volume of
40+# its own, and the clojure caches are this machine's.
41+ignore = [
42+ "flutter/build", "flutter/.home", "flutter/.dart_tool",
43+ "flutter/.clojuredart", "flutter/.cpcache",
44+ # The ClojureDart compiler's output. Uploading a laptop's copy would make
45+ # the rsync below overwrite the one the last container compiled, and every
46+ # file whose content differed would look new to the compiler and to
47+ # Flutter -- the incremental build undone by the thing meant to feed it.
48+ "flutter/lib/cljd-out",
49+ # The toolchain, which is a gigabyte of Flutter SDK and lives on the
50+ # volume out here.
51+ ".toolchain",
52+ ".cpcache", "result", "build", ".git",
53+ # An editor's linter rewrites this while the upload is reading it, and
54+ # Modal fails the whole run with "was modified during build process".
55+ ".clj-kondo",
56+]
57+
58+# One volume now, where there were two: the nix binary cache went with nix.
59+# `devshell` is the working state of an incremental loop, shared by every
60+# container that has one -- each gets its own directory under it, named for
61+# what it belongs to, so `web` and `flutter-desktop` never write the
62+# same tree. Modal Volumes have no locking, so those directory names are the
63+# only thing keeping them apart, and two runs of the *same* container must not
64+# overlap.
65+[volumes]
66+devshell = "/devshell"
67+
68+[resources]
69+cpu = 8
70+memory = 16384
71+timeout = 3600
72+
73+[run]
74+workdir = "/app"
75+# Source in, toolchain out of the volume, build in place. None of it
76+# evaluates anything: the old command spent its first minutes entering a
77+# devShell, printing an environment, caching that environment against
78+# flake.lock and copying a nix closure back afterwards, all to arrive at a
79+# PATH. A PATH is what `tools/toolchain.sh env` prints, out of a directory
80+# already on the volume.
81+#
82+# An incremental build is the point, and `nix build` could not give one: a
83+# derivation is all-or-nothing, so any edit is a fresh sandbox and a fresh
84+# compile of everything. Here the toolchain supplies the compiler and
85+# `flutter build` decides what is stale -- which is the whole reason
86+# `just build desktop` exists as the working-tree loop.
87+command = """
88+set -e
89+SHELL_DIR=/devshell/frq-desktop
90+
91+# Beside the working tree and NOT inside it: the rsync below runs with
92+# --delete, so anything under $SHELL_DIR that is not in /app is removed on
93+# every run. A cache kept in there would be deleted moments before it was
94+# consulted.
95+export FRQ_TOOLCHAIN=/devshell/frq-desktop.toolchain
96+mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN"
97+
98+echo "sync: /app -> $SHELL_DIR"
99+# rsync and not cp, with --checksum and not mtimes: Modal copies the source in
100+# with fresh timestamps every run, so a plain copy looks entirely new to
101+# Flutter and rebuilds the lot. --checksum compares content and leaves the
102+# unchanged files' timestamps alone, which is the whole basis of the
103+# incremental build.
104+#
105+# The excludes are the state we are here to keep -- overwriting them from /app
106+# would defeat the volume.
107+rsync -a --checksum --delete \
108+ --exclude 'flutter/.home/' \
109+ --exclude 'flutter/.clojuredart/' \
110+ --exclude 'flutter/build/' \
111+ --exclude 'flutter/lib/cljd-out/' \
112+ --exclude 'flutter/.dart_tool/' \
113+ --exclude '.toolchain/' \
114+ --exclude '.git' \
115+ /app/ "$SHELL_DIR/"
116+
117+cd "$SHELL_DIR"
118+# What survived from the last run, by presence and not by size: `du` here
119+# walked the pub cache, the toolchain and every object of the last build over
120+# a network volume, for numbers nobody acts on.
121+for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/build; do
122+ [ -d "$d" ] && echo " carried over: $d"
123+done
124+
125+# Run from the volume, where the state it reuses lives. `just` is not in this
126+# image and is not worth an apt line for one call: the recipe is a wrapper
127+# around the toolchain, and this is that wrapper.
128+tools/toolchain.sh exec -- bash -euo pipefail -c '
129+ cd flutter
130+ clojure -Sdeps "{:mvn/local-repo \\"$FRQ_M2\\"}" -M:cljd compile
131+ flutter build linux --debug'
132+
133+echo "built:"
134+du -sh flutter/build
135+"""
136+# Flutter keeps its settings under XDG_CONFIG_HOME and its own caches under
137+# XDG_CACHE_HOME. Both point into the volume so a second run finds what the
138+# first one decided. Set here rather than in the command so a shell into this
139+# container gets them too.
140+env = { XDG_CACHE_HOME = "/devshell/frq-desktop.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-desktop.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-desktop.toolchain" }
new file mode 100644
@@ -0,0 +1,140 @@
1+[container]
2+name = "frq-dev"
3+description = "the Flutter desktop build, incremental, from a pinned toolchain"
4+# `debian:13-slim` and not `arch-nix`: there is no nix in this container any
5+# more, and no flake left in the tree for it to enter. The build is
6+# `just build desktop`, which gets Flutter, a JDK, the Clojure CLI and Nim out
7+# of `tools/toolchain.sh` by pinned sha256 -- so what the image owes it is the
8+# C and GTK half Flutter's Linux target links against, and nothing else.
9+registry = "debian:13-slim"
10+# A Sandbox, not a Function: runs on a real VM, and the command is
11+# the sandbox's own process so it dies when the command does.
12+runtime = "sandbox"
13+
14+[build]
15+# The container lives inside the repo it builds, so the copy is rooted two
16+# levels up and `.` is the whole tree.
17+context = "../.."
18+include = ["."]
19+# The whole image build, and it is one apt line. What used to be here -- a nix
20+# store to populate and a devShell to print into /etc/devshell.sh, so that a
21+# shell attached to this container *was* the devShell -- is gone with the nix
22+# it was for. The toolchain script now plays that part, and it lives on the
23+# volume rather than in a layer.
24+#
25+# The first half is what the toolchain itself needs: git, because Flutter
26+# shells out to it against its own SDK checkout and refuses to run without
27+# one; unzip and xz-utils for the tarballs; ca-certificates so curl can verify
28+# what it fetches; rsync for the sync below.
29+#
30+# The second half is Flutter's Linux target: CMake, Ninja and pkg-config drive
31+# the build, GTK 3 is what the runner links, and a C toolchain compiles both
32+# that and whatever `nim c` is asked for. libssl is Nim's: nim.cfg is
33+# `-d:ssl`, and std/net resolves -lssl and -lcrypto through dynlib at run time.
34+setup = [
35+ "apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git rsync tar unzip xz-utils && rm -rf /var/lib/apt/lists/*",
36+ "apt-get update && apt-get install -y --no-install-recommends build-essential clang cmake ninja-build pkg-config libgtk-3-dev liblzma-dev libssl-dev && rm -rf /var/lib/apt/lists/*",
37+]
38+# The build state a local checkout carries: 395MB of a 441MB repo, uploaded on
39+# every start and wanted by nothing out there. Flutter builds into a volume of
40+# its own, and the clojure caches are this machine's.
41+ignore = [
42+ "flutter/build", "flutter/.home", "flutter/.dart_tool",
43+ "flutter/.clojuredart", "flutter/.cpcache",
44+ # The ClojureDart compiler's output. Uploading a laptop's copy would make
45+ # the rsync below overwrite the one the last container compiled, and every
46+ # file whose content differed would look new to the compiler and to
47+ # Flutter -- the incremental build undone by the thing meant to feed it.
48+ "flutter/lib/cljd-out",
49+ # The toolchain, which is a gigabyte of Flutter SDK and lives on the
50+ # volume out here.
51+ ".toolchain",
52+ ".cpcache", "result", "build", ".git",
53+ # An editor's linter rewrites this while the upload is reading it, and
54+ # Modal fails the whole run with "was modified during build process".
55+ ".clj-kondo",
56+]
57+
58+# One volume now, where there were two: the nix binary cache went with nix.
59+# `devshell` is the working state of an incremental loop, shared by every
60+# container that has one -- each gets its own directory under it, named for
61+# what it belongs to, so `web` and `flutter-desktop` never write the
62+# same tree. Modal Volumes have no locking, so those directory names are the
63+# only thing keeping them apart, and two runs of the *same* container must not
64+# overlap.
65+[volumes]
66+devshell = "/devshell"
67+
68+[resources]
69+cpu = 8
70+memory = 16384
71+timeout = 3600
72+
73+[run]
74+workdir = "/app"
75+# Source in, toolchain out of the volume, build in place. None of it
76+# evaluates anything: the old command spent its first minutes entering a
77+# devShell, printing an environment, caching that environment against
78+# flake.lock and copying a nix closure back afterwards, all to arrive at a
79+# PATH. A PATH is what `tools/toolchain.sh env` prints, out of a directory
80+# already on the volume.
81+#
82+# An incremental build is the point, and `nix build` could not give one: a
83+# derivation is all-or-nothing, so any edit is a fresh sandbox and a fresh
84+# compile of everything. Here the toolchain supplies the compiler and
85+# `flutter build` decides what is stale -- which is the whole reason
86+# `just build desktop` exists as the working-tree loop.
87+command = """
88+set -e
89+SHELL_DIR=/devshell/frq-desktop
90+
91+# Beside the working tree and NOT inside it: the rsync below runs with
92+# --delete, so anything under $SHELL_DIR that is not in /app is removed on
93+# every run. A cache kept in there would be deleted moments before it was
94+# consulted.
95+export FRQ_TOOLCHAIN=/devshell/frq-desktop.toolchain
96+mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN"
97+
98+echo "sync: /app -> $SHELL_DIR"
99+# rsync and not cp, with --checksum and not mtimes: Modal copies the source in
100+# with fresh timestamps every run, so a plain copy looks entirely new to
101+# Flutter and rebuilds the lot. --checksum compares content and leaves the
102+# unchanged files' timestamps alone, which is the whole basis of the
103+# incremental build.
104+#
105+# The excludes are the state we are here to keep -- overwriting them from /app
106+# would defeat the volume.
107+rsync -a --checksum --delete \
108+ --exclude 'flutter/.home/' \
109+ --exclude 'flutter/.clojuredart/' \
110+ --exclude 'flutter/build/' \
111+ --exclude 'flutter/lib/cljd-out/' \
112+ --exclude 'flutter/.dart_tool/' \
113+ --exclude '.toolchain/' \
114+ --exclude '.git' \
115+ /app/ "$SHELL_DIR/"
116+
117+cd "$SHELL_DIR"
118+# What survived from the last run, by presence and not by size: `du` here
119+# walked the pub cache, the toolchain and every object of the last build over
120+# a network volume, for numbers nobody acts on.
121+for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/build; do
122+ [ -d "$d" ] && echo " carried over: $d"
123+done
124+
125+# Run from the volume, where the state it reuses lives. `just` is not in this
126+# image and is not worth an apt line for one call: the recipe is a wrapper
127+# around the toolchain, and this is that wrapper.
128+tools/toolchain.sh exec -- bash -euo pipefail -c '
129+ cd flutter
130+ clojure -Sdeps "{:mvn/local-repo \\"$FRQ_M2\\"}" -M:cljd compile
131+ flutter build linux --debug'
132+
133+echo "built:"
134+du -sh flutter/build
135+"""
136+# Flutter keeps its settings under XDG_CONFIG_HOME and its own caches under
137+# XDG_CACHE_HOME. Both point into the volume so a second run finds what the
138+# first one decided. Set here rather than in the command so a shell into this
139+# container gets them too.
140+env = { XDG_CACHE_HOME = "/devshell/frq-desktop.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-desktop.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-desktop.toolchain" }
deleted .modal/flutter-dev/README.md +0 -16
deleted file mode 100644
@@ -1,16 +0,0 @@
1-# `flutter-dev`
2-
3- scripts/deploy flutter-dev
4- modal run .modal/flutter-dev/container.py
5-
6-Defined by `container.toml`; see `../spec.md` for the keys.
7-Built on the published `arch-nix` image.
8-
9-Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command
10-is the sandbox's own process, so it dies when the command exits --
11-no idle window and nothing to tear down. Note the VM restrictions:
12-no GPU, and memory is exactly what `[resources] memory` asks for.
13-
14-No nix at run time: nothing is substituted at build time and nothing
15-is evaluated at start. Add a `flake.nix` and set `[nix] flake`/`shim`
16-together if you want a devShell, knowing what it costs.
deleted file mode 100644
@@ -1,16 +0,0 @@
1-# `flutter-dev`
2-
3- scripts/deploy flutter-dev
4- modal run .modal/flutter-dev/container.py
5-
6-Defined by `container.toml`; see `../spec.md` for the keys.
7-Built on the published `arch-nix` image.
8-
9-Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command
10-is the sandbox's own process, so it dies when the command exits --
11-no idle window and nothing to tear down. Note the VM restrictions:
12-no GPU, and memory is exactly what `[resources] memory` asks for.
13-
14-No nix at run time: nothing is substituted at build time and nothing
15-is evaluated at start. Add a `flake.nix` and set `[nix] flake`/`shim`
16-together if you want a devShell, knowing what it costs.
deleted .modal/flutter-dev/container.toml +0 -160
deleted file mode 100644
@@ -1,160 +0,0 @@
1-[container]
2-name = "frq-flutter-dev"
3-description = "the Flutter desktop build, incremental, in a nix devShell"
4-base = "arch-nix"
5-# A Sandbox, not a Function: runs on a real VM, and the command is
6-# the sandbox's own process so it dies when the command does.
7-runtime = "sandbox"
8-
9-[build]
10-# The container lives inside the repo it builds, so the copy is rooted two
11-# levels up and `.` is the whole tree.
12-context = "../.."
13-include = ["."]
14-# `dev` is the shell you actually want. There is no default shell in this
15-# flake any more -- the one that used to be there belonged to the retired
16-# libcosmic frontend -- so a bare `nix develop` here fails rather than
17-# resolving to the wrong tree.
18-# The devShell, baked in rather than entered. `print-dev-env` writes the whole
19-# environment out as shell -- PATH, the compiler, every variable mkShell sets
20-# -- and realises its inputs on the way, so the closure becomes an image layer
21-# instead of a fetch every container pays for. Sourcing it from .bashrc means a
22-# shell attached to this container *is* the devShell: no `nix develop`, no
23-# clone of the flake's git inputs, no wait.
24-#
25-# `dev` stays for the case where the baked env is stale against a flake edit.
26-commands = [
27- "nix print-dev-env /app#flutter-desktop --accept-flake-config --extra-substituters file:///nix-cache > /etc/devshell.sh",
28- "echo '. /etc/devshell.sh' >> /root/.bashrc",
29- "printf '#!/bin/sh\\nexec nix develop /app#flutter-desktop \"$@\"\\n' > /usr/local/bin/dev && chmod +x /usr/local/bin/dev",
30-]
31-# The build state a local checkout carries: 395MB of a 441MB repo, uploaded on
32-# every start and wanted by nothing out there. Flutter builds into a volume of
33-# its own, and the clojure caches are this machine's.
34-ignore = [
35- "flutter/build", "flutter/.home", "flutter/.dart_tool",
36- "flutter/.clojuredart", "flutter/.cpcache",
37- ".cpcache", "result", "build", ".git",
38-]
39-
40-# Two volumes doing two different jobs. `nix-cache` is the binary cache every
41-# container here reads from and writes back to. `devshell` is the working
42-# state of a `nix develop` loop, and it is shared by every container that has
43-# one -- each gets its own directory under it, named for the devShell it
44-# belongs to, so two projects (or two shells of one project) never write the
45-# same tree. Modal Volumes have no locking, so the directories are the only
46-# thing keeping them apart, and two runs of the *same* devshell must not
47-# overlap.
48-[volumes]
49-nix-cache = "/nix-cache"
50-devshell = "/devshell"
51-
52-[resources]
53-cpu = 8
54-memory = 16384
55-timeout = 3600
56-
57-[run]
58-workdir = "/app"
59-# Nix for the dependencies, the ordinary toolchain for the build. `nix build`
60-# cannot do this: a derivation is all-or-nothing, so any edit is a fresh
61-# sandbox and a fresh compile of everything. Here the devShell supplies the
62-# compiler and the libraries, and `flutter build` decides what is stale --
63-# which is the whole reason `just flutter-desktop` exists as the working-tree
64-# loop rather than as another `nix build`.
65-#
66-# rsync and not cp, with --checksum and not mtimes: Modal copies the source in
67-# with fresh timestamps on every run, so a plain copy would look entirely new
68-# to Flutter and rebuild the lot. --checksum compares content, leaves the
69-# unchanged files' timestamps alone, and lets the incremental build work.
70-#
71-# The excludes are the state that must NOT be overwritten from /app -- it is
72-# what we are here to keep. `just flutter-desktop` seeds those caches only
73-# when they are missing, so finding them warm is all it takes.
74-command = """
75-set -e
76-# This container's own directory on the shared devshell volume, named for the
77-# devShell it keeps the state of. Anything else using this volume picks its
78-# own name and the two never meet.
79-SHELL_DIR=/devshell/frq-flutter-desktop
80-mkdir -p "$SHELL_DIR" "$SHELL_DIR/.cache"
81-
82-# A worktree's `.git` is a *file* naming a gitdir back on the machine that
83-# copied it in, and nix believes it and goes looking for a path that is not
84-# here. It has to go before any flake reference to /app.
85-rm -rf /app/.git
86-
87-echo "sync: /app -> $SHELL_DIR"
88-# `nix shell --command` and not `nix profile install`: a profile install puts
89-# rsync in ~/.nix-profile/bin, which is not on the PATH of the shell already
90-# running, so the very next line said `rsync: command not found`.
91-#
92-# rsync and not cp, with --checksum and not mtimes: Modal copies the source in
93-# with fresh timestamps every run, so a plain copy looks entirely new to
94-# Flutter and rebuilds the lot. --checksum compares content and leaves the
95-# unchanged files' timestamps alone, which is the whole basis of the
96-# incremental build.
97-#
98-# The excludes are the state we are here to keep -- overwriting them from /app
99-# would defeat the volume. `just flutter-desktop` seeds those caches only when
100-# they are missing, so finding them warm is all it takes.
101-nix shell nixpkgs#rsync --accept-flake-config \
102- --extra-substituters file:///nix-cache --command \
103- rsync -a --checksum --delete \
104- --exclude 'flutter/.home/' \
105- --exclude 'flutter/.clojuredart/' \
106- --exclude 'flutter/build/' \
107- --exclude 'flutter/.dart_tool/' \
108- --exclude '.git' \
109- /app/ "$SHELL_DIR/"
110-
111-cd "$SHELL_DIR"
112-echo "state carried over:"
113-du -sh flutter/.home flutter/.clojuredart flutter/build 2>/dev/null \
114- || echo " (none yet -- first run)"
115-
116-# Nix for the dependencies, the ordinary toolchain for the build. `nix build`
117-# cannot do this: a derivation is all-or-nothing, so any edit is a fresh
118-# sandbox and a fresh compile of everything. Here the devShell supplies the
119-# compiler and the libraries and `flutter build` decides what is stale.
120-# Evaluated from /app and built in the volume. Both halves matter: /app is the
121-# pristine copy, so nix stores a source tree of the repo rather than one
122-# carrying gigabytes of flutter/build, while the recipe still runs where the
123-# state it reuses lives -- `just -f` is what puts it there, since the recipe
124-# cds to its own justfile's directory.
125-nix develop /app#flutter-desktop --accept-flake-config \
126- --extra-substituters file:///nix-cache \
127- --max-jobs auto --command just -f "$SHELL_DIR/justfile" flutter-desktop
128-
129-echo "built:"
130-du -sh flutter/build
131-
132-# The devShell's closure is gigabytes of Flutter, Dart, clang and GTK, and the
133-# store it landed in belongs to the image rather than to a volume -- so
134-# without this every run re-fetches it from upstream. Written back, the next
135-# run substitutes it from file:///nix-cache instead.
136-if [ -f /nix-cache/nix-cache-info ]; then
137- echo "cache: writing the devShell closure back"
138- nix copy --no-check-sigs --all --to file:///nix-cache
139-fi
140-"""
141-# Nix's own cache, on the volume rather than in the container. Without it
142-# every Sandbox starts empty and `nix develop` re-clones the flake's git
143-# inputs -- nixgl and its transitives --
144-# because flake.lock pins which revision to fetch, not whether it is already
145-# on disk. Set here rather than in the command so an interactive shell into
146-# this container gets it too.
147-env = { XDG_CACHE_HOME = "/devshell/frq-flutter-desktop/.cache" }
148-
149-[nix]
150-# Every nix command in the container reads the mounted cache, including one
151-# typed by hand in a shell. Passing --extra-substituters per command only ever
152-# covered the scripts.
153-substituters = ["file:///nix-cache"]
154-# No devShell warming at image build time: this enters `nix develop` at run
155-# time, on the VM, where the cache answers for its closure. The ptyshim that
156-# warming would need under gVisor is deprecated and does not come back.
157-flake = false
158-shim = false
159-
160-# [experimental] overrides the sandbox default of vm_runtime = true.
deleted file mode 100644
@@ -1,160 +0,0 @@
1-[container]
2-name = "frq-flutter-dev"
3-description = "the Flutter desktop build, incremental, in a nix devShell"
4-base = "arch-nix"
5-# A Sandbox, not a Function: runs on a real VM, and the command is
6-# the sandbox's own process so it dies when the command does.
7-runtime = "sandbox"
8-
9-[build]
10-# The container lives inside the repo it builds, so the copy is rooted two
11-# levels up and `.` is the whole tree.
12-context = "../.."
13-include = ["."]
14-# `dev` is the shell you actually want. There is no default shell in this
15-# flake any more -- the one that used to be there belonged to the retired
16-# libcosmic frontend -- so a bare `nix develop` here fails rather than
17-# resolving to the wrong tree.
18-# The devShell, baked in rather than entered. `print-dev-env` writes the whole
19-# environment out as shell -- PATH, the compiler, every variable mkShell sets
20-# -- and realises its inputs on the way, so the closure becomes an image layer
21-# instead of a fetch every container pays for. Sourcing it from .bashrc means a
22-# shell attached to this container *is* the devShell: no `nix develop`, no
23-# clone of the flake's git inputs, no wait.
24-#
25-# `dev` stays for the case where the baked env is stale against a flake edit.
26-commands = [
27- "nix print-dev-env /app#flutter-desktop --accept-flake-config --extra-substituters file:///nix-cache > /etc/devshell.sh",
28- "echo '. /etc/devshell.sh' >> /root/.bashrc",
29- "printf '#!/bin/sh\\nexec nix develop /app#flutter-desktop \"$@\"\\n' > /usr/local/bin/dev && chmod +x /usr/local/bin/dev",
30-]
31-# The build state a local checkout carries: 395MB of a 441MB repo, uploaded on
32-# every start and wanted by nothing out there. Flutter builds into a volume of
33-# its own, and the clojure caches are this machine's.
34-ignore = [
35- "flutter/build", "flutter/.home", "flutter/.dart_tool",
36- "flutter/.clojuredart", "flutter/.cpcache",
37- ".cpcache", "result", "build", ".git",
38-]
39-
40-# Two volumes doing two different jobs. `nix-cache` is the binary cache every
41-# container here reads from and writes back to. `devshell` is the working
42-# state of a `nix develop` loop, and it is shared by every container that has
43-# one -- each gets its own directory under it, named for the devShell it
44-# belongs to, so two projects (or two shells of one project) never write the
45-# same tree. Modal Volumes have no locking, so the directories are the only
46-# thing keeping them apart, and two runs of the *same* devshell must not
47-# overlap.
48-[volumes]
49-nix-cache = "/nix-cache"
50-devshell = "/devshell"
51-
52-[resources]
53-cpu = 8
54-memory = 16384
55-timeout = 3600
56-
57-[run]
58-workdir = "/app"
59-# Nix for the dependencies, the ordinary toolchain for the build. `nix build`
60-# cannot do this: a derivation is all-or-nothing, so any edit is a fresh
61-# sandbox and a fresh compile of everything. Here the devShell supplies the
62-# compiler and the libraries, and `flutter build` decides what is stale --
63-# which is the whole reason `just flutter-desktop` exists as the working-tree
64-# loop rather than as another `nix build`.
65-#
66-# rsync and not cp, with --checksum and not mtimes: Modal copies the source in
67-# with fresh timestamps on every run, so a plain copy would look entirely new
68-# to Flutter and rebuild the lot. --checksum compares content, leaves the
69-# unchanged files' timestamps alone, and lets the incremental build work.
70-#
71-# The excludes are the state that must NOT be overwritten from /app -- it is
72-# what we are here to keep. `just flutter-desktop` seeds those caches only
73-# when they are missing, so finding them warm is all it takes.
74-command = """
75-set -e
76-# This container's own directory on the shared devshell volume, named for the
77-# devShell it keeps the state of. Anything else using this volume picks its
78-# own name and the two never meet.
79-SHELL_DIR=/devshell/frq-flutter-desktop
80-mkdir -p "$SHELL_DIR" "$SHELL_DIR/.cache"
81-
82-# A worktree's `.git` is a *file* naming a gitdir back on the machine that
83-# copied it in, and nix believes it and goes looking for a path that is not
84-# here. It has to go before any flake reference to /app.
85-rm -rf /app/.git
86-
87-echo "sync: /app -> $SHELL_DIR"
88-# `nix shell --command` and not `nix profile install`: a profile install puts
89-# rsync in ~/.nix-profile/bin, which is not on the PATH of the shell already
90-# running, so the very next line said `rsync: command not found`.
91-#
92-# rsync and not cp, with --checksum and not mtimes: Modal copies the source in
93-# with fresh timestamps every run, so a plain copy looks entirely new to
94-# Flutter and rebuilds the lot. --checksum compares content and leaves the
95-# unchanged files' timestamps alone, which is the whole basis of the
96-# incremental build.
97-#
98-# The excludes are the state we are here to keep -- overwriting them from /app
99-# would defeat the volume. `just flutter-desktop` seeds those caches only when
100-# they are missing, so finding them warm is all it takes.
101-nix shell nixpkgs#rsync --accept-flake-config \
102- --extra-substituters file:///nix-cache --command \
103- rsync -a --checksum --delete \
104- --exclude 'flutter/.home/' \
105- --exclude 'flutter/.clojuredart/' \
106- --exclude 'flutter/build/' \
107- --exclude 'flutter/.dart_tool/' \
108- --exclude '.git' \
109- /app/ "$SHELL_DIR/"
110-
111-cd "$SHELL_DIR"
112-echo "state carried over:"
113-du -sh flutter/.home flutter/.clojuredart flutter/build 2>/dev/null \
114- || echo " (none yet -- first run)"
115-
116-# Nix for the dependencies, the ordinary toolchain for the build. `nix build`
117-# cannot do this: a derivation is all-or-nothing, so any edit is a fresh
118-# sandbox and a fresh compile of everything. Here the devShell supplies the
119-# compiler and the libraries and `flutter build` decides what is stale.
120-# Evaluated from /app and built in the volume. Both halves matter: /app is the
121-# pristine copy, so nix stores a source tree of the repo rather than one
122-# carrying gigabytes of flutter/build, while the recipe still runs where the
123-# state it reuses lives -- `just -f` is what puts it there, since the recipe
124-# cds to its own justfile's directory.
125-nix develop /app#flutter-desktop --accept-flake-config \
126- --extra-substituters file:///nix-cache \
127- --max-jobs auto --command just -f "$SHELL_DIR/justfile" flutter-desktop
128-
129-echo "built:"
130-du -sh flutter/build
131-
132-# The devShell's closure is gigabytes of Flutter, Dart, clang and GTK, and the
133-# store it landed in belongs to the image rather than to a volume -- so
134-# without this every run re-fetches it from upstream. Written back, the next
135-# run substitutes it from file:///nix-cache instead.
136-if [ -f /nix-cache/nix-cache-info ]; then
137- echo "cache: writing the devShell closure back"
138- nix copy --no-check-sigs --all --to file:///nix-cache
139-fi
140-"""
141-# Nix's own cache, on the volume rather than in the container. Without it
142-# every Sandbox starts empty and `nix develop` re-clones the flake's git
143-# inputs -- nixgl and its transitives --
144-# because flake.lock pins which revision to fetch, not whether it is already
145-# on disk. Set here rather than in the command so an interactive shell into
146-# this container gets it too.
147-env = { XDG_CACHE_HOME = "/devshell/frq-flutter-desktop/.cache" }
148-
149-[nix]
150-# Every nix command in the container reads the mounted cache, including one
151-# typed by hand in a shell. Passing --extra-substituters per command only ever
152-# covered the scripts.
153-substituters = ["file:///nix-cache"]
154-# No devShell warming at image build time: this enters `nix develop` at run
155-# time, on the VM, where the cache answers for its closure. The ptyshim that
156-# warming would need under gVisor is deprecated and does not come back.
157-flake = false
158-shim = false
159-
160-# [experimental] overrides the sandbox default of vm_runtime = true.
renamed .modal/web/README.md +13 -12
similarity index 82%
rename from .modal/flutter-web/README.md
rename to .modal/web/README.md
@@ -1,9 +1,10 @@
1-# `flutter-web`
1+# `web`
22
3- modal run .modal/flutter-web/container.py
4- just modal flutter-web
3+ modal run .modal/web/container.py
4+ just modal web
55
6-Defined by `container.toml`; see `../spec.md` for the keys.
6+Defined by `container.toml`; `../_loader.py` is what reads it, and
7+its comments are the spec.
78 Built on `debian:13-slim`.
89
910 No nix, and that is the point of this container rather than an
@@ -18,14 +19,14 @@ copying a nix closure back to a volume afterwards -- does not happen
1819 at all. The toolchain lands on the volume and the second run finds
1920 it there.
2021
21-The other two Flutter targets keep their devShells: `apk` needs the
22-Android SDK and `flutter-desktop` needs GTK and a C++ toolchain, and
23-a host toolchain is what nix is better at than a tarball. The web
24-target needs a Dart and a JVM, which is what a tarball is for.
22+The `dev` container beside it works the same way now, and so does a
23+laptop: one script, one pinned set of tarballs, and whatever the
24+host has to bring for a given target -- GTK and a C++ toolchain for
25+the desktop build, Google's command-line tools for the APK.
2526
2627 Same incremental shape as before -- the working tree, the generated
2728 Dart under `flutter/lib/cljd-out` and Flutter's caches live on the
28-`devshell` volume, under `frq-flutter-web/` so the desktop
29+`devshell` volume, under `frq-web/` so the desktop
2930 container's directory beside it is untouched. None of them is copied
3031 in from the laptop: a checkout's copy of the compiler's output is
3132 not this container's, and overwriting the volume's with it is how an
@@ -44,8 +45,8 @@ To look at what it built, ask for the serve action -- `[network]
4445 ports` tunnels 8080 out, and the URL is printed once the sandbox is
4546 scheduled:
4647
47- modal run .modal/flutter-web/container.py \
48- --command 'cd /devshell/frq-flutter-web && tools/build-web.sh serve 8080'
48+ modal run .modal/web/container.py \
49+ --command 'cd /devshell/frq-web && tools/build-web.sh serve 8080'
4950
5051 That blocks until you Ctrl-C it, and it bills until you do.
5152
@@ -73,7 +74,7 @@ browser has no raw socket, so the IRC connection wants a WebSocket.
7374 and hands out `flutter/build/web`, so a rebuild in the Sandbox is
7475 picked up by the next cold start with nothing redeployed.
7576
76- modal deploy .modal/flutter-web/serve.py
77+ modal deploy .modal/web/serve.py
7778
7879 `[network] ports` tunnels 8080 out of the Sandbox as well, for the
7980 case where you want the build and the server to be one process.
similarity index 82%
rename from .modal/flutter-web/README.md
rename to .modal/web/README.md
@@ -1,9 +1,10 @@
1-# `flutter-web`1+# `web`
2 2
3- modal run .modal/flutter-web/container.py3+ modal run .modal/web/container.py
4- just modal flutter-web4+ just modal web
5 5
6-Defined by `container.toml`; see `../spec.md` for the keys.6+Defined by `container.toml`; `../_loader.py` is what reads it, and
7+its comments are the spec.
7 Built on `debian:13-slim`.8 Built on `debian:13-slim`.
8 9
9 No nix, and that is the point of this container rather than an10 No nix, and that is the point of this container rather than an
@@ -18,14 +19,14 @@ copying a nix closure back to a volume afterwards -- does not happen
18 at all. The toolchain lands on the volume and the second run finds19 at all. The toolchain lands on the volume and the second run finds
19 it there.20 it there.
20 21
21-The other two Flutter targets keep their devShells: `apk` needs the22+The `dev` container beside it works the same way now, and so does a
22-Android SDK and `flutter-desktop` needs GTK and a C++ toolchain, and23+laptop: one script, one pinned set of tarballs, and whatever the
23-a host toolchain is what nix is better at than a tarball. The web24+host has to bring for a given target -- GTK and a C++ toolchain for
24-target needs a Dart and a JVM, which is what a tarball is for.25+the desktop build, Google's command-line tools for the APK.
25 26
26 Same incremental shape as before -- the working tree, the generated27 Same incremental shape as before -- the working tree, the generated
27 Dart under `flutter/lib/cljd-out` and Flutter's caches live on the28 Dart under `flutter/lib/cljd-out` and Flutter's caches live on the
28-`devshell` volume, under `frq-flutter-web/` so the desktop29+`devshell` volume, under `frq-web/` so the desktop
29 container's directory beside it is untouched. None of them is copied30 container's directory beside it is untouched. None of them is copied
30 in from the laptop: a checkout's copy of the compiler's output is31 in from the laptop: a checkout's copy of the compiler's output is
31 not this container's, and overwriting the volume's with it is how an32 not this container's, and overwriting the volume's with it is how an
@@ -44,8 +45,8 @@ To look at what it built, ask for the serve action -- `[network]
44 ports` tunnels 8080 out, and the URL is printed once the sandbox is45 ports` tunnels 8080 out, and the URL is printed once the sandbox is
45 scheduled:46 scheduled:
46 47
47- modal run .modal/flutter-web/container.py \48+ modal run .modal/web/container.py \
48- --command 'cd /devshell/frq-flutter-web && tools/build-web.sh serve 8080'49+ --command 'cd /devshell/frq-web && tools/build-web.sh serve 8080'
49 50
50 That blocks until you Ctrl-C it, and it bills until you do.51 That blocks until you Ctrl-C it, and it bills until you do.
51 52
@@ -73,7 +74,7 @@ browser has no raw socket, so the IRC connection wants a WebSocket.
73 and hands out `flutter/build/web`, so a rebuild in the Sandbox is74 and hands out `flutter/build/web`, so a rebuild in the Sandbox is
74 picked up by the next cold start with nothing redeployed.75 picked up by the next cold start with nothing redeployed.
75 76
76- modal deploy .modal/flutter-web/serve.py77+ modal deploy .modal/web/serve.py
77 78
78 `[network] ports` tunnels 8080 out of the Sandbox as well, for the79 `[network] ports` tunnels 8080 out of the Sandbox as well, for the
79 case where you want the build and the server to be one process.80 case where you want the build and the server to be one process.
renamed .modal/web/container.py +5 -4
similarity index 81%
rename from .modal/flutter-dev/container.py
rename to .modal/web/container.py
@@ -26,10 +26,11 @@ def main(command: str = "", shell: bool = False):
2626 c.run_sandbox(command)
2727 return
2828
29- # `modal shell --image` only takes registry references, so it cannot be
30- # pointed at a published Modal image like arch-nix. Attaching to a running
31- # Sandbox can, and that Sandbox is this container: same image, same
32- # volumes, same env.
29+ # `modal shell --image` takes a registry reference and nothing else, so
30+ # it cannot be pointed at the image this container actually runs -- the
31+ # one with its setup steps and its source copy in it. Attaching to a
32+ # running Sandbox can, and that Sandbox is this container: same image,
33+ # same volumes, same env.
3334 sb = c.open_sandbox()
3435 print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}")
3536 print(f" attach: modal shell {sb.object_id} (from another terminal)")
similarity index 81%
rename from .modal/flutter-dev/container.py
rename to .modal/web/container.py
@@ -26,10 +26,11 @@ def main(command: str = "", shell: bool = False):
26 c.run_sandbox(command)26 c.run_sandbox(command)
27 return27 return
28 28
29- # `modal shell --image` only takes registry references, so it cannot be29+ # `modal shell --image` takes a registry reference and nothing else, so
30- # pointed at a published Modal image like arch-nix. Attaching to a running30+ # it cannot be pointed at the image this container actually runs -- the
31- # Sandbox can, and that Sandbox is this container: same image, same31+ # one with its setup steps and its source copy in it. Attaching to a
32- # volumes, same env.32+ # running Sandbox can, and that Sandbox is this container: same image,
33+ # same volumes, same env.
33 sb = c.open_sandbox()34 sb = c.open_sandbox()
34 print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}")35 print(f"sandbox {sb.object_id} up, with {', '.join(c.volumes) or 'no volumes'}")
35 print(f" attach: modal shell {sb.object_id} (from another terminal)")36 print(f" attach: modal shell {sb.object_id} (from another terminal)")
renamed .modal/web/container.toml +6 -6
similarity index 92%
rename from .modal/flutter-web/container.toml
rename to .modal/web/container.toml
@@ -1,5 +1,5 @@
11 [container]
2-name = "frq-flutter-web"
2+name = "frq-web"
33 description = "the Flutter web build, incremental, from a pinned toolchain"
44 # `debian:13-slim` and not `arch-nix`: there is no nix in this container any
55 # more. The build is `tools/build-web.sh`, which fetches its own Flutter, JDK
@@ -57,7 +57,7 @@ ignore = [
5757 # One volume now, where there were two: the nix binary cache went with nix.
5858 # `devshell` is the working state of an incremental loop, shared by every
5959 # container that has one -- each gets its own directory under it, named for
60-# what it belongs to, so `flutter-web` and `flutter-desktop` never write the
60+# what it belongs to, so `web` and `flutter-desktop` never write the
6161 # same tree. Modal Volumes have no locking, so those directory names are the
6262 # only thing keeping them apart, and two runs of the *same* container must
6363 # not overlap.
@@ -87,13 +87,13 @@ workdir = "/app"
8787 # directory that is already on the volume.
8888 command = """
8989 set -e
90-SHELL_DIR=/devshell/frq-flutter-web
90+SHELL_DIR=/devshell/frq-web
9191
9292 # Beside the working tree and NOT inside it: the rsync below runs with
9393 # --delete, so anything under $SHELL_DIR that is not in /app is removed on
9494 # every run. A cache kept in there would be deleted moments before it was
9595 # consulted -- which is what happened to the last one that tried.
96-export FRQ_TOOLCHAIN=/devshell/frq-flutter-web.toolchain
96+export FRQ_TOOLCHAIN=/devshell/frq-web.toolchain
9797 mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN"
9898
9999 echo "sync: /app -> $SHELL_DIR"
@@ -125,7 +125,7 @@ for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/buil
125125 [ -d "$d" ] && echo " carried over: $d"
126126 done
127127
128-# The same script `just flutter-web` runs, with the same single build mode:
128+# The same script `just build web` runs, with the same single build mode:
129129 # what is served and what a laptop compiles are the same bundle.
130130 tools/build-web.sh build
131131 """
@@ -133,4 +133,4 @@ tools/build-web.sh build
133133 # XDG_CONFIG_HOME, and its own caches under XDG_CACHE_HOME. Both point into
134134 # the volume so a second run finds what the first one decided. Set here
135135 # rather than in the command so a shell into this container gets them too.
136-env = { XDG_CACHE_HOME = "/devshell/frq-flutter-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-flutter-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-flutter-web.toolchain" }
136+env = { XDG_CACHE_HOME = "/devshell/frq-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-web.toolchain" }
similarity index 92%
rename from .modal/flutter-web/container.toml
rename to .modal/web/container.toml
@@ -1,5 +1,5 @@
1 [container]1 [container]
2-name = "frq-flutter-web"2+name = "frq-web"
3 description = "the Flutter web build, incremental, from a pinned toolchain"3 description = "the Flutter web build, incremental, from a pinned toolchain"
4 # `debian:13-slim` and not `arch-nix`: there is no nix in this container any4 # `debian:13-slim` and not `arch-nix`: there is no nix in this container any
5 # more. The build is `tools/build-web.sh`, which fetches its own Flutter, JDK5 # more. The build is `tools/build-web.sh`, which fetches its own Flutter, JDK
@@ -57,7 +57,7 @@ ignore = [
57 # One volume now, where there were two: the nix binary cache went with nix.57 # One volume now, where there were two: the nix binary cache went with nix.
58 # `devshell` is the working state of an incremental loop, shared by every58 # `devshell` is the working state of an incremental loop, shared by every
59 # container that has one -- each gets its own directory under it, named for59 # container that has one -- each gets its own directory under it, named for
60-# what it belongs to, so `flutter-web` and `flutter-desktop` never write the60+# what it belongs to, so `web` and `flutter-desktop` never write the
61 # same tree. Modal Volumes have no locking, so those directory names are the61 # same tree. Modal Volumes have no locking, so those directory names are the
62 # only thing keeping them apart, and two runs of the *same* container must62 # only thing keeping them apart, and two runs of the *same* container must
63 # not overlap.63 # not overlap.
@@ -87,13 +87,13 @@ workdir = "/app"
87 # directory that is already on the volume.87 # directory that is already on the volume.
88 command = """88 command = """
89 set -e89 set -e
90-SHELL_DIR=/devshell/frq-flutter-web90+SHELL_DIR=/devshell/frq-web
91 91
92 # Beside the working tree and NOT inside it: the rsync below runs with92 # Beside the working tree and NOT inside it: the rsync below runs with
93 # --delete, so anything under $SHELL_DIR that is not in /app is removed on93 # --delete, so anything under $SHELL_DIR that is not in /app is removed on
94 # every run. A cache kept in there would be deleted moments before it was94 # every run. A cache kept in there would be deleted moments before it was
95 # consulted -- which is what happened to the last one that tried.95 # consulted -- which is what happened to the last one that tried.
96-export FRQ_TOOLCHAIN=/devshell/frq-flutter-web.toolchain96+export FRQ_TOOLCHAIN=/devshell/frq-web.toolchain
97 mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN"97 mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN"
98 98
99 echo "sync: /app -> $SHELL_DIR"99 echo "sync: /app -> $SHELL_DIR"
@@ -125,7 +125,7 @@ for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/buil
125 [ -d "$d" ] && echo " carried over: $d"125 [ -d "$d" ] && echo " carried over: $d"
126 done126 done
127 127
128-# The same script `just flutter-web` runs, with the same single build mode:128+# The same script `just build web` runs, with the same single build mode:
129 # what is served and what a laptop compiles are the same bundle.129 # what is served and what a laptop compiles are the same bundle.
130 tools/build-web.sh build130 tools/build-web.sh build
131 """131 """
@@ -133,4 +133,4 @@ tools/build-web.sh build
133 # XDG_CONFIG_HOME, and its own caches under XDG_CACHE_HOME. Both point into133 # XDG_CONFIG_HOME, and its own caches under XDG_CACHE_HOME. Both point into
134 # the volume so a second run finds what the first one decided. Set here134 # the volume so a second run finds what the first one decided. Set here
135 # rather than in the command so a shell into this container gets them too.135 # rather than in the command so a shell into this container gets them too.
136-env = { XDG_CACHE_HOME = "/devshell/frq-flutter-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-flutter-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-flutter-web.toolchain" }136+env = { XDG_CACHE_HOME = "/devshell/frq-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-web.toolchain" }
renamed .modal/web/serve.py +5 -5
similarity index 97%
rename from .modal/flutter-web/serve.py
rename to .modal/web/serve.py
@@ -2,13 +2,13 @@
22
33 Not part of `container.py`, and not a key in `container.toml`, because it is
44 the other half of a split the build already makes: the Sandbox compiles into
5-`/devshell/frq-flutter-web` and exits, and what it leaves behind is a
5+`/devshell/frq-web` and exits, and what it leaves behind is a
66 directory of static files that outlives it. A Function mounting the same
77 volume can hand those out without rebuilding anything, and can scale to zero
88 between readers -- which a Sandbox holding a tunnel open cannot.
99
10- modal serve .modal/flutter-web/serve.py # while editing, auto-reloads
11- modal deploy .modal/flutter-web/serve.py # a URL that stays
10+ modal serve .modal/web/serve.py # while editing, auto-reloads
11+ modal deploy .modal/web/serve.py # a URL that stays
1212
1313 Deliberately NOT built on the container's own image. That image carries the
1414 repo and a few gigabytes of devShell closure, all of it for a compile that
@@ -26,9 +26,9 @@ import modal
2626 # lazy, so naming it here costs nothing until a container actually mounts it.
2727 DEVSHELL = modal.Volume.from_name("devshell", create_if_missing=True)
2828
29-# Where `just flutter-web` leaves its output, under this devShell's own
29+# Where `just web` leaves its output, under this devShell's own
3030 # directory on the shared volume -- the same path container.toml builds in.
31-WEB_ROOT = "/devshell/frq-flutter-web/flutter/build/web"
31+WEB_ROOT = "/devshell/frq-web/flutter/build/web"
3232
3333 PORT = 8080
3434
similarity index 97%
rename from .modal/flutter-web/serve.py
rename to .modal/web/serve.py
@@ -2,13 +2,13 @@
2 2
3 Not part of `container.py`, and not a key in `container.toml`, because it is3 Not part of `container.py`, and not a key in `container.toml`, because it is
4 the other half of a split the build already makes: the Sandbox compiles into4 the other half of a split the build already makes: the Sandbox compiles into
5-`/devshell/frq-flutter-web` and exits, and what it leaves behind is a5+`/devshell/frq-web` and exits, and what it leaves behind is a
6 directory of static files that outlives it. A Function mounting the same6 directory of static files that outlives it. A Function mounting the same
7 volume can hand those out without rebuilding anything, and can scale to zero7 volume can hand those out without rebuilding anything, and can scale to zero
8 between readers -- which a Sandbox holding a tunnel open cannot.8 between readers -- which a Sandbox holding a tunnel open cannot.
9 9
10- modal serve .modal/flutter-web/serve.py # while editing, auto-reloads10+ modal serve .modal/web/serve.py # while editing, auto-reloads
11- modal deploy .modal/flutter-web/serve.py # a URL that stays11+ modal deploy .modal/web/serve.py # a URL that stays
12 12
13 Deliberately NOT built on the container's own image. That image carries the13 Deliberately NOT built on the container's own image. That image carries the
14 repo and a few gigabytes of devShell closure, all of it for a compile that14 repo and a few gigabytes of devShell closure, all of it for a compile that
@@ -26,9 +26,9 @@ import modal
26 # lazy, so naming it here costs nothing until a container actually mounts it.26 # lazy, so naming it here costs nothing until a container actually mounts it.
27 DEVSHELL = modal.Volume.from_name("devshell", create_if_missing=True)27 DEVSHELL = modal.Volume.from_name("devshell", create_if_missing=True)
28 28
29-# Where `just flutter-web` leaves its output, under this devShell's own29+# Where `just web` leaves its output, under this devShell's own
30 # directory on the shared volume -- the same path container.toml builds in.30 # directory on the shared volume -- the same path container.toml builds in.
31-WEB_ROOT = "/devshell/frq-flutter-web/flutter/build/web"31+WEB_ROOT = "/devshell/frq-web/flutter/build/web"
32 32
33 PORT = 808033 PORT = 8080
34 34
modified .rickub/workflows/build.yml +3 -3
@@ -148,7 +148,7 @@ jobs:
148148 # nowhere else, and `modal app logs` cannot reach an ephemeral run —
149149 # so this terminal is the only place the build is visible. tee, not
150150 # tail: a run killed mid-pipe through tail takes its output with it.
151- run: modal run .modal/flutter-web/container.py 2>&1 | tee /tmp/frq-build.log
151+ run: modal run .modal/web/container.py 2>&1 | tee /tmp/frq-build.log
152152
153153 # The Sandbox leaves the bundle on the devshell volume rather than
154154 # anywhere a runner can see, so fetch it back out.
@@ -157,13 +157,13 @@ jobs:
157157 MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }}
158158 MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }}
159159 # Into a directory that already exists, which `modal volume get` then
160- # creates `web/` inside — the same shape `just web-local` uses. Naming
160+ # creates `web/` inside — the same shape `just serve` uses. Naming
161161 # `web` as the destination itself is what failed, with `[Errno 21] Is a
162162 # directory`, after the Modal build had already succeeded.
163163 run: |
164164 mkdir -p dist
165165 modal volume get --force devshell \
166- frq-flutter-web/flutter/build/web dist
166+ frq-web/flutter/build/web dist
167167
168168 - uses: actions/upload-artifact@v4
169169 with:
@@ -148,7 +148,7 @@ jobs:
148 # nowhere else, and `modal app logs` cannot reach an ephemeral run —148 # nowhere else, and `modal app logs` cannot reach an ephemeral run —
149 # so this terminal is the only place the build is visible. tee, not149 # so this terminal is the only place the build is visible. tee, not
150 # tail: a run killed mid-pipe through tail takes its output with it.150 # tail: a run killed mid-pipe through tail takes its output with it.
151- run: modal run .modal/flutter-web/container.py 2>&1 | tee /tmp/frq-build.log151+ run: modal run .modal/web/container.py 2>&1 | tee /tmp/frq-build.log
152 152
153 # The Sandbox leaves the bundle on the devshell volume rather than153 # The Sandbox leaves the bundle on the devshell volume rather than
154 # anywhere a runner can see, so fetch it back out.154 # anywhere a runner can see, so fetch it back out.
@@ -157,13 +157,13 @@ jobs:
157 MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }}157 MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }}
158 MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }}158 MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }}
159 # Into a directory that already exists, which `modal volume get` then159 # Into a directory that already exists, which `modal volume get` then
160- # creates `web/` inside — the same shape `just web-local` uses. Naming160+ # creates `web/` inside — the same shape `just serve` uses. Naming
161 # `web` as the destination itself is what failed, with `[Errno 21] Is a161 # `web` as the destination itself is what failed, with `[Errno 21] Is a
162 # directory`, after the Modal build had already succeeded.162 # directory`, after the Modal build had already succeeded.
163 run: |163 run: |
164 mkdir -p dist164 mkdir -p dist
165 modal volume get --force devshell \165 modal volume get --force devshell \
166- frq-flutter-web/flutter/build/web dist166+ frq-web/flutter/build/web dist
167 167
168 - uses: actions/upload-artifact@v4168 - uses: actions/upload-artifact@v4
169 with:169 with:
modified CLAUDE.md +31 -40
@@ -1,44 +1,36 @@
11 # Working in this repo
22
3-## Nix
3+## The toolchain, and where builds happen
44
5-**STOP BUILDING LOCALLY. Build on Modal.** This machine is for editing and for
6-evaluating — `nix flake check`, `nix eval`, `nix build --dry-run`, `nix repl`
7-— and not for realising a derivation. A local build of the whole graph gets
8-killed for memory long before it finishes, and the minutes spent finding that
9-out are minutes not spent on the change. So:
5+There is no nix in the build any more. `tools/toolchain.sh` fetches Flutter
6+(which carries Dart), a JDK, the Clojure CLI and Nim as sha256-pinned tarballs
7+into `.toolchain/`, and every `just` recipe runs inside the environment that
8+script prints. `just tools android` adds Google's command-line tools, which is
9+what `just build apk` needs before Gradle can have sdkmanager finish the SDK
10+off. The host still brings a C compiler, OpenSSL, git, curl, unzip and
11+python3 — and GTK with the usual CMake/Ninja/pkg-config for the Linux targets.
12+
13+**Prefer Modal for a long build.** A cold Flutter toolchain plus a full
14+compile is a lot of laptop, and the containers in `.modal/` do it on a real
15+machine:
1016
1117 ```bash
12-modal run .modal/flutter-web/container.py # the web bundle, on Modal
13-modal run .modal/flutter-dev/container.py # the incremental Flutter loop
18+just modal web # the web bundle, on Modal
19+just modal dev # the incremental Flutter loop
1420 ```
1521
1622 The containers in `.modal/` are not a third source tree: they are CI config
17-that happens to live here, the way `.github/` would be.
18-
19-`--dry-run` locally to see what *would* be built, then hand the build to Modal.
20-The one exception is a derivation you already know is trivial and already
21-substitutable; if you are unsure, it is not the exception.
23+that happens to live here, the way `.github/` would be. They run the very same
24+`tools/toolchain.sh`, which is why a plain Debian image is enough.
2225
2326 `modal app logs` is no substitute for watching that command: it resolves
2427 deployed apps by name, not the ephemeral one a `modal run` creates, and carries
2528 nothing until the Sandbox starts — the image build streams to the client and
2629 nowhere else.
2730
28-You are already running inside the Arch distrobox, where `nix` lives, so run
29-the evaluating commands directly — do not wrap them in `distrobox enter`.
30-
31-The containers run as Modal **Sandboxes on a real VM**, which is what makes a
32-build work out there at all: the ptyshim that used to stand in for a working
33-pty under gVisor is deprecated, and nothing here should reintroduce it. Neither
34-container carries nix: `tools/toolchain.sh` fetches Flutter, a JDK and the
35-Clojure CLI by sha256 onto the `devshell` Volume, and the build runs out of
36-those.
37-
38-A remote builder (`eu.nixbuild.net`) is also configured here, for the case
39-where you want a derivation built somewhere other than Modal: `--store
40-ssh-ng://eu.nixbuild.net --eval-store auto` rather than a `builders` entry, so
41-the whole graph stays there and only .drv files go up.
31+The containers run as Modal **Sandboxes on a real VM** rather than under
32+gVisor: a real kernel, a working pty, and memory that is exactly what
33+`[resources] memory` asks for.
4234
4335 One thing this container is *not* representative of: `/etc/localtime` is a
4436 regular file here rather than a symlink, so anything that reads the zone out
@@ -56,7 +48,7 @@ Let them write to the terminal, or `tee` them if you want a copy to grep
5648 afterwards:
5749
5850 ```bash
59-modal run .modal/flutter-web/container.py 2>&1 | tee /tmp/frq-build.log
51+modal run .modal/web/container.py 2>&1 | tee /tmp/frq-build.log
6052 ```
6153
6254 Trim afterwards, on the file, where the whole run is still there to re-read.
@@ -83,7 +75,7 @@ ClojureDart — the core exists to have less Clojure in the tree, and adding
8375 more of it to call the thing replacing it is the wrong direction. ClojureDart
8476 shrinks from both ends.
8577
86-`just nim-test` and `just dart-test` need no Flutter, which is most of the
78+`just test nim` and `just test dart` need no Flutter, which is most of the
8779 point: the whole boundary is checkable in about a second.
8880
8981 A module is not deleted from `common/` when its Nim version lands: the web
@@ -120,19 +112,18 @@ every push.
120112
121113 `flutter/` builds three things, from one `clojure -M:cljd compile`:
122114
123-`just apk`, out of the flake's own `.#flutter` shell (clojure, jdk17, flutter)
124-and its `.#android-sdk` package. Impure on purpose: Gradle fetches its own
125-dependencies and writes into `ANDROID_HOME`, so the recipe copies the store SDK
126-to `flutter/.home` and lets it finish there.
115+`just build apk`. Impure on purpose: Gradle resolves its own dependencies over
116+the network and has sdkmanager install a platform and build-tools into
117+`ANDROID_HOME` as it goes, which is why that SDK lives in `.toolchain/` and is
118+ours to write to.
127119
128-`just flutter-desktop`, out of `.#flutter-desktop` (the same clojure and
129-flutter, with cmake, ninja, pkg-config and gtk3 where the JDK and the SDK are).
130-Impure for the network half of the same reasons and no writable-SDK dance,
131-since nothing writes into the store. nixGL off NixOS.
120+`just build desktop`, Flutter's Linux target — CMake, Ninja, pkg-config and
121+GTK from the host where the APK wants a JDK and an SDK. Impure for the network
122+half of the same reasons.
132123
133-`just flutter-web`, out of no nix shell at all — `tools/toolchain.sh` fetches
134-the three pinned tarballs it needs, which is what lets `.modal/flutter-web/`
135-run the same script on a plain Debian image.
124+`just build web`, which needs least of all: a Dart, a JVM and a browser, and
125+the browser is not ours. That is what lets `.modal/web/` run the same
126+`tools/build-web.sh` on a plain Debian image.
136127
137128 The consequence for `common/` is that "the phone" is not a synonym for "the
138129 ClojureDart side": three targets compile it. An implementation that branches on
@@ -1,44 +1,36 @@
1 # Working in this repo1 # Working in this repo
2 2
3-## Nix3+## The toolchain, and where builds happen
4 4
5-**STOP BUILDING LOCALLY. Build on Modal.** This machine is for editing and for5+There is no nix in the build any more. `tools/toolchain.sh` fetches Flutter
6-evaluating — `nix flake check`, `nix eval`, `nix build --dry-run`, `nix repl`6+(which carries Dart), a JDK, the Clojure CLI and Nim as sha256-pinned tarballs
7-— and not for realising a derivation. A local build of the whole graph gets7+into `.toolchain/`, and every `just` recipe runs inside the environment that
8-killed for memory long before it finishes, and the minutes spent finding that8+script prints. `just tools android` adds Google's command-line tools, which is
9-out are minutes not spent on the change. So:9+what `just build apk` needs before Gradle can have sdkmanager finish the SDK
10+off. The host still brings a C compiler, OpenSSL, git, curl, unzip and
11+python3 — and GTK with the usual CMake/Ninja/pkg-config for the Linux targets.
12+
13+**Prefer Modal for a long build.** A cold Flutter toolchain plus a full
14+compile is a lot of laptop, and the containers in `.modal/` do it on a real
15+machine:
10 16
11 ```bash17 ```bash
12-modal run .modal/flutter-web/container.py # the web bundle, on Modal18+just modal web # the web bundle, on Modal
13-modal run .modal/flutter-dev/container.py # the incremental Flutter loop19+just modal dev # the incremental Flutter loop
14 ```20 ```
15 21
16 The containers in `.modal/` are not a third source tree: they are CI config22 The containers in `.modal/` are not a third source tree: they are CI config
17-that happens to live here, the way `.github/` would be.23+that happens to live here, the way `.github/` would be. They run the very same
18-24+`tools/toolchain.sh`, which is why a plain Debian image is enough.
19-`--dry-run` locally to see what *would* be built, then hand the build to Modal.
20-The one exception is a derivation you already know is trivial and already
21-substitutable; if you are unsure, it is not the exception.
22 25
23 `modal app logs` is no substitute for watching that command: it resolves26 `modal app logs` is no substitute for watching that command: it resolves
24 deployed apps by name, not the ephemeral one a `modal run` creates, and carries27 deployed apps by name, not the ephemeral one a `modal run` creates, and carries
25 nothing until the Sandbox starts — the image build streams to the client and28 nothing until the Sandbox starts — the image build streams to the client and
26 nowhere else.29 nowhere else.
27 30
28-You are already running inside the Arch distrobox, where `nix` lives, so run31+The containers run as Modal **Sandboxes on a real VM** rather than under
29-the evaluating commands directly — do not wrap them in `distrobox enter`.32+gVisor: a real kernel, a working pty, and memory that is exactly what
30-33+`[resources] memory` asks for.
31-The containers run as Modal **Sandboxes on a real VM**, which is what makes a
32-build work out there at all: the ptyshim that used to stand in for a working
33-pty under gVisor is deprecated, and nothing here should reintroduce it. Neither
34-container carries nix: `tools/toolchain.sh` fetches Flutter, a JDK and the
35-Clojure CLI by sha256 onto the `devshell` Volume, and the build runs out of
36-those.
37-
38-A remote builder (`eu.nixbuild.net`) is also configured here, for the case
39-where you want a derivation built somewhere other than Modal: `--store
40-ssh-ng://eu.nixbuild.net --eval-store auto` rather than a `builders` entry, so
41-the whole graph stays there and only .drv files go up.
42 34
43 One thing this container is *not* representative of: `/etc/localtime` is a35 One thing this container is *not* representative of: `/etc/localtime` is a
44 regular file here rather than a symlink, so anything that reads the zone out36 regular file here rather than a symlink, so anything that reads the zone out
@@ -56,7 +48,7 @@ Let them write to the terminal, or `tee` them if you want a copy to grep
56 afterwards:48 afterwards:
57 49
58 ```bash50 ```bash
59-modal run .modal/flutter-web/container.py 2>&1 | tee /tmp/frq-build.log51+modal run .modal/web/container.py 2>&1 | tee /tmp/frq-build.log
60 ```52 ```
61 53
62 Trim afterwards, on the file, where the whole run is still there to re-read.54 Trim afterwards, on the file, where the whole run is still there to re-read.
@@ -83,7 +75,7 @@ ClojureDart — the core exists to have less Clojure in the tree, and adding
83 more of it to call the thing replacing it is the wrong direction. ClojureDart75 more of it to call the thing replacing it is the wrong direction. ClojureDart
84 shrinks from both ends.76 shrinks from both ends.
85 77
86-`just nim-test` and `just dart-test` need no Flutter, which is most of the78+`just test nim` and `just test dart` need no Flutter, which is most of the
87 point: the whole boundary is checkable in about a second.79 point: the whole boundary is checkable in about a second.
88 80
89 A module is not deleted from `common/` when its Nim version lands: the web81 A module is not deleted from `common/` when its Nim version lands: the web
@@ -120,19 +112,18 @@ every push.
120 112
121 `flutter/` builds three things, from one `clojure -M:cljd compile`:113 `flutter/` builds three things, from one `clojure -M:cljd compile`:
122 114
123-`just apk`, out of the flake's own `.#flutter` shell (clojure, jdk17, flutter)115+`just build apk`. Impure on purpose: Gradle resolves its own dependencies over
124-and its `.#android-sdk` package. Impure on purpose: Gradle fetches its own116+the network and has sdkmanager install a platform and build-tools into
125-dependencies and writes into `ANDROID_HOME`, so the recipe copies the store SDK117+`ANDROID_HOME` as it goes, which is why that SDK lives in `.toolchain/` and is
126-to `flutter/.home` and lets it finish there.118+ours to write to.
127 119
128-`just flutter-desktop`, out of `.#flutter-desktop` (the same clojure and120+`just build desktop`, Flutter's Linux target — CMake, Ninja, pkg-config and
129-flutter, with cmake, ninja, pkg-config and gtk3 where the JDK and the SDK are).121+GTK from the host where the APK wants a JDK and an SDK. Impure for the network
130-Impure for the network half of the same reasons and no writable-SDK dance,122+half of the same reasons.
131-since nothing writes into the store. nixGL off NixOS.
132 123
133-`just flutter-web`, out of no nix shell at all — `tools/toolchain.sh` fetches124+`just build web`, which needs least of all: a Dart, a JVM and a browser, and
134-the three pinned tarballs it needs, which is what lets `.modal/flutter-web/`125+the browser is not ours. That is what lets `.modal/web/` run the same
135-run the same script on a plain Debian image.126+`tools/build-web.sh` on a plain Debian image.
136 127
137 The consequence for `common/` is that "the phone" is not a synonym for "the128 The consequence for `common/` is that "the phone" is not a synonym for "the
138 ClojureDart side": three targets compile it. An implementation that branches on129 ClojureDart side": three targets compile it. An implementation that branches on
modified README.md +11 -12
@@ -53,7 +53,7 @@ There used to be a third tree, `src/`, and another runtime under it: jolt, with
5353 **libcosmic** in a desktop window and by `libjolttui` in a terminal, plus an
5454 AV media plane over MoQ. It is gone. Flutter is the only frontend now, which is
5555 why `common/` no longer carries `#?(:jolt ...)` reader conditionals and why the
56-calls, terminal and `nix run .#frq` sections that used to be here are not.
56+calls and terminal sections that used to be here are not.
5757
5858 ## Tracing
5959
@@ -63,18 +63,17 @@ Android is logcat.
6363 ## Running
6464
6565 ```bash
66-just flutter-desktop run # the Linux window
67-just apk run # onto a connected Android device
68-just flutter-web serve # a browser, on :8080
66+just run desktop # the Linux window
67+just run apk # onto a connected Android device
68+just run web # a browser, on :8080
6969 ```
7070
71-Every recipe lives in the `justfile` itself. The two that need a toolchain from
72-Nix re-enter `nix develop` and come back to the same recipe, so `just
73-flutter-desktop` and `nix develop .#flutter-desktop --command just
74-flutter-desktop` are one code path rather than two. `just flutter-web` needs no
75-Nix at all: `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by
76-sha256, which is what lets `.modal/flutter-web/` run the same script on a plain
77-Debian image.
71+Every recipe lives in the `justfile` itself, and none of them needs Nix:
72+`tools/toolchain.sh` fetches Flutter, a JDK, the Clojure CLI and Nim by
73+sha256 into `.toolchain/`, and every recipe runs inside that. It is the same
74+script `.modal/` runs, which is what lets a plain Debian image build this.
75+`just build apk` additionally asks it for Google's command-line tools, and
76+sdkmanager finishes the Android SDK off.
7877
7978 All three are one `clojure -M:cljd compile` over `flutter/src` and `common/`,
8079 and differ only in which Flutter target runs afterwards.
@@ -90,7 +89,7 @@ cargo run --release --bin freeq-server # in the freeq checkout
9089 A browser has no TCP, so the web build wants a WebSocket URL in the Server
9190 field — `wss://irc.freeq.at/irc`. And Bluesky sign-in only completes on
9291 `localhost`, because that is the one origin freeq's auth broker will redirect
93-back to: `just web-local` is what serves the Modal-built bundle there.
92+back to: `just serve` is what serves the Modal-built bundle there.
9493
9594 ## Signing in
9695
@@ -53,7 +53,7 @@ There used to be a third tree, `src/`, and another runtime under it: jolt, with
53 **libcosmic** in a desktop window and by `libjolttui` in a terminal, plus an53 **libcosmic** in a desktop window and by `libjolttui` in a terminal, plus an
54 AV media plane over MoQ. It is gone. Flutter is the only frontend now, which is54 AV media plane over MoQ. It is gone. Flutter is the only frontend now, which is
55 why `common/` no longer carries `#?(:jolt ...)` reader conditionals and why the55 why `common/` no longer carries `#?(:jolt ...)` reader conditionals and why the
56-calls, terminal and `nix run .#frq` sections that used to be here are not.56+calls and terminal sections that used to be here are not.
57 57
58 ## Tracing58 ## Tracing
59 59
@@ -63,18 +63,17 @@ Android is logcat.
63 ## Running63 ## Running
64 64
65 ```bash65 ```bash
66-just flutter-desktop run # the Linux window66+just run desktop # the Linux window
67-just apk run # onto a connected Android device67+just run apk # onto a connected Android device
68-just flutter-web serve # a browser, on :808068+just run web # a browser, on :8080
69 ```69 ```
70 70
71-Every recipe lives in the `justfile` itself. The two that need a toolchain from71+Every recipe lives in the `justfile` itself, and none of them needs Nix:
72-Nix re-enter `nix develop` and come back to the same recipe, so `just72+`tools/toolchain.sh` fetches Flutter, a JDK, the Clojure CLI and Nim by
73-flutter-desktop` and `nix develop .#flutter-desktop --command just73+sha256 into `.toolchain/`, and every recipe runs inside that. It is the same
74-flutter-desktop` are one code path rather than two. `just flutter-web` needs no74+script `.modal/` runs, which is what lets a plain Debian image build this.
75-Nix at all: `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by75+`just build apk` additionally asks it for Google's command-line tools, and
76-sha256, which is what lets `.modal/flutter-web/` run the same script on a plain76+sdkmanager finishes the Android SDK off.
77-Debian image.
78 77
79 All three are one `clojure -M:cljd compile` over `flutter/src` and `common/`,78 All three are one `clojure -M:cljd compile` over `flutter/src` and `common/`,
80 and differ only in which Flutter target runs afterwards.79 and differ only in which Flutter target runs afterwards.
@@ -90,7 +89,7 @@ cargo run --release --bin freeq-server # in the freeq checkout
90 A browser has no TCP, so the web build wants a WebSocket URL in the Server89 A browser has no TCP, so the web build wants a WebSocket URL in the Server
91 field — `wss://irc.freeq.at/irc`. And Bluesky sign-in only completes on90 field — `wss://irc.freeq.at/irc`. And Bluesky sign-in only completes on
92 `localhost`, because that is the one origin freeq's auth broker will redirect91 `localhost`, because that is the one origin freeq's auth broker will redirect
93-back to: `just web-local` is what serves the Modal-built bundle there.92+back to: `just serve` is what serves the Modal-built bundle there.
94 93
95 ## Signing in94 ## Signing in
96 95
modified dart/README.md +1 -1
@@ -11,7 +11,7 @@ things:
1111 * **It tests on the plain Dart VM.** `flutter/pubspec.yaml` depends on the
1212 Flutter SDK, so `dart pub get` cannot resolve it at all — anything living
1313 there needs a Flutter toolchain to run one assertion about a string. This
14- package resolves and tests in a second. `just dart-test`.
14+ package resolves and tests in a second. `just test dart`.
1515 * **It says which way the dependency goes.** The Flutter app depends on this
1616 by path. Nothing here may depend on Flutter, and if that ever becomes
1717 tempting the thing being written belongs on the other side of the line.
@@ -11,7 +11,7 @@ things:
11 * **It tests on the plain Dart VM.** `flutter/pubspec.yaml` depends on the11 * **It tests on the plain Dart VM.** `flutter/pubspec.yaml` depends on the
12 Flutter SDK, so `dart pub get` cannot resolve it at all — anything living12 Flutter SDK, so `dart pub get` cannot resolve it at all — anything living
13 there needs a Flutter toolchain to run one assertion about a string. This13 there needs a Flutter toolchain to run one assertion about a string. This
14- package resolves and tests in a second. `just dart-test`.14+ package resolves and tests in a second. `just test dart`.
15 * **It says which way the dependency goes.** The Flutter app depends on this15 * **It says which way the dependency goes.** The Flutter app depends on this
16 by path. Nothing here may depend on Flutter, and if that ever becomes16 by path. Nothing here may depend on Flutter, and if that ever becomes
17 tempting the thing being written belongs on the other side of the line.17 tempting the thing being written belongs on the other side of the line.
deleted flake.lock +0 -82
deleted file mode 100644
@@ -1,82 +0,0 @@
1-{
2- "nodes": {
3- "flake-utils": {
4- "inputs": {
5- "systems": "systems"
6- },
7- "locked": {
8- "lastModified": 1731533236,
9- "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=",
10- "owner": "numtide",
11- "repo": "flake-utils",
12- "rev": "11707dc2f618dd54ca8739b309ec4fc024de578b",
13- "type": "github"
14- },
15- "original": {
16- "owner": "numtide",
17- "repo": "flake-utils",
18- "type": "github"
19- }
20- },
21- "nixgl": {
22- "inputs": {
23- "flake-utils": "flake-utils",
24- "nixpkgs": [
25- "nixpkgs"
26- ]
27- },
28- "locked": {
29- "lastModified": 1762090880,
30- "narHash": "sha256-fbRQzIGPkjZa83MowjbD2ALaJf9y6KMDdJBQMKFeY/8=",
31- "owner": "nix-community",
32- "repo": "nixGL",
33- "rev": "b6105297e6f0cd041670c3e8628394d4ee247ed5",
34- "type": "github"
35- },
36- "original": {
37- "owner": "nix-community",
38- "repo": "nixGL",
39- "type": "github"
40- }
41- },
42- "nixpkgs": {
43- "locked": {
44- "lastModified": 1788039129,
45- "narHash": "sha256-pa4Q0qErvCvzCaaUph7Sm37RhR4xvPrYI8Lgz6k85+A=",
46- "owner": "NixOS",
47- "repo": "nixpkgs",
48- "rev": "d2f67949798825fe853f7c5d0492b8bf016d3f88",
49- "type": "github"
50- },
51- "original": {
52- "owner": "NixOS",
53- "ref": "nixos-unstable",
54- "repo": "nixpkgs",
55- "type": "github"
56- }
57- },
58- "root": {
59- "inputs": {
60- "nixgl": "nixgl",
61- "nixpkgs": "nixpkgs"
62- }
63- },
64- "systems": {
65- "locked": {
66- "lastModified": 1681028828,
67- "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
68- "owner": "nix-systems",
69- "repo": "default",
70- "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
71- "type": "github"
72- },
73- "original": {
74- "owner": "nix-systems",
75- "repo": "default",
76- "type": "github"
77- }
78- }
79- },
80- "root": "root",
81- "version": 7
82-}
deleted file mode 100644
@@ -1,82 +0,0 @@
1-{
2- "nodes": {
3- "flake-utils": {
4- "inputs": {
5- "systems": "systems"
6- },
7- "locked": {
8- "lastModified": 1731533236,
9- "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=",
10- "owner": "numtide",
11- "repo": "flake-utils",
12- "rev": "11707dc2f618dd54ca8739b309ec4fc024de578b",
13- "type": "github"
14- },
15- "original": {
16- "owner": "numtide",
17- "repo": "flake-utils",
18- "type": "github"
19- }
20- },
21- "nixgl": {
22- "inputs": {
23- "flake-utils": "flake-utils",
24- "nixpkgs": [
25- "nixpkgs"
26- ]
27- },
28- "locked": {
29- "lastModified": 1762090880,
30- "narHash": "sha256-fbRQzIGPkjZa83MowjbD2ALaJf9y6KMDdJBQMKFeY/8=",
31- "owner": "nix-community",
32- "repo": "nixGL",
33- "rev": "b6105297e6f0cd041670c3e8628394d4ee247ed5",
34- "type": "github"
35- },
36- "original": {
37- "owner": "nix-community",
38- "repo": "nixGL",
39- "type": "github"
40- }
41- },
42- "nixpkgs": {
43- "locked": {
44- "lastModified": 1788039129,
45- "narHash": "sha256-pa4Q0qErvCvzCaaUph7Sm37RhR4xvPrYI8Lgz6k85+A=",
46- "owner": "NixOS",
47- "repo": "nixpkgs",
48- "rev": "d2f67949798825fe853f7c5d0492b8bf016d3f88",
49- "type": "github"
50- },
51- "original": {
52- "owner": "NixOS",
53- "ref": "nixos-unstable",
54- "repo": "nixpkgs",
55- "type": "github"
56- }
57- },
58- "root": {
59- "inputs": {
60- "nixgl": "nixgl",
61- "nixpkgs": "nixpkgs"
62- }
63- },
64- "systems": {
65- "locked": {
66- "lastModified": 1681028828,
67- "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
68- "owner": "nix-systems",
69- "repo": "default",
70- "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
71- "type": "github"
72- },
73- "original": {
74- "owner": "nix-systems",
75- "repo": "default",
76- "type": "github"
77- }
78- }
79- },
80- "root": "root",
81- "version": 7
82-}
deleted flake.nix +0 -642
deleted file mode 100644
@@ -1,642 +0,0 @@
1-{
2- # frq is a Flutter app whose source is ClojureDart, so what this flake
3- # provides is toolchains rather than a built program: the `flutter` shell
4- # that `just apk` compiles in, the `flutter-desktop` shell for the Linux
5- # target, and the Android SDK the first of those copies somewhere writable.
6- #
7- # nix develop .#flutter-desktop --command just flutter-desktop run
8- #
9- # On a machine that is not NixOS the GL driver is the host's and the loader
10- # will not find it, so the window never opens ("GL display: argument does not
11- # name a valid config"). The recipes handle that themselves: off NixOS they
12- # hand the process to nixGL, which puts the host's driver ahead of the
13- # store's. A distrobox/container Arch is the same case as a bare one.
14- description = "frq a freeq client in Flutter";
15-
16- inputs = {
17- nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
18-
19- # Only ever used off NixOS, to put the host GL driver on the loader path.
20- nixgl = {
21- url = "github:nix-community/nixGL";
22- inputs.nixpkgs.follows = "nixpkgs";
23- };
24- };
25-
26- outputs = { self, nixpkgs, nixgl }:
27- let
28- systems = [ "x86_64-linux" "aarch64-linux" ];
29- forEachSystem = f:
30- nixpkgs.lib.genAttrs systems (system: f nixpkgs.legacyPackages.${system});
31-
32- # Mesa, despite the name: it covers Intel and AMD alike. The NVIDIA
33- # wrappers are the ones that need --impure (they read the host kernel
34- # module's version), which is why this only ever reaches for Intel.
35- #
36- # Built from nixGL's default.nix rather than taken from its flake
37- # outputs, for the one argument the flake hardcodes on: `enable32bits`,
38- # which on x86_64 puts a second, i686 copy of mesa, its LLVM, and
39- # intel-media-driver into the wrapper. frq is 64-bit on both halves —
40- # the Rust cdylibs and the Chez runtime — so nothing here ever opens the
41- # 32-bit driver, and carrying it is most of the dev shell's closure.
42- nixGLFor = pkgs: (import nixgl {
43- inherit pkgs;
44- enable32bits = false;
45- }).nixGLIntel;
46-
47- # The Android SDK wants two things `nixpkgs.legacyPackages` cannot give:
48- # `allowUnfree`, because the SDK's own licence is not free, and
49- # `android_sdk.accept_license`, which is how you say so in a file rather
50- # than at a prompt a build has no terminal for. Neither can be set on a
51- # legacyPackages attribute after the fact, so this is a second import of
52- # the same locked nixpkgs rather than a second nixpkgs.
53- #
54- # This used to live in `just apk` as a `nix build --impure --expr` with
55- # `builtins.getFlake "github:NixOS/nixpkgs/nixos-unstable"` inside it —
56- # which fetched whatever nixos-unstable was that morning, not what
57- # flake.lock pins, so the SDK under the APK and the nixpkgs under
58- # everything else were free to drift apart. Here they are the same rev.
59- androidPkgsFor = system: import nixpkgs {
60- inherit system;
61- config = {
62- allowUnfree = true;
63- android_sdk.accept_license = true;
64- };
65- };
66-
67- # Only the floor Gradle stands on. It installs build-tools and a platform
68- # into ANDROID_HOME itself as it goes — see `just apk` for why that means
69- # a writable copy — so composing more of them here buys nothing.
70- #
71- # includeNDK = false deliberately: the app is Dart and path_provider is
72- # platform channels, so there is no native code to need one, and asking
73- # for it is a few hundred megabytes and a Gradle fetch of that exact NDK.
74- androidSdkFor = system:
75- let android = androidPkgsFor system; in
76- (android.androidenv.composeAndroidPackages {
77- cmdLineToolsVersion = "13.0";
78- buildToolsVersions = [ "34.0.0" ];
79- platformVersions = [ "35" "34" ];
80- includeNDK = false;
81- }).androidsdk;
82-
83- in
84- {
85- packages = forEachSystem (pkgs:
86- let
87- inherit (pkgs) lib;
88- in
89- {
90- # The Android SDK `just apk` copies into flutter/.home. A package
91- # rather than something the recipe evaluates inline, so that
92- # `nix build .#android-sdk` is how you pre-warm it and `nix flake
93- # show` admits it exists.
94- android-sdk = androidSdkFor pkgs.stdenv.hostPlatform.system;
95-
96- # There were `appimage` outputs here, and what they were for was a
97- # host without Nix: they squashed the whole closure into one
98- # runnable file, Mesa included, and the Mesa was not waste — off
99- # NixOS the launcher goes through nixGL, which needs a store Mesa to
100- # put the host's driver in front of. Nothing asks for that shape any
101- # more, and they were the last thing evaluating nix-appimage, which
102- # is why that input is gone too.
103-
104- # Everything `clojure -M:cljd compile` would otherwise reach the
105- # network for, fetched once and hashed.
106- #
107- # The compile needs three caches, and the reason this is one
108- # derivation rather than three is that only one of them is obvious.
109- # Maven and gitlibs are the ordinary tools.deps pair. The third is
110- # ClojureDart's own: `ensure-cljd-analyzer!` writes a *second*, whole
111- # pub project to `.clojuredart/cache/<cljd sha>/cljd_helper`, runs
112- # `pub add analyzer` in it, and then runs `bin/analyzer.dart` out of
113- # it for the duration of the compile — so a sandbox needs that
114- # project already resolved, not just the app's dependencies.
115- #
116- # Fixed-output, so it is allowed the network the rest of the build is
117- # not. What that costs is a hash to maintain, and the thing worth
118- # being exact about is *when*: this derivation never sees frq's
119- # source. It compiles a three-line throwaway project against the same
120- # `flutter/deps.edn` and the same `flutter/pubspec.yaml`, so the hash
121- # moves when a dependency moves and not when a screen changes. A
122- # stub, rather than `-P` and a hand-built analyzer dir, because
123- # running the real compiler once is the only way to be sure the
124- # caches are the ones it actually wants.
125- #
126- # PUB_CACHE lands in $out on purpose. The package_config.json inside
127- # cljd_helper carries absolute paths to whatever resolved it, so
128- # resolving into a build directory would bake in paths that stop
129- # existing the moment this derivation finishes. Pointed at $out they
130- # are store paths, and still true.
131- cljd-deps =
132- let
133- flutterPkg = pkgs.flutter;
134- in
135- pkgs.stdenvNoCC.mkDerivation {
136- name = "frq-cljd-deps";
137- dontUnpack = true;
138-
139- nativeBuildInputs = [
140- pkgs.clojure
141- pkgs.jdk17
142- flutterPkg
143- pkgs.git
144- pkgs.cacert
145- ];
146-
147- buildCommand = ''
148- export HOME="$NIX_BUILD_TOP/home"
149- # Resolved in the build directory and copied to $out at the
150- # end, never written there directly. A fixed-output derivation
151- # may not reference a store path and its own output is a store
152- # path, so pub writing its cache's absolute location into its
153- # own metadata is enough to fail the check.
154- cache="$NIX_BUILD_TOP/cache"
155- export PUB_CACHE="$cache/pub-cache"
156- export GITLIBS="$cache/gitlibs"
157- export SSL_CERT_FILE="${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt"
158- mkdir -p "$HOME" "$PUB_CACHE" "$GITLIBS" "$cache/m2"
159-
160- # The stub: our dependency files, nothing of our source.
161- # ../common is a :local/root dependency now, so it has to exist
162- # *and* carry a deps.edn for tools.deps to resolve an empty
163- # directory with that one file in it is enough.
164- proj="$NIX_BUILD_TOP/stub"
165- mkdir -p "$proj/src/stub" "$NIX_BUILD_TOP/common"
166- cp ${./common/deps.edn} "$NIX_BUILD_TOP/common/deps.edn"
167- cp ${./flutter/deps.edn} "$proj/deps.edn"
168- cp ${./flutter/pubspec.yaml} "$proj/pubspec.yaml"
169- # The lock, or `pub get` resolves against pub.dev and takes
170- # whatever satisfies the ranges today. Every build then fetches
171- # a slightly different set and the fixed-output hash is a
172- # promise nothing can keep.
173- cp ${./flutter/pubspec.lock} "$proj/pubspec.lock"
174- chmod u+w "$proj/deps.edn" "$proj/pubspec.yaml" "$proj/pubspec.lock"
175- cat > "$proj/src/stub/main.cljd" <<'EOF'
176- (ns stub.main)
177- (defn main [] nil)
178- EOF
179-
180- cd "$proj"
181- # `:main` has to name the stub, or the compiler goes looking for
182- # frq.main in a tree that is not here.
183- sed -i 's/:main frq\.main/:main stub.main/' deps.edn
184-
185- flutter config --no-analytics &>/dev/null || true
186- flutter config --enable-linux-desktop >/dev/null || true
187-
188- clojure -Sdeps '{:mvn/local-repo "'"$cache"'/m2"}' -M:cljd compile
189-
190- # What the compile left behind, and only that. The analyzer
191- # project is keyed by the ClojureDart sha, so the directory
192- # under cache/ is copied wholesale rather than named here.
193- mkdir -p "$out/clojuredart"
194- cp -r .clojuredart/cache "$out/clojuredart/cache"
195- cp -r "$cache/m2" "$out/m2"
196- cp -r "$cache/gitlibs" "$out/gitlibs"
197- cp -r "$PUB_CACHE" "$out/pub-cache"
198-
199- # A fixed-output derivation may not reference a store path, and
200- # a resolved pub project is nothing but store paths:
201- # package_config.json names the Flutter SDK and every package
202- # in the cache by absolute path. So the analyzer project ships
203- # *unresolved* its pubspec and its analyzer.dart and nothing
204- # else and `flutter pub get --offline` re-resolves it against
205- # this cache at build time, where naming the store is allowed.
206- find "$out" \( -name '.dart_tool' -o -name '.flutter-plugins' \
207- -o -name '.flutter-plugins-dependencies' \) -prune -exec rm -rf {} +
208- find "$out" -name '.packages' -delete
209-
210-
211- # A fixed-output hash is a promise that two runs agree, so
212- # everything a tool writes *about* a run rather than about a
213- # dependency has to go: pub's log carries timestamps, Maven
214- # rewrites its resolution metadata on every resolve, and
215- # tools.gitlibs keeps bare clones it only needs in order to
216- # make a checkout. None of it is read offline.
217- #
218- # active_roots is the one that was actually breaking this. Pub
219- # records the project directories using the cache, sharded by
220- # a hash of the path, and $NIX_BUILD_TOP is different on every
221- # run -- so two builds whose hosted/ trees were byte-identical
222- # still disagreed, purely over which directory had asked. Four
223- # builds gave four hashes until this went.
224- rm -rf "$out/pub-cache/log" "$out/pub-cache/_temp" \
225- "$out/pub-cache/git" "$out/pub-cache/global_packages" \
226- "$out/pub-cache/bin" "$out/pub-cache/active_roots"
227-
228- # tools.gitlibs keeps a bare clone per URL under _repos/, and a
229- # bare clone is packfiles which two runs of the same fetch do
230- # not have to produce byte for byte. It cannot simply be
231- # deleted, because `procure` calls `ensure-git-dir` before it
232- # looks at anything else and would clone it again, over a
233- # network this has and the build that uses it does not.
234- #
235- # It does not need the objects, though. `procure` finds the sha
236- # with `match-exact` against the checkout already in libs/, so
237- # the bare repo only has to exist. Emptied and re-initialised,
238- # it is a fixed handful of files from the pinned git and the
239- # same on every run.
240- find "$out/gitlibs/_repos" -name HEAD | while read -r head; do
241- repo="$(dirname "$head")"
242- rm -rf "$repo"
243- git init --bare -q "$repo"
244- # The sample hooks are shell scripts, so they carry a
245- # `#!/nix/store/.../bash` line which is exactly the kind of
246- # store reference a fixed-output derivation may not hold. An
247- # empty bare repo nothing ever runs has no use for them.
248- rm -rf "$repo/hooks"
249- done
250- # pub's version listings, which record when they were fetched.
251- # This is the one that actually moved between two runs of this
252- # derivation: the package sources under hosted/ were identical
253- # and the listings beside them were not. Nothing offline reads
254- # them a resolution that already has every package on disk
255- # never asks pub.dev what versions exist.
256- find "$out/pub-cache" -name '.cache' -type d -prune -exec rm -rf {} +
257- find "$out/m2" \( -name '*.lastUpdated' -o -name '_remote.repositories' \
258- -o -name 'resolver-status.properties' -o -name '*.part' \
259- -o -name 'maven-metadata-*.xml*' \) -delete
260- find "$out" \( -name '.DS_Store' -o -name '*.log' -o -name '.git' \) \
261- -prune -exec rm -rf {} +
262- find "$out" -type d -empty -delete
263- chmod -R u+w "$out"
264-
265- # Last, after every cleanup above: anything still naming the
266- # store fails the fixed-output check, and the error names one
267- # path out of thousands of files. This names the files.
268- if refs="$(grep -rlI /nix/store "$out" 2>/dev/null)" && [ -n "$refs" ]; then
269- echo "cljd-deps: these still reference the store:" >&2
270- echo "$refs" | head -20 >&2
271- fi
272-
273- # If two runs disagree, this says which half to look in. Cheap,
274- # and the alternative is a hash mismatch with nothing attached.
275- for d in "$out"/*; do
276- echo "cljd-deps subtree $(basename "$d") $( (cd "$d" && find . -type f \
277- -exec sha256sum {} + | sort -k2 | sha256sum) )" >&2
278- done
279- for d in "$out"/pub-cache/*/*; do
280- [ -d "$d" ] || continue
281- # The path relative to $out, not the basename: two runs that
282- # disagree here disagree about *which* directory exists, and
283- # a bare `09` names nothing you can go and look at.
284- echo "cljd-deps pub ''${d#$out/} $( (cd "$d" && find . -type f \
285- -exec sha256sum {} + | sort -k2 | sha256sum) )" >&2
286- done
287- '';
288-
289- outputHashMode = "recursive";
290- outputHashAlgo = "sha256";
291- # Moves when flutter/deps.edn or flutter/pubspec.yaml move, and
292- # not when frq's own source does — see the stub above.
293- outputHash = "sha256-qSGx7WFdVyV7yu4R+EjiQHZcLqwDjYSlohiZcXB43DY=";
294- };
295-
296- # The Flutter desktop GUI, built rather than run out of the tree.
297- #
298- # `just flutter-desktop` is the working-tree loop and this is its
299- # opposite number: the source is the flake's, the output is a store
300- # path, and the build is a sandbox with no network. It is the first thing here that builds
301- # purely — the APK cannot, because Gradle fetches as it goes.
302- #
303- # Two stages, because the Dart does not exist until ClojureDart writes
304- # it. `preBuild` runs the compiler over `flutter/src` and `common/`
305- # with `--offline`, out of the caches `cljd-deps` fetched; everything
306- # after that is an ordinary Flutter application as far as nixpkgs is
307- # concerned.
308- #
309- # The caches are copied in rather than used where they lie. Maven,
310- # tools.gitlibs and pub all expect to be able to write to their own
311- # cache — a lock file, a resolved marker — and the store is read-only,
312- # so pointing them at $out of a fixed-output derivation fails in three
313- # different ways at three different depths.
314- #
315- # `src` is the whole tree and not `flutter/`: `flutter/deps.edn` puts
316- # `../common` on the classpath, which is the entire point of that
317- # directory, and a source root of `flutter/` would leave the screens
318- # outside it.
319- flutter-desktop-unwrapped = pkgs.flutter.buildFlutterApplication rec {
320- pname = "frq-flutter";
321- version = "0.1.0";
322-
323- src = lib.cleanSourceWith {
324- src = ./.;
325- # Two trees, and only two: `flutter/` is the app and `common/`
326- # is the screens its deps.edn puts on the classpath. The root is
327- # still the source root because of that `../common`, but letting
328- # the *whole* root in means every file in the repo is an input —
329- # so editing flake.nix or CLAUDE.md
330- # invalidated the entire Dart compile and paid ten minutes for a
331- # change the Flutter build cannot even see.
332- #
333- # Matched on the path relative to the root rather than on
334- # basename: `src` as a basename would also exclude `flutter/src`
335- # and `common/src`, which is everything that matters.
336- filter =
337- let root = toString ./.; in
338- path: type:
339- let
340- rel = lib.removePrefix (root + "/") (toString path);
341- inTree = d: rel == d || lib.hasPrefix (d + "/") rel;
342- in
343- (inTree "flutter" || inTree "common")
344- # Build trees and caches, which are large, machine-specific
345- # and would make every one of them a new store path.
346- && !(builtins.elem (baseNameOf path) [
347- "build" ".home" ".clojuredart" ".cpcache" "cljd-out"
348- ".dart_tool" "result" ".git" "buck-out"
349- ]);
350- };
351- sourceRoot = "source/flutter";
352-
353- # Read at eval time, so the lock in git is the lock that is built.
354- autoPubspecLock = ./flutter/pubspec.lock;
355-
356- # git, because tools.deps resolves the ClojureDart dependency through
357- # tools.gitlibs even when every byte of it is already on disk — see
358- # the _repos note in cljd-deps.
359- nativeBuildInputs = [ pkgs.clojure pkgs.jdk17 pkgs.git ];
360-
361- preBuild = ''
362- export PUB_CACHE="$NIX_BUILD_TOP/pub-cache"
363- export GITLIBS="$NIX_BUILD_TOP/gitlibs"
364- cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/pub-cache "$PUB_CACHE"
365- cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/gitlibs "$GITLIBS"
366- cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/m2 "$NIX_BUILD_TOP/m2"
367- mkdir -p .clojuredart
368- cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/clojuredart/cache .clojuredart/cache
369- chmod -R u+w "$PUB_CACHE" "$GITLIBS" "$NIX_BUILD_TOP/m2" .clojuredart
370-
371- # Resolve the analyzer project here rather than in cljd-deps,
372- # which was not allowed to name the store. Offline, out of the
373- # cache that derivation did fetch. ClojureDart only reaches for
374- # the network when `bin/analyzer.dart` is missing, and it is not.
375- for helper in .clojuredart/cache/*/cljd_helper; do
376- ( cd "$helper" && flutter pub get --offline )
377- done
378-
379- # --offline is what keeps `pub get` out of a sandbox that has no
380- # network; the analyzer project it would otherwise resolve is
381- # already in .clojuredart, put there by cljd-deps.
382- clojure -Sdeps "{:mvn/local-repo \"$NIX_BUILD_TOP/m2\"}" \
383- -M:cljd compile --offline
384- '';
385-
386- meta = {
387- description = "frq's screens on Flutter's Linux target (no GL launcher)";
388- mainProgram = "frq";
389- platforms = systems;
390- };
391- };
392-
393- # The same shape as `frq` above: a launcher, and a package that is a
394- # symlink to it. The reason is the same one `frqScript` gives — on
395- # NixOS the store's Mesa is the system's and the window opens, and
396- # anywhere else the real driver is the host's, so the process is
397- # handed to nixGL. Without it the store build dies on a distrobox
398- # Arch with "No provider of eglGetPlatformDisplayEXT found", which is
399- # that failure wearing an EGL hat.
400- #
401- # A wrapper *around* the built application rather than a `postFixup`
402- # inside it, because buildFlutterApplication's own dartFixupHook runs
403- # after postFixup and rewrites `bin/frq` — so anything done to that
404- # path from inside is undone on the way out.
405- flutter-desktop =
406- let
407- unwrapped =
408- self.packages.${pkgs.stdenv.hostPlatform.system}.flutter-desktop-unwrapped;
409-
410- # buildFlutterApplication's own wrapper appends a bare `/lib` to
411- # LD_LIBRARY_PATH — the host's, not the bundle's. Off NixOS that
412- # is a foreign library directory in front of nothing, and the app
413- # dies in the loader before main: first
414- #
415- # /lib/libc.so.6: undefined symbol: __pointer_chk_guard
416- #
417- # and, once the store's glibc is put ahead of it,
418- #
419- # libc.so.6: version `GLIBC_2.43' not found
420- # (required by /lib/libglib-2.0.so.0)
421- #
422- # which is the same bug wearing the other hat: the host's glib
423- # against the store's glibc. Ordering cannot fix a mixture, so
424- # the entry goes rather than moves. The wrapper is generated, so
425- # this edits a copy and asserts the edit landed — a silent miss
426- # here is a runtime failure on someone else's machine.
427- fixed = pkgs.runCommand "frq-flutter-wrapper" { } ''
428- mkdir -p "$out/bin"
429- sed "s|'/lib'||g" ${unwrapped}/bin/frq > "$out/bin/frq"
430- chmod +x "$out/bin/frq"
431- if grep -q "'/lib'" "$out/bin/frq"; then
432- echo "the /lib entry outlived the edit; look at the wrapper" >&2
433- exit 1
434- fi
435- '';
436-
437- script = pkgs.writeShellScript "frq" ''
438- runner=""
439- [ -e /run/current-system ] || runner="${nixGLFor pkgs}/bin/nixGLIntel"
440- exec ''${runner} ${fixed}/bin/frq "$@"
441- '';
442- in
443- pkgs.runCommand "frq-flutter-0.1.0"
444- {
445- meta = {
446- description = "frq's screens on Flutter's Linux target";
447- mainProgram = "frq";
448- platforms = systems;
449- };
450- }
451- ''
452- mkdir -p "$out/bin"
453- ln -s ${script} "$out/bin/frq"
454- '';
455- });
456-
457- devShells = forEachSystem (pkgs:
458- let
459- inherit (pkgs) lib;
460- in
461- {
462- # Dart on its own, for the tests that need no Flutter: the FFI
463- # binding to the Nim core runs on the plain VM, and making it wait
464- # for a Flutter toolchain would throw away the reason it is fast.
465- #
466- # Flutter bundles a Dart, so this is a duplicate in one sense. It is
467- # also thirty times smaller, and the point of the boundary is that
468- # you can check it without the thing on the other side of it.
469- dart = pkgs.mkShellNoCC {
470- name = "frq-dart";
471- packages = [ pkgs.dart pkgs.just ];
472-
473- # libfrqcore.so is linked against OpenSSL, and the process that
474- # dlopens it has to be able to find one. Named here rather than
475- # left to the host: a machine whose libssl is a different soname
476- # fails at `frq_init` with a message about the wrong library.
477- LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.openssl ];
478-
479- FRQ_DART = "1";
480- };
481-
482- # Nim, for `nim/` — the portable core as a native library. Just the
483- # compiler: the core has no dependencies outside Nim's own standard
484- # library, deliberately, because a dependency here is one that has
485- # to cross-compile to every target the Dart side runs on.
486- #
487- # `nim c` shells out to a C compiler, so this is mkShell and not
488- # mkShellNoCC: stdenv brings the one Nim will find.
489- nim = pkgs.mkShell {
490- name = "frq-nim";
491- packages = [ pkgs.nim pkgs.just ];
492-
493- # OpenSSL, because `-d:ssl` in nim/nim.cfg makes std/net link
494- # -lssl and -lcrypto: the IRC connection is TLS on :6697, which is
495- # the only port freeq actually listens on.
496- buildInputs = [ pkgs.openssl ];
497-
498- # And on the loader path as well as the linker's. Nim resolves the
499- # OpenSSL entry points through dynlib at run time, so without this
500- # `newContext` finds nothing behind the symbol and dies with a
501- # SIGSEGV that says nothing about SSL at all.
502- LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.openssl ];
503-
504- # The recipe's re-entry test, the way FRQ_FLUTTER_DESKTOP is the
505- # desktop one's.
506- FRQ_NIM = "1";
507- };
508-
509- # The APK toolchain. It is not in a default shell any more because
510- # there is no default shell: what used to be one belonged to the
511- # libcosmic frontend, and a Flutter build asks for a toolchain by
512- # name.
513- #
514- # Flutter brings its own Dart, Gradle and a JDK's worth of
515- # closure: Flutter brings its own Dart, Gradle and a JDK's worth of
516- # closure, and a desktop build has no use for any of it.
517- #
518- # `just apk` used to name these as `nix shell nixpkgs#clojure
519- # nixpkgs#jdk17 nixpkgs#flutter`, which is the flake registry's
520- # nixpkgs and not this flake's — so the Flutter under the APK
521- # floated while everything else was locked. Same three packages,
522- # from flake.lock now.
523- #
524- # JDK 17 and not newer on purpose: the Flutter template's Gradle
525- # plugin pins a Gradle that rejects a JDK it was released before,
526- # and the failure reads as an unsupported class file version rather
527- # than as a version mismatch.
528- flutter = pkgs.mkShellNoCC {
529- name = "frq-flutter";
530-
531- # git, because tools.deps resolves the ClojureDart dependency
532- # through tools.gitlibs even when every byte of it is already in
533- # the seeded cache — the same reason flutter-desktop-unwrapped
534- # names it. The host's git has always been there to answer; naming
535- # it means the shell does not depend on that.
536- packages = [ pkgs.clojure pkgs.jdk17 pkgs.flutter pkgs.just pkgs.git ];
537-
538- # Where the recipe copies from. Naming it here is also what makes
539- # entering the shell build it, so the first `just apk` does not
540- # stop for a few hundred megabytes of SDK with nothing said about
541- # why.
542- FRQ_ANDROID_SDK =
543- "${androidSdkFor pkgs.stdenv.hostPlatform.system}/libexec/android-sdk";
544-
545- # The Maven, gitlibs and pub caches the ClojureDart compile would
546- # otherwise fetch, plus the analyzer project it writes under
547- # .clojuredart. Here for both of FRQ_ANDROID_SDK's reasons: it is
548- # where the recipe copies from, and naming it is what makes
549- # entering the shell build it.
550- #
551- # The compile still runs online. These are a warm start and not a
552- # pin — `--offline` would be, and would turn adding a line to
553- # flutter/deps.edn into a re-hash of cljd-deps before anything
554- # compiled again. The sandbox build takes that trade because it
555- # has no network; the loop someone edits in should not.
556- FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}";
557- };
558-
559- # The other desktop GUI. Same ClojureDart half as the APK — one
560- # `clojure -M:cljd compile`, one flutter/src — over Flutter's Linux
561- # target instead of its Android one, so `flutter/linux/` is the
562- # runner and CMake and Ninja are the build rather than Gradle.
563- #
564- # A separate shell from `flutter` rather than one that carries both,
565- # because the halves are disjoint: this wants GTK and a C++ toolchain
566- # and no JDK, and the APK wants a JDK and an SDK and no GTK. Sharing
567- # them would mean every desktop build paying for a few hundred
568- # megabytes of Android SDK it never opens, which is the same argument
569- # that keeps Flutter out of the default shell.
570- #
571- # mkShell and not mkShellNoCC, unlike every other shell here: this is
572- # the one that actually compiles C++. stdenv brings the compiler, and
573- # gtk3 in buildInputs is what puts its .pc file where the runner's
574- # `pkg_check_modules(GTK gtk+-3.0)` can find it.
575- flutter-desktop = pkgs.mkShell {
576- name = "frq-flutter-desktop";
577-
578- # clojure and flutter are the APK shell's, and deliberately the
579- # same two: the Dart that runs here is generated by the same
580- # compiler from the same source, and a second Flutter version
581- # under it would be a second set of engine artifacts and a second
582- # answer to "does the phone build match the desktop one".
583- nativeBuildInputs = [
584- pkgs.clojure
585- pkgs.flutter
586- pkgs.just
587- pkgs.cmake
588- pkgs.ninja
589- pkgs.pkg-config
590- ];
591-
592- # gtk3 is the runner's own dependency; the rest are url_launcher's
593- # Linux implementation, which is a GTK plugin compiled into the
594- # bundle. path_provider needs nothing here — its Linux half is
595- # pure Dart over the XDG directories.
596- buildInputs = [ pkgs.gtk3 pkgs.glib ];
597-
598- # Where libfrqcore.so's OpenSSL lives. A named variable and NOT
599- # LD_LIBRARY_PATH, deliberately: this shell also runs Flutter
600- # through nixGL, which does its own careful things to the loader
601- # path, and a blanket LD_LIBRARY_PATH here is the sort of thing
602- # that breaks GL on one machine and not another. The `nim-spike`
603- # recipe prepends this for the app it launches and nothing else.
604- FRQ_OPENSSL_LIB = lib.makeLibraryPath [ pkgs.openssl ];
605-
606- # Flutter paints through GL, and off NixOS the driver that can do
607- # that is the host's, not the store's.
608- NIXGL = "${nixGLFor pkgs}/bin/nixGLIntel";
609-
610- # The `flutter` shell's, deliberately the same one and for the
611- # same reason clojure and flutter are: the ClojureDart half of
612- # both builds is one compile over one deps.edn, so a second set of
613- # caches would be a second answer to what it resolved against.
614- # This recipe reads the variable directly — it is inside this
615- # shell before it does any of the work — where `just apk` reaches
616- # for the flake output itself.
617- FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}";
618-
619- # The recipe's re-entry test. Nothing else sets it, so `just flutter-desktop`
620- # outside the shell re-enters and lands back on the same recipe —
621- # no flag to forget, and no second code path for someone who runs
622- # `nix develop .#flutter-desktop --command just flutter-desktop`
623- # by hand.
624- FRQ_FLUTTER_DESKTOP = "1";
625- };
626-
627- # No `flutter-web` shell here any more. The web target was the one
628- # that needed nothing of the host -- no JDK and no Android SDK as
629- # the APK wants, no GTK and no C++ and no nixGL as the desktop one
630- # does -- and a devShell whose only job is to hand over a Dart and
631- # a JVM is a devShell that a pinned tarball can replace. It did:
632- # `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by
633- # sha256, `tools/build-web.sh` builds out of them, and
634- # `.modal/flutter-web/` runs that same script on a plain Debian
635- # image with no store to populate.
636- #
637- # The two shells above stay. What they supply is a host toolchain,
638- # which is exactly what nix is better at than a tarball.
639- });
640-
641- };
642-}
deleted file mode 100644
@@ -1,642 +0,0 @@
1-{
2- # frq is a Flutter app whose source is ClojureDart, so what this flake
3- # provides is toolchains rather than a built program: the `flutter` shell
4- # that `just apk` compiles in, the `flutter-desktop` shell for the Linux
5- # target, and the Android SDK the first of those copies somewhere writable.
6- #
7- # nix develop .#flutter-desktop --command just flutter-desktop run
8- #
9- # On a machine that is not NixOS the GL driver is the host's and the loader
10- # will not find it, so the window never opens ("GL display: argument does not
11- # name a valid config"). The recipes handle that themselves: off NixOS they
12- # hand the process to nixGL, which puts the host's driver ahead of the
13- # store's. A distrobox/container Arch is the same case as a bare one.
14- description = "frq a freeq client in Flutter";
15-
16- inputs = {
17- nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
18-
19- # Only ever used off NixOS, to put the host GL driver on the loader path.
20- nixgl = {
21- url = "github:nix-community/nixGL";
22- inputs.nixpkgs.follows = "nixpkgs";
23- };
24- };
25-
26- outputs = { self, nixpkgs, nixgl }:
27- let
28- systems = [ "x86_64-linux" "aarch64-linux" ];
29- forEachSystem = f:
30- nixpkgs.lib.genAttrs systems (system: f nixpkgs.legacyPackages.${system});
31-
32- # Mesa, despite the name: it covers Intel and AMD alike. The NVIDIA
33- # wrappers are the ones that need --impure (they read the host kernel
34- # module's version), which is why this only ever reaches for Intel.
35- #
36- # Built from nixGL's default.nix rather than taken from its flake
37- # outputs, for the one argument the flake hardcodes on: `enable32bits`,
38- # which on x86_64 puts a second, i686 copy of mesa, its LLVM, and
39- # intel-media-driver into the wrapper. frq is 64-bit on both halves —
40- # the Rust cdylibs and the Chez runtime — so nothing here ever opens the
41- # 32-bit driver, and carrying it is most of the dev shell's closure.
42- nixGLFor = pkgs: (import nixgl {
43- inherit pkgs;
44- enable32bits = false;
45- }).nixGLIntel;
46-
47- # The Android SDK wants two things `nixpkgs.legacyPackages` cannot give:
48- # `allowUnfree`, because the SDK's own licence is not free, and
49- # `android_sdk.accept_license`, which is how you say so in a file rather
50- # than at a prompt a build has no terminal for. Neither can be set on a
51- # legacyPackages attribute after the fact, so this is a second import of
52- # the same locked nixpkgs rather than a second nixpkgs.
53- #
54- # This used to live in `just apk` as a `nix build --impure --expr` with
55- # `builtins.getFlake "github:NixOS/nixpkgs/nixos-unstable"` inside it —
56- # which fetched whatever nixos-unstable was that morning, not what
57- # flake.lock pins, so the SDK under the APK and the nixpkgs under
58- # everything else were free to drift apart. Here they are the same rev.
59- androidPkgsFor = system: import nixpkgs {
60- inherit system;
61- config = {
62- allowUnfree = true;
63- android_sdk.accept_license = true;
64- };
65- };
66-
67- # Only the floor Gradle stands on. It installs build-tools and a platform
68- # into ANDROID_HOME itself as it goes — see `just apk` for why that means
69- # a writable copy — so composing more of them here buys nothing.
70- #
71- # includeNDK = false deliberately: the app is Dart and path_provider is
72- # platform channels, so there is no native code to need one, and asking
73- # for it is a few hundred megabytes and a Gradle fetch of that exact NDK.
74- androidSdkFor = system:
75- let android = androidPkgsFor system; in
76- (android.androidenv.composeAndroidPackages {
77- cmdLineToolsVersion = "13.0";
78- buildToolsVersions = [ "34.0.0" ];
79- platformVersions = [ "35" "34" ];
80- includeNDK = false;
81- }).androidsdk;
82-
83- in
84- {
85- packages = forEachSystem (pkgs:
86- let
87- inherit (pkgs) lib;
88- in
89- {
90- # The Android SDK `just apk` copies into flutter/.home. A package
91- # rather than something the recipe evaluates inline, so that
92- # `nix build .#android-sdk` is how you pre-warm it and `nix flake
93- # show` admits it exists.
94- android-sdk = androidSdkFor pkgs.stdenv.hostPlatform.system;
95-
96- # There were `appimage` outputs here, and what they were for was a
97- # host without Nix: they squashed the whole closure into one
98- # runnable file, Mesa included, and the Mesa was not waste — off
99- # NixOS the launcher goes through nixGL, which needs a store Mesa to
100- # put the host's driver in front of. Nothing asks for that shape any
101- # more, and they were the last thing evaluating nix-appimage, which
102- # is why that input is gone too.
103-
104- # Everything `clojure -M:cljd compile` would otherwise reach the
105- # network for, fetched once and hashed.
106- #
107- # The compile needs three caches, and the reason this is one
108- # derivation rather than three is that only one of them is obvious.
109- # Maven and gitlibs are the ordinary tools.deps pair. The third is
110- # ClojureDart's own: `ensure-cljd-analyzer!` writes a *second*, whole
111- # pub project to `.clojuredart/cache/<cljd sha>/cljd_helper`, runs
112- # `pub add analyzer` in it, and then runs `bin/analyzer.dart` out of
113- # it for the duration of the compile — so a sandbox needs that
114- # project already resolved, not just the app's dependencies.
115- #
116- # Fixed-output, so it is allowed the network the rest of the build is
117- # not. What that costs is a hash to maintain, and the thing worth
118- # being exact about is *when*: this derivation never sees frq's
119- # source. It compiles a three-line throwaway project against the same
120- # `flutter/deps.edn` and the same `flutter/pubspec.yaml`, so the hash
121- # moves when a dependency moves and not when a screen changes. A
122- # stub, rather than `-P` and a hand-built analyzer dir, because
123- # running the real compiler once is the only way to be sure the
124- # caches are the ones it actually wants.
125- #
126- # PUB_CACHE lands in $out on purpose. The package_config.json inside
127- # cljd_helper carries absolute paths to whatever resolved it, so
128- # resolving into a build directory would bake in paths that stop
129- # existing the moment this derivation finishes. Pointed at $out they
130- # are store paths, and still true.
131- cljd-deps =
132- let
133- flutterPkg = pkgs.flutter;
134- in
135- pkgs.stdenvNoCC.mkDerivation {
136- name = "frq-cljd-deps";
137- dontUnpack = true;
138-
139- nativeBuildInputs = [
140- pkgs.clojure
141- pkgs.jdk17
142- flutterPkg
143- pkgs.git
144- pkgs.cacert
145- ];
146-
147- buildCommand = ''
148- export HOME="$NIX_BUILD_TOP/home"
149- # Resolved in the build directory and copied to $out at the
150- # end, never written there directly. A fixed-output derivation
151- # may not reference a store path and its own output is a store
152- # path, so pub writing its cache's absolute location into its
153- # own metadata is enough to fail the check.
154- cache="$NIX_BUILD_TOP/cache"
155- export PUB_CACHE="$cache/pub-cache"
156- export GITLIBS="$cache/gitlibs"
157- export SSL_CERT_FILE="${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt"
158- mkdir -p "$HOME" "$PUB_CACHE" "$GITLIBS" "$cache/m2"
159-
160- # The stub: our dependency files, nothing of our source.
161- # ../common is a :local/root dependency now, so it has to exist
162- # *and* carry a deps.edn for tools.deps to resolve an empty
163- # directory with that one file in it is enough.
164- proj="$NIX_BUILD_TOP/stub"
165- mkdir -p "$proj/src/stub" "$NIX_BUILD_TOP/common"
166- cp ${./common/deps.edn} "$NIX_BUILD_TOP/common/deps.edn"
167- cp ${./flutter/deps.edn} "$proj/deps.edn"
168- cp ${./flutter/pubspec.yaml} "$proj/pubspec.yaml"
169- # The lock, or `pub get` resolves against pub.dev and takes
170- # whatever satisfies the ranges today. Every build then fetches
171- # a slightly different set and the fixed-output hash is a
172- # promise nothing can keep.
173- cp ${./flutter/pubspec.lock} "$proj/pubspec.lock"
174- chmod u+w "$proj/deps.edn" "$proj/pubspec.yaml" "$proj/pubspec.lock"
175- cat > "$proj/src/stub/main.cljd" <<'EOF'
176- (ns stub.main)
177- (defn main [] nil)
178- EOF
179-
180- cd "$proj"
181- # `:main` has to name the stub, or the compiler goes looking for
182- # frq.main in a tree that is not here.
183- sed -i 's/:main frq\.main/:main stub.main/' deps.edn
184-
185- flutter config --no-analytics &>/dev/null || true
186- flutter config --enable-linux-desktop >/dev/null || true
187-
188- clojure -Sdeps '{:mvn/local-repo "'"$cache"'/m2"}' -M:cljd compile
189-
190- # What the compile left behind, and only that. The analyzer
191- # project is keyed by the ClojureDart sha, so the directory
192- # under cache/ is copied wholesale rather than named here.
193- mkdir -p "$out/clojuredart"
194- cp -r .clojuredart/cache "$out/clojuredart/cache"
195- cp -r "$cache/m2" "$out/m2"
196- cp -r "$cache/gitlibs" "$out/gitlibs"
197- cp -r "$PUB_CACHE" "$out/pub-cache"
198-
199- # A fixed-output derivation may not reference a store path, and
200- # a resolved pub project is nothing but store paths:
201- # package_config.json names the Flutter SDK and every package
202- # in the cache by absolute path. So the analyzer project ships
203- # *unresolved* its pubspec and its analyzer.dart and nothing
204- # else and `flutter pub get --offline` re-resolves it against
205- # this cache at build time, where naming the store is allowed.
206- find "$out" \( -name '.dart_tool' -o -name '.flutter-plugins' \
207- -o -name '.flutter-plugins-dependencies' \) -prune -exec rm -rf {} +
208- find "$out" -name '.packages' -delete
209-
210-
211- # A fixed-output hash is a promise that two runs agree, so
212- # everything a tool writes *about* a run rather than about a
213- # dependency has to go: pub's log carries timestamps, Maven
214- # rewrites its resolution metadata on every resolve, and
215- # tools.gitlibs keeps bare clones it only needs in order to
216- # make a checkout. None of it is read offline.
217- #
218- # active_roots is the one that was actually breaking this. Pub
219- # records the project directories using the cache, sharded by
220- # a hash of the path, and $NIX_BUILD_TOP is different on every
221- # run -- so two builds whose hosted/ trees were byte-identical
222- # still disagreed, purely over which directory had asked. Four
223- # builds gave four hashes until this went.
224- rm -rf "$out/pub-cache/log" "$out/pub-cache/_temp" \
225- "$out/pub-cache/git" "$out/pub-cache/global_packages" \
226- "$out/pub-cache/bin" "$out/pub-cache/active_roots"
227-
228- # tools.gitlibs keeps a bare clone per URL under _repos/, and a
229- # bare clone is packfiles which two runs of the same fetch do
230- # not have to produce byte for byte. It cannot simply be
231- # deleted, because `procure` calls `ensure-git-dir` before it
232- # looks at anything else and would clone it again, over a
233- # network this has and the build that uses it does not.
234- #
235- # It does not need the objects, though. `procure` finds the sha
236- # with `match-exact` against the checkout already in libs/, so
237- # the bare repo only has to exist. Emptied and re-initialised,
238- # it is a fixed handful of files from the pinned git and the
239- # same on every run.
240- find "$out/gitlibs/_repos" -name HEAD | while read -r head; do
241- repo="$(dirname "$head")"
242- rm -rf "$repo"
243- git init --bare -q "$repo"
244- # The sample hooks are shell scripts, so they carry a
245- # `#!/nix/store/.../bash` line which is exactly the kind of
246- # store reference a fixed-output derivation may not hold. An
247- # empty bare repo nothing ever runs has no use for them.
248- rm -rf "$repo/hooks"
249- done
250- # pub's version listings, which record when they were fetched.
251- # This is the one that actually moved between two runs of this
252- # derivation: the package sources under hosted/ were identical
253- # and the listings beside them were not. Nothing offline reads
254- # them a resolution that already has every package on disk
255- # never asks pub.dev what versions exist.
256- find "$out/pub-cache" -name '.cache' -type d -prune -exec rm -rf {} +
257- find "$out/m2" \( -name '*.lastUpdated' -o -name '_remote.repositories' \
258- -o -name 'resolver-status.properties' -o -name '*.part' \
259- -o -name 'maven-metadata-*.xml*' \) -delete
260- find "$out" \( -name '.DS_Store' -o -name '*.log' -o -name '.git' \) \
261- -prune -exec rm -rf {} +
262- find "$out" -type d -empty -delete
263- chmod -R u+w "$out"
264-
265- # Last, after every cleanup above: anything still naming the
266- # store fails the fixed-output check, and the error names one
267- # path out of thousands of files. This names the files.
268- if refs="$(grep -rlI /nix/store "$out" 2>/dev/null)" && [ -n "$refs" ]; then
269- echo "cljd-deps: these still reference the store:" >&2
270- echo "$refs" | head -20 >&2
271- fi
272-
273- # If two runs disagree, this says which half to look in. Cheap,
274- # and the alternative is a hash mismatch with nothing attached.
275- for d in "$out"/*; do
276- echo "cljd-deps subtree $(basename "$d") $( (cd "$d" && find . -type f \
277- -exec sha256sum {} + | sort -k2 | sha256sum) )" >&2
278- done
279- for d in "$out"/pub-cache/*/*; do
280- [ -d "$d" ] || continue
281- # The path relative to $out, not the basename: two runs that
282- # disagree here disagree about *which* directory exists, and
283- # a bare `09` names nothing you can go and look at.
284- echo "cljd-deps pub ''${d#$out/} $( (cd "$d" && find . -type f \
285- -exec sha256sum {} + | sort -k2 | sha256sum) )" >&2
286- done
287- '';
288-
289- outputHashMode = "recursive";
290- outputHashAlgo = "sha256";
291- # Moves when flutter/deps.edn or flutter/pubspec.yaml move, and
292- # not when frq's own source does — see the stub above.
293- outputHash = "sha256-qSGx7WFdVyV7yu4R+EjiQHZcLqwDjYSlohiZcXB43DY=";
294- };
295-
296- # The Flutter desktop GUI, built rather than run out of the tree.
297- #
298- # `just flutter-desktop` is the working-tree loop and this is its
299- # opposite number: the source is the flake's, the output is a store
300- # path, and the build is a sandbox with no network. It is the first thing here that builds
301- # purely — the APK cannot, because Gradle fetches as it goes.
302- #
303- # Two stages, because the Dart does not exist until ClojureDart writes
304- # it. `preBuild` runs the compiler over `flutter/src` and `common/`
305- # with `--offline`, out of the caches `cljd-deps` fetched; everything
306- # after that is an ordinary Flutter application as far as nixpkgs is
307- # concerned.
308- #
309- # The caches are copied in rather than used where they lie. Maven,
310- # tools.gitlibs and pub all expect to be able to write to their own
311- # cache — a lock file, a resolved marker — and the store is read-only,
312- # so pointing them at $out of a fixed-output derivation fails in three
313- # different ways at three different depths.
314- #
315- # `src` is the whole tree and not `flutter/`: `flutter/deps.edn` puts
316- # `../common` on the classpath, which is the entire point of that
317- # directory, and a source root of `flutter/` would leave the screens
318- # outside it.
319- flutter-desktop-unwrapped = pkgs.flutter.buildFlutterApplication rec {
320- pname = "frq-flutter";
321- version = "0.1.0";
322-
323- src = lib.cleanSourceWith {
324- src = ./.;
325- # Two trees, and only two: `flutter/` is the app and `common/`
326- # is the screens its deps.edn puts on the classpath. The root is
327- # still the source root because of that `../common`, but letting
328- # the *whole* root in means every file in the repo is an input —
329- # so editing flake.nix or CLAUDE.md
330- # invalidated the entire Dart compile and paid ten minutes for a
331- # change the Flutter build cannot even see.
332- #
333- # Matched on the path relative to the root rather than on
334- # basename: `src` as a basename would also exclude `flutter/src`
335- # and `common/src`, which is everything that matters.
336- filter =
337- let root = toString ./.; in
338- path: type:
339- let
340- rel = lib.removePrefix (root + "/") (toString path);
341- inTree = d: rel == d || lib.hasPrefix (d + "/") rel;
342- in
343- (inTree "flutter" || inTree "common")
344- # Build trees and caches, which are large, machine-specific
345- # and would make every one of them a new store path.
346- && !(builtins.elem (baseNameOf path) [
347- "build" ".home" ".clojuredart" ".cpcache" "cljd-out"
348- ".dart_tool" "result" ".git" "buck-out"
349- ]);
350- };
351- sourceRoot = "source/flutter";
352-
353- # Read at eval time, so the lock in git is the lock that is built.
354- autoPubspecLock = ./flutter/pubspec.lock;
355-
356- # git, because tools.deps resolves the ClojureDart dependency through
357- # tools.gitlibs even when every byte of it is already on disk — see
358- # the _repos note in cljd-deps.
359- nativeBuildInputs = [ pkgs.clojure pkgs.jdk17 pkgs.git ];
360-
361- preBuild = ''
362- export PUB_CACHE="$NIX_BUILD_TOP/pub-cache"
363- export GITLIBS="$NIX_BUILD_TOP/gitlibs"
364- cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/pub-cache "$PUB_CACHE"
365- cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/gitlibs "$GITLIBS"
366- cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/m2 "$NIX_BUILD_TOP/m2"
367- mkdir -p .clojuredart
368- cp -r ${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}/clojuredart/cache .clojuredart/cache
369- chmod -R u+w "$PUB_CACHE" "$GITLIBS" "$NIX_BUILD_TOP/m2" .clojuredart
370-
371- # Resolve the analyzer project here rather than in cljd-deps,
372- # which was not allowed to name the store. Offline, out of the
373- # cache that derivation did fetch. ClojureDart only reaches for
374- # the network when `bin/analyzer.dart` is missing, and it is not.
375- for helper in .clojuredart/cache/*/cljd_helper; do
376- ( cd "$helper" && flutter pub get --offline )
377- done
378-
379- # --offline is what keeps `pub get` out of a sandbox that has no
380- # network; the analyzer project it would otherwise resolve is
381- # already in .clojuredart, put there by cljd-deps.
382- clojure -Sdeps "{:mvn/local-repo \"$NIX_BUILD_TOP/m2\"}" \
383- -M:cljd compile --offline
384- '';
385-
386- meta = {
387- description = "frq's screens on Flutter's Linux target (no GL launcher)";
388- mainProgram = "frq";
389- platforms = systems;
390- };
391- };
392-
393- # The same shape as `frq` above: a launcher, and a package that is a
394- # symlink to it. The reason is the same one `frqScript` gives — on
395- # NixOS the store's Mesa is the system's and the window opens, and
396- # anywhere else the real driver is the host's, so the process is
397- # handed to nixGL. Without it the store build dies on a distrobox
398- # Arch with "No provider of eglGetPlatformDisplayEXT found", which is
399- # that failure wearing an EGL hat.
400- #
401- # A wrapper *around* the built application rather than a `postFixup`
402- # inside it, because buildFlutterApplication's own dartFixupHook runs
403- # after postFixup and rewrites `bin/frq` — so anything done to that
404- # path from inside is undone on the way out.
405- flutter-desktop =
406- let
407- unwrapped =
408- self.packages.${pkgs.stdenv.hostPlatform.system}.flutter-desktop-unwrapped;
409-
410- # buildFlutterApplication's own wrapper appends a bare `/lib` to
411- # LD_LIBRARY_PATH — the host's, not the bundle's. Off NixOS that
412- # is a foreign library directory in front of nothing, and the app
413- # dies in the loader before main: first
414- #
415- # /lib/libc.so.6: undefined symbol: __pointer_chk_guard
416- #
417- # and, once the store's glibc is put ahead of it,
418- #
419- # libc.so.6: version `GLIBC_2.43' not found
420- # (required by /lib/libglib-2.0.so.0)
421- #
422- # which is the same bug wearing the other hat: the host's glib
423- # against the store's glibc. Ordering cannot fix a mixture, so
424- # the entry goes rather than moves. The wrapper is generated, so
425- # this edits a copy and asserts the edit landed — a silent miss
426- # here is a runtime failure on someone else's machine.
427- fixed = pkgs.runCommand "frq-flutter-wrapper" { } ''
428- mkdir -p "$out/bin"
429- sed "s|'/lib'||g" ${unwrapped}/bin/frq > "$out/bin/frq"
430- chmod +x "$out/bin/frq"
431- if grep -q "'/lib'" "$out/bin/frq"; then
432- echo "the /lib entry outlived the edit; look at the wrapper" >&2
433- exit 1
434- fi
435- '';
436-
437- script = pkgs.writeShellScript "frq" ''
438- runner=""
439- [ -e /run/current-system ] || runner="${nixGLFor pkgs}/bin/nixGLIntel"
440- exec ''${runner} ${fixed}/bin/frq "$@"
441- '';
442- in
443- pkgs.runCommand "frq-flutter-0.1.0"
444- {
445- meta = {
446- description = "frq's screens on Flutter's Linux target";
447- mainProgram = "frq";
448- platforms = systems;
449- };
450- }
451- ''
452- mkdir -p "$out/bin"
453- ln -s ${script} "$out/bin/frq"
454- '';
455- });
456-
457- devShells = forEachSystem (pkgs:
458- let
459- inherit (pkgs) lib;
460- in
461- {
462- # Dart on its own, for the tests that need no Flutter: the FFI
463- # binding to the Nim core runs on the plain VM, and making it wait
464- # for a Flutter toolchain would throw away the reason it is fast.
465- #
466- # Flutter bundles a Dart, so this is a duplicate in one sense. It is
467- # also thirty times smaller, and the point of the boundary is that
468- # you can check it without the thing on the other side of it.
469- dart = pkgs.mkShellNoCC {
470- name = "frq-dart";
471- packages = [ pkgs.dart pkgs.just ];
472-
473- # libfrqcore.so is linked against OpenSSL, and the process that
474- # dlopens it has to be able to find one. Named here rather than
475- # left to the host: a machine whose libssl is a different soname
476- # fails at `frq_init` with a message about the wrong library.
477- LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.openssl ];
478-
479- FRQ_DART = "1";
480- };
481-
482- # Nim, for `nim/` — the portable core as a native library. Just the
483- # compiler: the core has no dependencies outside Nim's own standard
484- # library, deliberately, because a dependency here is one that has
485- # to cross-compile to every target the Dart side runs on.
486- #
487- # `nim c` shells out to a C compiler, so this is mkShell and not
488- # mkShellNoCC: stdenv brings the one Nim will find.
489- nim = pkgs.mkShell {
490- name = "frq-nim";
491- packages = [ pkgs.nim pkgs.just ];
492-
493- # OpenSSL, because `-d:ssl` in nim/nim.cfg makes std/net link
494- # -lssl and -lcrypto: the IRC connection is TLS on :6697, which is
495- # the only port freeq actually listens on.
496- buildInputs = [ pkgs.openssl ];
497-
498- # And on the loader path as well as the linker's. Nim resolves the
499- # OpenSSL entry points through dynlib at run time, so without this
500- # `newContext` finds nothing behind the symbol and dies with a
501- # SIGSEGV that says nothing about SSL at all.
502- LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.openssl ];
503-
504- # The recipe's re-entry test, the way FRQ_FLUTTER_DESKTOP is the
505- # desktop one's.
506- FRQ_NIM = "1";
507- };
508-
509- # The APK toolchain. It is not in a default shell any more because
510- # there is no default shell: what used to be one belonged to the
511- # libcosmic frontend, and a Flutter build asks for a toolchain by
512- # name.
513- #
514- # Flutter brings its own Dart, Gradle and a JDK's worth of
515- # closure: Flutter brings its own Dart, Gradle and a JDK's worth of
516- # closure, and a desktop build has no use for any of it.
517- #
518- # `just apk` used to name these as `nix shell nixpkgs#clojure
519- # nixpkgs#jdk17 nixpkgs#flutter`, which is the flake registry's
520- # nixpkgs and not this flake's — so the Flutter under the APK
521- # floated while everything else was locked. Same three packages,
522- # from flake.lock now.
523- #
524- # JDK 17 and not newer on purpose: the Flutter template's Gradle
525- # plugin pins a Gradle that rejects a JDK it was released before,
526- # and the failure reads as an unsupported class file version rather
527- # than as a version mismatch.
528- flutter = pkgs.mkShellNoCC {
529- name = "frq-flutter";
530-
531- # git, because tools.deps resolves the ClojureDart dependency
532- # through tools.gitlibs even when every byte of it is already in
533- # the seeded cache — the same reason flutter-desktop-unwrapped
534- # names it. The host's git has always been there to answer; naming
535- # it means the shell does not depend on that.
536- packages = [ pkgs.clojure pkgs.jdk17 pkgs.flutter pkgs.just pkgs.git ];
537-
538- # Where the recipe copies from. Naming it here is also what makes
539- # entering the shell build it, so the first `just apk` does not
540- # stop for a few hundred megabytes of SDK with nothing said about
541- # why.
542- FRQ_ANDROID_SDK =
543- "${androidSdkFor pkgs.stdenv.hostPlatform.system}/libexec/android-sdk";
544-
545- # The Maven, gitlibs and pub caches the ClojureDart compile would
546- # otherwise fetch, plus the analyzer project it writes under
547- # .clojuredart. Here for both of FRQ_ANDROID_SDK's reasons: it is
548- # where the recipe copies from, and naming it is what makes
549- # entering the shell build it.
550- #
551- # The compile still runs online. These are a warm start and not a
552- # pin — `--offline` would be, and would turn adding a line to
553- # flutter/deps.edn into a re-hash of cljd-deps before anything
554- # compiled again. The sandbox build takes that trade because it
555- # has no network; the loop someone edits in should not.
556- FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}";
557- };
558-
559- # The other desktop GUI. Same ClojureDart half as the APK — one
560- # `clojure -M:cljd compile`, one flutter/src — over Flutter's Linux
561- # target instead of its Android one, so `flutter/linux/` is the
562- # runner and CMake and Ninja are the build rather than Gradle.
563- #
564- # A separate shell from `flutter` rather than one that carries both,
565- # because the halves are disjoint: this wants GTK and a C++ toolchain
566- # and no JDK, and the APK wants a JDK and an SDK and no GTK. Sharing
567- # them would mean every desktop build paying for a few hundred
568- # megabytes of Android SDK it never opens, which is the same argument
569- # that keeps Flutter out of the default shell.
570- #
571- # mkShell and not mkShellNoCC, unlike every other shell here: this is
572- # the one that actually compiles C++. stdenv brings the compiler, and
573- # gtk3 in buildInputs is what puts its .pc file where the runner's
574- # `pkg_check_modules(GTK gtk+-3.0)` can find it.
575- flutter-desktop = pkgs.mkShell {
576- name = "frq-flutter-desktop";
577-
578- # clojure and flutter are the APK shell's, and deliberately the
579- # same two: the Dart that runs here is generated by the same
580- # compiler from the same source, and a second Flutter version
581- # under it would be a second set of engine artifacts and a second
582- # answer to "does the phone build match the desktop one".
583- nativeBuildInputs = [
584- pkgs.clojure
585- pkgs.flutter
586- pkgs.just
587- pkgs.cmake
588- pkgs.ninja
589- pkgs.pkg-config
590- ];
591-
592- # gtk3 is the runner's own dependency; the rest are url_launcher's
593- # Linux implementation, which is a GTK plugin compiled into the
594- # bundle. path_provider needs nothing here — its Linux half is
595- # pure Dart over the XDG directories.
596- buildInputs = [ pkgs.gtk3 pkgs.glib ];
597-
598- # Where libfrqcore.so's OpenSSL lives. A named variable and NOT
599- # LD_LIBRARY_PATH, deliberately: this shell also runs Flutter
600- # through nixGL, which does its own careful things to the loader
601- # path, and a blanket LD_LIBRARY_PATH here is the sort of thing
602- # that breaks GL on one machine and not another. The `nim-spike`
603- # recipe prepends this for the app it launches and nothing else.
604- FRQ_OPENSSL_LIB = lib.makeLibraryPath [ pkgs.openssl ];
605-
606- # Flutter paints through GL, and off NixOS the driver that can do
607- # that is the host's, not the store's.
608- NIXGL = "${nixGLFor pkgs}/bin/nixGLIntel";
609-
610- # The `flutter` shell's, deliberately the same one and for the
611- # same reason clojure and flutter are: the ClojureDart half of
612- # both builds is one compile over one deps.edn, so a second set of
613- # caches would be a second answer to what it resolved against.
614- # This recipe reads the variable directly — it is inside this
615- # shell before it does any of the work — where `just apk` reaches
616- # for the flake output itself.
617- FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}";
618-
619- # The recipe's re-entry test. Nothing else sets it, so `just flutter-desktop`
620- # outside the shell re-enters and lands back on the same recipe —
621- # no flag to forget, and no second code path for someone who runs
622- # `nix develop .#flutter-desktop --command just flutter-desktop`
623- # by hand.
624- FRQ_FLUTTER_DESKTOP = "1";
625- };
626-
627- # No `flutter-web` shell here any more. The web target was the one
628- # that needed nothing of the host -- no JDK and no Android SDK as
629- # the APK wants, no GTK and no C++ and no nixGL as the desktop one
630- # does -- and a devShell whose only job is to hand over a Dart and
631- # a JVM is a devShell that a pinned tarball can replace. It did:
632- # `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by
633- # sha256, `tools/build-web.sh` builds out of them, and
634- # `.modal/flutter-web/` runs that same script on a plain Debian
635- # image with no store to populate.
636- #
637- # The two shells above stay. What they supply is a host toolchain,
638- # which is exactly what nix is better at than a tarball.
639- });
640-
641- };
642-}
modified flutter/README.md +24 -40
@@ -76,39 +76,36 @@ all of them — `clojure -M:cljd compile` over `src/` and `../common` — and wh
7676 differs is only what Flutter is asked to wrap it in.
7777
7878 ```bash
79-just apk # the debug APK
80-just apk install # and onto a connected device
81-just apk run # and launched
82-just apk log # logcat
79+just build apk # the debug APK
80+just run apk # onto a connected device, and launched
81+just run log # logcat
8382
84-just flutter-desktop # the debug Linux bundle
85-just flutter-desktop run # and the window
83+just build desktop # the debug Linux bundle
84+just run desktop # and the window
8685
87-just flutter-web # the web bundle
88-just flutter-web serve # and served on :8080
86+just build web # the web bundle
87+just run web # and served on :8080
8988 ```
9089
9190 ### The desktop one
9291
93-`just flutter-desktop` is this tree under Flutter's Linux target — the same
92+`just build desktop` is this tree under Flutter's Linux target — the same
9493 screens out of `common/frq/screens/` as the APK, with `frq.hiccup` emitting
9594 Flutter widgets. There used to be a second desktop GUI beside it, libcosmic
9695 under jolt, walking the same hiccup through a different renderer; it is gone.
9796
98-Its toolchain is `devShells.flutter-desktop`, which is the APK shell with the
99-Android half swapped out: clojure and Flutter are the same two packages at the
100-same pinned rev, and CMake, Ninja, pkg-config and GTK stand where the JDK and
97+Its toolchain is the same `.toolchain/` every other target uses — clojure and
98+Flutter, at the same pins — with CMake, Ninja, pkg-config and GTK coming from
99+the host where the JDK and
101100 the SDK do. Kept separate rather than merged into one shell because the halves
102101 are disjoint — a desktop build has no use for a few hundred megabytes of
103102 Android SDK.
104103
105104 Still impure, for one of the two reasons the APK is: pub.dev resolution and
106-Flutter's engine artifacts are network. What it does *not* need is the
107-writable-`ANDROID_HOME` dance, since nothing here writes into the store — so
108-there is no `flutter/.home` on this path.
105+Flutter's engine artifacts are network. What it does *not* need is the Android
106+SDK, so `just build desktop` never asks `tools/toolchain.sh` for one.
109107
110-nixGL off NixOS: Flutter paints through GL and the driver that can do that is
111-the host's.
108+GL is the host's, as is the driver that can do it.
112109
113110 `linux/` is the Flutter template's GTK runner, renamed — `frq` rather than
114111 `cljd_flutter`, and `uk.nandi.frq` rather than `com.example.cljd_flutter`, so
@@ -132,28 +129,15 @@ Two things the desktop target changed in the Dart, both of them cases where
132129 ### The APK
133130
134131 Impure on purpose. Gradle resolves its own dependencies over the network and
135-installs build-tools and a platform into `ANDROID_HOME` as it goes, so it can
136-neither run in a sandbox nor write to the store. What nix gives is the
137-toolchain — clojure, a JDK, Flutter, and an SDK composed by androidenv — and
138-the recipe copies that SDK to `flutter/.home` for Gradle to finish off. That
139-copy and everything Gradle leaves behind are gitignored.
140-
141-All of it is the flake's, which it did not used to be. The toolchain was
142-`nix shell nixpkgs#clojure nixpkgs#jdk17 nixpkgs#flutter` and the SDK was a
143-`nix build --impure --expr` around `builtins.getFlake
144-"github:NixOS/nixpkgs/nixos-unstable"` — two references to an *unlocked*
145-nixpkgs, so the Flutter that compiled the APK and the nixpkgs under everything
146-else could drift apart without flake.lock changing a line. They are
147-`devShells.<system>.flutter` and `packages.<system>.android-sdk` now, at the
148-pinned rev, and the recipe is `nix develop .#flutter --command` over
149-`nix build .#android-sdk`.
150-
151-The SDK needs `allowUnfree` and `android_sdk.accept_license`, which cannot be
152-set on a `legacyPackages` attribute after the fact — hence `androidPkgsFor` in
153-the flake, a second `import` of the same locked input rather than a second
154-nixpkgs. The Flutter toolchain is kept out of the default dev shell: it brings
155-its own Dart and a JDK's worth of closure, and a desktop build wants none of
156-it.
132+has sdkmanager install build-tools and a platform into `ANDROID_HOME` as it
133+goes, so it cannot run in a sandbox and the SDK it writes to has to be ours.
134+`just tools android` fetches Google's command-line tools by pinned sha256 into
135+`.toolchain/android-sdk` and accepts the licences; Gradle finishes the job from
136+there. All of it is gitignored.
137+
138+The platform and build-tools versions are deliberately not pinned here: they
139+come from whatever Flutter asks Gradle for, and pinning them in a second place
140+is how the two drift apart.
157141
158142 Two things the Flutter template wanted that are deliberately not here. There is
159143 no `ndkVersion` in `android/app/build.gradle.kts`: setting it makes Gradle
@@ -249,7 +233,7 @@ move from. What remains is work the port never covered:
249233 listen on localhost for a browser it does not own. That wants an app link or
250234 a custom scheme, an intent filter, and a redirect URI the broker will accept
251235 — a decision about freeq's broker, not a porting problem. The web build has
252- its own answer in `frq.oauth.web`, and `just web-local` is why it only
236+ its own answer in `frq.oauth.web`, and `just serve` is why it only
253237 completes on localhost.
254238 2. **Calls.** The signaling is IRC and is still in the screens; the media plane
255239 it drove was `libjoltmoq` — Opus, H.264, V4L2, ALSA, MoQ over QUIC — under
@@ -76,39 +76,36 @@ all of them — `clojure -M:cljd compile` over `src/` and `../common` — and wh
76 differs is only what Flutter is asked to wrap it in.76 differs is only what Flutter is asked to wrap it in.
77 77
78 ```bash78 ```bash
79-just apk # the debug APK79+just build apk # the debug APK
80-just apk install # and onto a connected device80+just run apk # onto a connected device, and launched
81-just apk run # and launched81+just run log # logcat
82-just apk log # logcat
83 82
84-just flutter-desktop # the debug Linux bundle83+just build desktop # the debug Linux bundle
85-just flutter-desktop run # and the window84+just run desktop # and the window
86 85
87-just flutter-web # the web bundle86+just build web # the web bundle
88-just flutter-web serve # and served on :808087+just run web # and served on :8080
89 ```88 ```
90 89
91 ### The desktop one90 ### The desktop one
92 91
93-`just flutter-desktop` is this tree under Flutter's Linux target — the same92+`just build desktop` is this tree under Flutter's Linux target — the same
94 screens out of `common/frq/screens/` as the APK, with `frq.hiccup` emitting93 screens out of `common/frq/screens/` as the APK, with `frq.hiccup` emitting
95 Flutter widgets. There used to be a second desktop GUI beside it, libcosmic94 Flutter widgets. There used to be a second desktop GUI beside it, libcosmic
96 under jolt, walking the same hiccup through a different renderer; it is gone.95 under jolt, walking the same hiccup through a different renderer; it is gone.
97 96
98-Its toolchain is `devShells.flutter-desktop`, which is the APK shell with the97+Its toolchain is the same `.toolchain/` every other target uses — clojure and
99-Android half swapped out: clojure and Flutter are the same two packages at the98+Flutter, at the same pins — with CMake, Ninja, pkg-config and GTK coming from
100-same pinned rev, and CMake, Ninja, pkg-config and GTK stand where the JDK and99+the host where the JDK and
101 the SDK do. Kept separate rather than merged into one shell because the halves100 the SDK do. Kept separate rather than merged into one shell because the halves
102 are disjoint — a desktop build has no use for a few hundred megabytes of101 are disjoint — a desktop build has no use for a few hundred megabytes of
103 Android SDK.102 Android SDK.
104 103
105 Still impure, for one of the two reasons the APK is: pub.dev resolution and104 Still impure, for one of the two reasons the APK is: pub.dev resolution and
106-Flutter's engine artifacts are network. What it does *not* need is the105+Flutter's engine artifacts are network. What it does *not* need is the Android
107-writable-`ANDROID_HOME` dance, since nothing here writes into the store — so106+SDK, so `just build desktop` never asks `tools/toolchain.sh` for one.
108-there is no `flutter/.home` on this path.
109 107
110-nixGL off NixOS: Flutter paints through GL and the driver that can do that is108+GL is the host's, as is the driver that can do it.
111-the host's.
112 109
113 `linux/` is the Flutter template's GTK runner, renamed — `frq` rather than110 `linux/` is the Flutter template's GTK runner, renamed — `frq` rather than
114 `cljd_flutter`, and `uk.nandi.frq` rather than `com.example.cljd_flutter`, so111 `cljd_flutter`, and `uk.nandi.frq` rather than `com.example.cljd_flutter`, so
@@ -132,28 +129,15 @@ Two things the desktop target changed in the Dart, both of them cases where
132 ### The APK129 ### The APK
133 130
134 Impure on purpose. Gradle resolves its own dependencies over the network and131 Impure on purpose. Gradle resolves its own dependencies over the network and
135-installs build-tools and a platform into `ANDROID_HOME` as it goes, so it can132+has sdkmanager install build-tools and a platform into `ANDROID_HOME` as it
136-neither run in a sandbox nor write to the store. What nix gives is the133+goes, so it cannot run in a sandbox and the SDK it writes to has to be ours.
137-toolchain — clojure, a JDK, Flutter, and an SDK composed by androidenv — and134+`just tools android` fetches Google's command-line tools by pinned sha256 into
138-the recipe copies that SDK to `flutter/.home` for Gradle to finish off. That135+`.toolchain/android-sdk` and accepts the licences; Gradle finishes the job from
139-copy and everything Gradle leaves behind are gitignored.136+there. All of it is gitignored.
140-137+
141-All of it is the flake's, which it did not used to be. The toolchain was138+The platform and build-tools versions are deliberately not pinned here: they
142-`nix shell nixpkgs#clojure nixpkgs#jdk17 nixpkgs#flutter` and the SDK was a139+come from whatever Flutter asks Gradle for, and pinning them in a second place
143-`nix build --impure --expr` around `builtins.getFlake140+is how the two drift apart.
144-"github:NixOS/nixpkgs/nixos-unstable"` — two references to an *unlocked*
145-nixpkgs, so the Flutter that compiled the APK and the nixpkgs under everything
146-else could drift apart without flake.lock changing a line. They are
147-`devShells.<system>.flutter` and `packages.<system>.android-sdk` now, at the
148-pinned rev, and the recipe is `nix develop .#flutter --command` over
149-`nix build .#android-sdk`.
150-
151-The SDK needs `allowUnfree` and `android_sdk.accept_license`, which cannot be
152-set on a `legacyPackages` attribute after the fact — hence `androidPkgsFor` in
153-the flake, a second `import` of the same locked input rather than a second
154-nixpkgs. The Flutter toolchain is kept out of the default dev shell: it brings
155-its own Dart and a JDK's worth of closure, and a desktop build wants none of
156-it.
157 141
158 Two things the Flutter template wanted that are deliberately not here. There is142 Two things the Flutter template wanted that are deliberately not here. There is
159 no `ndkVersion` in `android/app/build.gradle.kts`: setting it makes Gradle143 no `ndkVersion` in `android/app/build.gradle.kts`: setting it makes Gradle
@@ -249,7 +233,7 @@ move from. What remains is work the port never covered:
249 listen on localhost for a browser it does not own. That wants an app link or233 listen on localhost for a browser it does not own. That wants an app link or
250 a custom scheme, an intent filter, and a redirect URI the broker will accept234 a custom scheme, an intent filter, and a redirect URI the broker will accept
251 — a decision about freeq's broker, not a porting problem. The web build has235 — a decision about freeq's broker, not a porting problem. The web build has
252- its own answer in `frq.oauth.web`, and `just web-local` is why it only236+ its own answer in `frq.oauth.web`, and `just serve` is why it only
253 completes on localhost.237 completes on localhost.
254 2. **Calls.** The signaling is IRC and is still in the screens; the media plane238 2. **Calls.** The signaling is IRC and is still in the screens; the media plane
255 it drove was `libjoltmoq` — Opus, H.264, V4L2, ALSA, MoQ over QUIC — under239 it drove was `libjoltmoq` — Opus, H.264, V4L2, ALSA, MoQ over QUIC — under
modified justfile +203 -445
@@ -1,384 +1,171 @@
1-# The work is here now, in recipe bodies, where it used to be in scripts/ —
2-# babashka scripts run through a scripts/bb that went looking for a babashka.
3-# That indirection bought one thing worth having, a shared way to reach nix on
4-# a host that keeps it in a container, and `nix` below is the whole of it.
1+# Six verbs over one tree.
52 #
6-# Every recipe that builds has the same shape: outside the dev shell, re-enter
7-# it and come back to this same recipe; inside, do the work. The re-entry test
8-# is an environment variable only the shell sets — no flag to forget, and no
9-# second code path for someone who runs `nix develop --command just ...` by
10-# hand.
3+# There is no nix here any more. `tools/toolchain.sh` fetches Flutter (which
4+# carries Dart), a JDK, the Clojure CLI and Nim as sha256-pinned tarballs into
5+# `.toolchain/`, and every recipe below runs inside the environment that
6+# script prints. One mechanism, and the same one the containers in `.modal/`
7+# use — which is what lets a plain Debian image build this.
8+#
9+# What the host still brings: a C compiler (Nim shells out to one), OpenSSL
10+# (nim.cfg is `-d:ssl`), git, curl, unzip, python3. For `run desktop` and the
11+# other Linux builds, GTK and the usual CMake/Ninja/pkg-config.
12+#
13+# just build TARGET apk desktop web lib ui app
14+# just run TARGET apk desktop web ui app
15+# just test SUITE all nim dart common live
16+# just modal CONTAINER dev web
17+# just serve [PORT] the Modal-built web bundle, on localhost
18+# just tools ... the toolchain itself
1119
1220 set shell := ["bash", "-euo", "pipefail", "-c"]
1321
14-# Every recipe below is a `#!` script and passes its arguments on with "$@".
15-# Without this that is empty in one — just interpolates into a shebang recipe
16-# rather than handing it argv — and a recipe silently ignored its flags.
22+# Every recipe is a `#!` script. Without this, "$@" is empty in one of them —
23+# just interpolates into a shebang recipe rather than handing it argv.
1724 set positional-arguments
1825
19-# nix is not on every host this runs on: on the machine these recipes were
20-# written for it lives in an Arch distrobox, at the same path — which is why
21-# the container is entered rather than the tree copied into it. See CLAUDE.md.
22-nix := `command -v nix >/dev/null 2>&1 && echo nix || echo "distrobox enter arch -- nix"`
23-
24-# --max-jobs 0 is what sends the work to the `builders` entry rather than
25-# compiling it here. Left to the default, nix prefers the local machine, and a
26-# cold Flutter toolchain is a lot of compiling on a laptop — for a derivation a
27-# remote builder has likely built already. FRQ_MAX_JOBS=auto is the way out on
28-# a machine with no builder configured.
29-jobs := env("FRQ_MAX_JOBS", "0")
26+root := justfile_directory()
27+tc := justfile_directory() / "tools/toolchain.sh"
3028
3129 default:
3230 @just --list
3331
34-# The APK: ClojureDart compiled to Dart, then Flutter's Gradle build.
35-#
36-# Impure on purpose, and worth saying why rather than leaving it to be
37-# discovered. Gradle resolves its own dependencies over the network and
38-# installs build-tools and a platform into ANDROID_HOME as it goes, so it
39-# cannot run in a sandbox and cannot write to the store. What nix gives here
40-# is the toolchain — clojure, a JDK, Flutter, and an SDK composed by
41-# androidenv — and the recipe copies that SDK somewhere writable
42-# (flutter/.home) for Gradle to finish off. That copy and everything Gradle
43-# leaves behind are gitignored.
44-#
45-# No ndkVersion in android/app/build.gradle.kts, for the same reason: the
46-# Flutter template sets it, setting it makes Gradle fetch that exact NDK, and
47-# there is no native code here to need one.
48-#
49-# just apk build the debug APK
50-# just apk install build it and put it on a connected device
51-# just apk run install and launch
52-# just apk log logcat, filtered to this app
53-apk action="build":
32+# The toolchain, directly: `just tools versions`, `just tools android`,
33+# `just tools exec -- flutter doctor`.
34+[doc('the toolchain itself: versions, android, exec -- CMD')]
35+tools *args:
36+ #!/usr/bin/env bash
37+ cd "{{root}}"
38+ exec "{{tc}}" "$@"
39+
40+# Build a target.
41+#
42+# apk the Android app, via Gradle
43+# desktop the ClojureDart app on Flutter's Linux target
44+# web the same screens compiled to JavaScript, into build/web
45+# lib the Nim core as build/nim/libfrqcore.so
46+# ui Nim owning the state and the screens, painted by Flutter
47+# app the ClojureDart app with the Nim core as its transport only
48+[doc('build a target: apk desktop web lib ui app')]
49+build target="desktop":
5450 #!/usr/bin/env bash
5551 set -euo pipefail
56- cd "{{justfile_directory()}}/flutter"
57-
58- # The flake's, not an --impure --expr against whatever nixos-unstable is
59- # today: the licence config the SDK needs lives in `androidPkgsFor` now,
60- # so this is an ordinary output at the rev flake.lock pins.
61- sdk="$(nix build --no-link --print-out-paths \
62- "{{justfile_directory()}}#android-sdk")/libexec/android-sdk"
63-
64- # adb keeps the key the phone has already trusted under the real HOME, and
65- # HOME moves below so Gradle can write into the SDK copy. Told where to
66- # look, adb keeps its identity; left to find $HOME/.android it generates a
67- # new one, the device stops recognising this machine, and the deploy ends
68- # in "no devices/emulators found" while `adb devices` in any other shell
69- # lists it perfectly well.
70- export ANDROID_USER_HOME="${ANDROID_USER_HOME:-$HOME/.android}"
71-
72- export HOME="$PWD/.home"
73- export ANDROID_HOME="$HOME/android-sdk"
74- export ANDROID_SDK_ROOT="$ANDROID_HOME"
75- mkdir -p "$HOME"
76-
77- # Gradle writes into ANDROID_HOME, so it is a copy rather than the store
78- # path. Made once and kept: re-copying would throw away the build-tools
79- # and platform Gradle installed into it on the last run.
80- if [ ! -d "$ANDROID_HOME" ]; then
81- cp -r "$sdk" "$ANDROID_HOME"
82- chmod -R u+w "$ANDROID_HOME"
83- fi
84-
85- # The compile's three caches, same flake-output-and-copy shape as the SDK
86- # and for the same reason: tools.deps and pub both write into theirs, and
87- # the store is read-only. What this buys is the "Resolving dependencies…
88- # Downloading packages…" that used to open every run.
89- deps="$(nix build --no-link --print-out-paths \
90- "{{justfile_directory()}}#cljd-deps")"
91-
92- # Neither of these follows HOME. Both are read off the JVM's user.home,
93- # which comes from /etc/passwd rather than the environment — so moving
94- # HOME below is not enough to move them, and left alone they would be the
95- # real ~/.m2 and ~/.gitlibs, shared with every other project on the box.
96- export GITLIBS="$HOME/gitlibs"
97- m2="$HOME/m2"
98- export PUB_CACHE="$HOME/.pub-cache"
99-
100- # Seeded once each and then left alone, exactly as ANDROID_HOME is: after
101- # the first compile these hold whatever the working tree has asked for
102- # since, and re-copying would throw that away.
103- seed() {
104- [ -e "$2" ] && return 0
105- mkdir -p "$(dirname "$2")"
106- cp -r "$deps/$1" "$2"
107- chmod -R u+w "$2"
108- }
109- seed m2 "$m2"
110- seed gitlibs "$GITLIBS"
111- seed pub-cache "$PUB_CACHE"
112- seed clojuredart/cache "$PWD/.clojuredart/cache"
113-
114- # Also the flake's. `nix shell nixpkgs#...` read the registry, which is a
115- # different and unlocked nixpkgs — the Flutter that built the APK could
116- # move under it without flake.lock changing a line.
117- flutter="nix develop {{justfile_directory()}}#flutter --command"
118-
119- # cljd-deps ships the analyzer project unresolved — a fixed-output
120- # derivation may not name the store, and a resolved pub project is
121- # nothing but store paths. So it is resolved here instead, offline,
122- # against the cache that derivation did fetch. ClojureDart reaches for the
123- # network only when bin/analyzer.dart is missing, and after this it is not.
124- for helper in .clojuredart/cache/*/cljd_helper; do
125- [ -d "$helper" ] || continue
126- [ -e "$helper/.dart_tool/package_config.json" ] && continue
127- ( cd "$helper" && $flutter flutter pub get --offline )
128- done
129-
130- $flutter clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile
131-
132- # Rewritten every run: it carries absolute store paths, and the flutter
133- # one moves whenever nixpkgs does.
134- $flutter flutter config --android-sdk "$ANDROID_HOME" >/dev/null
135-
136- apk=build/app/outputs/flutter-apk/app-debug.apk
137- adb="${ADB:-$ANDROID_HOME/platform-tools/adb}"
138-
139- case "{{action}}" in
140- build) $flutter flutter build apk --debug ;;
141- install) $flutter flutter build apk --debug && "$adb" install -r "$apk" ;;
142- run) $flutter flutter build apk --debug && "$adb" install -r "$apk" \
143- && "$adb" shell monkey -p uk.nandi.frq -c android.intent.category.LAUNCHER 1 ;;
144- log) "$adb" logcat -s flutter ;;
145- *) echo "usage: just apk [build|install|run|log]" >&2; exit 1 ;;
52+ cd "{{root}}"
53+ case "{{target}}" in
54+ apk) just _flutter apk build ;;
55+ desktop) just _flutter desktop build ;;
56+ web) exec tools/build-web.sh build ;;
57+ lib) just _nim-lib ;;
58+ ui) just _flutter ui build ;;
59+ app) just _flutter app build ;;
60+ *) echo "usage: just build [apk|desktop|web|lib|ui|app]" >&2; exit 1 ;;
14661 esac
14762
148-# What may appear in common/, checked. Needs nothing built: it reads the
149-# source, so it is the one check that runs anywhere, and CI runs exactly this.
150-check-common:
151- #!/usr/bin/env bash
152- python3 tools/check-common.py common
153-
154-# The desktop GUI: the same screens the APK paints, on Flutter's Linux target.
155-#
156-# This recipe and `just apk` are two targets over one tree, and the split is
157-# the one the APK already draws. Everything under `common/` — the screens, the
158-# cells, `frq.io` — is shared; what differs is who answers the host. So this
159-# is `just apk` with the Android half taken out: the same `clojure -M:cljd compile` over the same flutter/src,
160-# then Flutter's Linux target rather than its Android one. CMake and Ninja
161-# instead of Gradle, `flutter/linux/` as the runner, no SDK and no JDK.
63+# Build a target and start it.
16264 #
163-# Still impure, for one of the two reasons `apk` is: pub.dev resolution and
164-# Flutter's own engine artifacts are network. What it does NOT need is the
165-# writable-ANDROID_HOME dance — nothing here writes into the store — so there
166-# is no `flutter/.home` on this path.
65+# apk install on a connected device and launch it
66+# web serve build/web on ARG (default 8080)
67+# the rest open the window
16768 #
168-# nixGL because Flutter paints through GL, and off NixOS the driver is the
169-# host's.
170-#
171-# just flutter-desktop build the debug bundle
172-# just flutter-desktop run build it and open the window
173-flutter-desktop action="build":
69+# just run apk
70+# just run web 3000
71+[doc('build a target and start it')]
72+run target="desktop" arg="":
17473 #!/usr/bin/env bash
17574 set -euo pipefail
176- cd "{{justfile_directory()}}"
177- if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then
178- exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \
179- --command just flutter-desktop "$@"
180- fi
181- cd flutter
182-
183- # The same three caches `just apk` seeds, in the same place and out of the
184- # same flake output — one compiler, one set of dependencies, and no reason
185- # for the two frontends to keep a copy each. `.home/` is `apk`'s directory
186- # by name and this is the only thing put there from here, which is the
187- # point: whichever recipe runs first pays for the copy and the other finds
188- # it warm.
189- #
190- # No `nix build` here, unlike `apk`: this recipe is already inside the
191- # shell that names FRQ_CLJD_DEPS by the time it gets this far, and `apk`
192- # needs the path before it enters anything.
193- #
194- # m2 and gitlibs are set here for the reason they are set there — the JVM
195- # reads user.home out of /etc/passwd, so neither follows HOME and left
196- # alone they are the real ~/.m2 and ~/.gitlibs.
197- export PUB_CACHE="$PWD/.home/.pub-cache"
198- export GITLIBS="$PWD/.home/gitlibs"
199- m2="$PWD/.home/m2"
200-
201- seed() {
202- [ -e "$2" ] && return 0
203- mkdir -p "$(dirname "$2")"
204- cp -r "$FRQ_CLJD_DEPS/$1" "$2"
205- chmod -R u+w "$2"
206- }
207- seed m2 "$m2"
208- seed gitlibs "$GITLIBS"
209- seed pub-cache "$PUB_CACHE"
210- seed clojuredart/cache "$PWD/.clojuredart/cache"
211-
212- # Resolved here rather than in cljd-deps, which was not allowed to name
213- # the store — see the same loop in `apk`.
214- for helper in .clojuredart/cache/*/cljd_helper; do
215- [ -d "$helper" ] || continue
216- [ -e "$helper/.dart_tool/package_config.json" ] && continue
217- ( cd "$helper" && flutter pub get --offline )
218- done
219-
220- clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile
221- flutter build linux --debug
222-
223- # x64/arm64 is Flutter's own name for the host arch, not uname's.
224- case "$(uname -m)" in
225- x86_64) arch=x64 ;;
226- aarch64) arch=arm64 ;;
227- *) echo "unknown arch $(uname -m)" >&2; exit 1 ;;
228- esac
229- bundle="build/linux/$arch/debug/bundle"
230-
231- case "{{action}}" in
232- build) echo "built $PWD/$bundle/frq" ;;
233- run)
234- runner=()
235- [ -e /run/current-system ] || runner=("$NIXGL")
236- exec "${runner[@]}" "$bundle/frq"
237- ;;
238- *) echo "usage: just flutter-desktop [build|run]" >&2; exit 1 ;;
75+ cd "{{root}}"
76+ case "{{target}}" in
77+ apk) just _flutter apk run ;;
78+ desktop) just _flutter desktop run ;;
79+ web) port="${2:-}"; exec tools/build-web.sh serve "${port:-8080}" ;;
80+ ui) just _flutter ui run ;;
81+ app) just _flutter app run ;;
82+ log) just tools exec -- adb logcat -s flutter ;;
83+ *) echo "usage: just run [apk|desktop|web|ui|app|log]" >&2; exit 1 ;;
23984 esac
24085
241-# The third frontend: the same screens again, compiled to JavaScript.
242-#
243-# `flutter-desktop` with the Linux half taken out. One `clojure -M:cljd
244-# compile` over the same flutter/src and common/, then Flutter's web target
245-# instead of its Linux one — dart2js instead of CMake and Ninja, and a
246-# directory of static files instead of a bundle with an executable in it.
247-#
248-# And the one target with no nix in it. The other two need the host: a JDK
249-# and the Android SDK for `apk`, GTK and a C++ toolchain and nixGL for
250-# `flutter-desktop`. This one needs a Dart, a JVM and a browser, and the
251-# browser is not ours — so `tools/toolchain.sh` fetches the first two as
252-# pinned tarballs into `.toolchain/` and there is nothing left for a devShell
253-# to supply. That is what lets the container in `.modal/flutter-web/` drop
254-# its image build too: same script, same three pins, no store to populate.
255-#
256-# Impure for the reason the other two are: pub.dev resolution and Flutter's
257-# engine artifacts are network, and now the toolchain is as well — pinned by
258-# sha256, which is the reproducibility that was worth having out of the store.
259-#
260-# The entry point is `frq.main-web`, not `frq.main`: path_provider has no web
261-# implementation, so the `getApplicationSupportDirectory` that `frq.main`
262-# awaits throws MissingPluginException before any widget is built. The web
263-# entry installs `frq.io.web` — localStorage behind the same seam — and awaits
264-# nothing. `frq.net.dart` is still the socket half, so connecting will want a
265-# WebSocket before this does more than paint.
266-#
267-# One build and no `--debug` variant, because there is nothing to gain from
268-# one: dart2js at -O1 measured 52.5s against the release build's 49.8s on the
269-# same source change here, so a second, larger bundle would buy noise. See
270-# `tools/build-web.sh`, which writes the numbers down.
271-#
272-# just flutter-web build build/web
273-# just flutter-web serve build it and serve it on $PORT (8080)
274-# just flutter-web serve 3000 ...on another port
275-flutter-web action="build" port="8080":
86+# Test a suite.
87+#
88+# common what may appear in common/, read off the source. Needs no
89+# toolchain at all, which is why CI runs exactly this.
90+# nim the Nim core. No Flutter, no Dart, no SDK — a rule about the IRC
91+# wire format is checkable in a second.
92+# dart the Dart side of the FFI boundary, on the plain Dart VM. Passing
93+# `nim` and failing this one is a marshalling bug, which is why the
94+# two are separate suites.
95+# live the whole stack against a real freeq. Not in `all`: it wants a
96+# network and a running server.
97+#
98+# just test all of them
99+# just test nim tircparse one Nim file
100+[doc('run a suite: all common nim dart live')]
101+test suite="all" *args:
276102 #!/usr/bin/env bash
277103 set -euo pipefail
278- # A wrapper and nothing else. The build is a shell script because the
279- # container runs it too, and a container that had to install `just` to
280- # start would be one dependency away from the point.
281- exec "{{justfile_directory()}}/tools/build-web.sh" {{action}} {{port}}
104+ cd "{{root}}"
105+ shift
106+ case "{{suite}}" in
107+ all) just test common && just test nim && just test dart ;;
108+ common) exec python3 tools/check-common.py common ;;
109+ nim) just _nim-test "$@" ;;
110+ dart) just _nim-lib
111+ exec "{{tc}}" exec -- bash -c \
112+ 'cd dart/frq_core && dart pub get && dart test -r expanded' ;;
113+ live) just _nim-lib
114+ exec "{{tc}}" exec -- bash -c \
115+ 'cd dart/frq_core && dart pub get >/dev/null && dart run tool/live_ui.dart "$@"' _ "$@" ;;
116+ *) echo "usage: just test [all|common|nim|dart|live]" >&2; exit 1 ;;
117+ esac
282118
283-# The containers in `.modal/`, run on Modal rather than here. This machine
284-# evaluates and Modal builds — see CLAUDE.md, which says so rather more
285-# firmly — and these two recipes are the whole interface to that.
119+# The containers in `.modal/`, run on Modal rather than here: this machine
120+# evaluates and Modal builds. See CLAUDE.md, which says so rather more firmly.
286121 #
287-# Named for where the work happens, the way `flutter-desktop` is named for
288-# what paints: there is no re-entry test here because nothing
289-# re-enters. There is no `nix` variable either, and that used to be because
290-# nix ran out there — now it is because neither of these containers has any
291-# nix in it at all.
122+# `--shell` leaves a sandbox running with the container's own image, volumes
123+# and environment, and prints the command to attach to it. It blocks — an
124+# ephemeral app takes its sandbox down when the entrypoint returns — so attach
125+# from a second terminal and Ctrl-C here when done. The sandbox bills until
126+# you do.
292127 #
293-# just modal flutter-dev the incremental Flutter loop
294-# just modal flutter-web the web bundle
295-modal container="flutter-dev" *args:
128+# just modal dev
129+# just modal web --shell
130+[doc('run a .modal/ container on Modal')]
131+modal container="dev" *args:
296132 #!/usr/bin/env bash
297133 set -euo pipefail
298- cd "{{justfile_directory()}}"
299- shift
134+ cd "{{root}}"
135+ shift || true
300136 exec modal run ".modal/{{container}}/container.py" "$@"
301137
302138 # The Modal-built web bundle, served from this machine on localhost.
303139 #
304-# Not a local build: `modal volume get` pulls what `just modal flutter-web`
305-# already compiled out of the devshell volume, so this needs no Flutter, no
306-# Dart and no nix — only python, which the flake shell has and so does the
307-# machine.
308-#
309-# It exists for one reason, and the reason is the auth broker rather than
310-# convenience. freeq's broker finishes an OAuth login by redirecting the
311-# browser to `return_to`, and it will only redirect to an origin on its
312-# allowlist: its own https hosts, and `http://localhost` or `http://127.0.0.1`
313-# on ANY port. A build served from anywhere else — the Modal URL included —
314-# gets `400 Invalid return_to URL` and can never complete a Bluesky sign-in,
315-# no matter what the client does. localhost is the one allowlisted origin we
316-# can serve from, so this is how Bluesky sign-in is tested.
317-#
318-# Guest and app-password sign-in need none of this; they work on the deployed
319-# URL, because neither goes near the broker.
320-#
321-# just web-local fetch and serve on :8080
322-# just web-local 3000 another port; any port is allowlisted
323-web-local port="8080":
140+# Not a local build: `modal volume get` pulls down what `just modal
141+# web` already compiled, so this needs only python3.
142+#
143+# It exists for the auth broker rather than for convenience. freeq's broker
144+# finishes an OAuth login by redirecting to `return_to`, and only to an origin
145+# on its allowlist: its own https hosts, and http://localhost or
146+# http://127.0.0.1 on ANY port. Served from anywhere else — the Modal URL
147+# included — a Bluesky sign-in gets `400 Invalid return_to URL` and can never
148+# complete. Guest and app-password sign-in work on the deployed URL; neither
149+# goes near the broker.
150+[doc('serve the Modal-built web bundle on localhost')]
151+serve port="8080":
324152 #!/usr/bin/env bash
325153 set -euo pipefail
326- cd "{{justfile_directory()}}"
327- out="{{justfile_directory()}}/.web-local"
154+ cd "{{root}}"
155+ out="{{root}}/.web-local"
328156 mkdir -p "$out"
329157 echo "fetching the Modal-built bundle…"
330158 # --force: this is a mirror of the volume, and a stale file left behind
331159 # would be served in preference to the one just built.
332- modal volume get --force devshell frq-flutter-web/flutter/build/web "$out"
160+ modal volume get --force devshell frq-web/flutter/build/web "$out"
333161 echo
334162 echo " http://localhost:{{port}}"
335163 echo
336- echo "Bluesky sign-in works here and not on the Modal URL: the broker"
337- echo "allowlists localhost on any port. Put wss://irc.freeq.at/irc in the"
338- echo "Server field — a browser has no TCP."
164+ echo "Bluesky sign-in works here and not on the Modal URL. Put"
165+ echo "wss://irc.freeq.at/irc in the Server field — a browser has no TCP."
339166 exec python3 -m http.server {{port}} --bind 127.0.0.1 --directory "$out/web"
340167
341-# A sandbox left running with the container's own image, volumes and
342-# environment, and the command to get into it. `modal shell --image` cannot
343-# be pointed at a published Modal image like arch-nix, so attaching to a
344-# running sandbox is the only way to get a shell that is the container.
345-#
346-# It blocks: an ephemeral app stops when its entrypoint returns and takes the
347-# sandbox with it. Attach from a second terminal, and Ctrl-C here when done —
348-# the sandbox bills until you do.
349-#
350-# just modal-shell flutter-dev, the usual one
351-# just modal-shell flutter-web the web bundle container
352-modal-shell container="flutter-dev":
353- #!/usr/bin/env bash
354- set -euo pipefail
355- cd "{{justfile_directory()}}"
356- exec modal run ".modal/{{container}}/container.py" --shell
357-
358-# The Nim core's test suite.
359-#
360-# It needs no Flutter, no Dart and no Android SDK — which is the point of
361-# having the logic here rather than under `common/`: a rule about the IRC wire
362-# format can be checked in a second, on any machine, without a toolchain that
363-# takes minutes to enter.
364-#
365-# just nim-test the whole suite
366-# just nim-test tircparse one file
367-nim-test file="":
368- #!/usr/bin/env bash
369- set -euo pipefail
370- cd "{{justfile_directory()}}"
371- if [ -z "${FRQ_NIM:-}" ]; then
372- exec {{nix}} develop .#nim --max-jobs {{jobs}} --command just nim-test "$@"
373- fi
374- cd nim
375- if [ -n "{{file}}" ]; then
376- exec nim c -r --hints:off --path:src "tests/{{file}}.nim"
377- fi
378- for t in tests/t*.nim; do
379- echo "== $t"
380- nim c -r --hints:off --path:src "$t"
381- done
168+# --- the work behind the verbs ------------------------------------------
382169
383170 # The Nim core as a shared library, into build/nim.
384171 #
@@ -387,117 +174,88 @@ nim-test file="":
387174 # from another heap and dereferencing it segfaults. ORC's heap is shared.
388175 #
389176 # `-d:release` and not `-d:danger`: the bounds checks are what turn a
390-# malformed line off a socket into an exception instead of a read past the end
391-# of a buffer, and this parses exactly that.
392-nim-lib:
177+# malformed line off a socket into an exception rather than a read past the
178+# end of a buffer, and this parses exactly that.
179+[private]
180+_nim-lib:
393181 #!/usr/bin/env bash
394182 set -euo pipefail
395- cd "{{justfile_directory()}}"
396- if [ -z "${FRQ_NIM:-}" ]; then
397- exec {{nix}} develop .#nim --max-jobs {{jobs}} --command just nim-lib
398- fi
399- out="{{justfile_directory()}}/build/nim"
183+ cd "{{root}}"
184+ out="{{root}}/build/nim"
400185 mkdir -p "$out"
401- cd nim
402- nim c --app:lib --mm:orc -d:release --hints:off \
403- --path:src --out:"$out/libfrqcore.so" src/frq_core.nim
404- echo "built $out/libfrqcore.so"
405- nm -D --defined-only "$out/libfrqcore.so" | grep ' T frq_' || true
406-
407-# The Dart side of the Nim boundary, on the plain Dart VM.
408-#
409-# No Flutter, no emulator, no ClojureDart — `dart/frq_core` is ordinary Dart
410-# over `dart:ffi` and is not a Flutter package, so the test that proves the
411-# marshalling runs in a second. Passing `just nim-test` and failing this one is a
412-# marshalling bug, which is the whole reason the two suites are separate.
413-#
414-# Builds the library first: the test dlopens a real .so and there is no point
415-# reporting that it could not find one.
416-dart-test:
186+ exec "{{tc}}" exec -- bash -euo pipefail -c '
187+ cd nim
188+ nim c --app:lib --mm:orc -d:release --hints:off \
189+ --path:src --out:"'"$out"'/libfrqcore.so" src/frq_core.nim
190+ echo "built '"$out"'/libfrqcore.so"
191+ nm -D --defined-only "'"$out"'/libfrqcore.so" | grep " T frq_" || true'
192+
193+[private]
194+_nim-test *args:
417195 #!/usr/bin/env bash
418196 set -euo pipefail
419- cd "{{justfile_directory()}}"
420- if [ -z "${FRQ_DART:-}" ]; then
421- # nim-lib before the re-entry, not after: it enters a shell of its own
422- # and doing it on the far side would build the library twice.
423- just nim-lib
424- exec {{nix}} develop .#dart --max-jobs {{jobs}} --command just dart-test
425- fi
426- cd dart/frq_core
427- dart pub get
428- dart test -r expanded
429-
430-# The whole Nim stack against a real freeq: socket, state and screens.
431-#
432-# Connects, waits for the room list, opens a room and prints what the tree
433-# actually contains — through the FFI, so it is the path the window uses.
434-# Not in any suite: it needs a network and a running freeq.
435-nim-live *args:
436- #!/usr/bin/env bash
437- set -euo pipefail
438- cd "{{justfile_directory()}}"
439- if [ -z "${FRQ_DART:-}" ]; then
440- just nim-lib
441- exec {{nix}} develop .#dart --max-jobs {{jobs}} --command just nim-live "$@"
442- fi
443- shift || true
444- cd dart/frq_core
445- dart pub get >/dev/null
446- exec dart run tool/live_ui.dart "$@"
447-
448-# frq with Nim owning the state and the screens, rendered by Flutter.
449-#
450-# No ClojureDart on this path. `lib/main_nim.dart` asks the Nim core for a
451-# widget tree and paints it; the screens are ports of `common/frq/screens/`.
452-#
453-# just nim-ui build it
454-# just nim-ui run open the window
455-nim-ui action="build":
456- #!/usr/bin/env bash
457- set -euo pipefail
458- cd "{{justfile_directory()}}"
459- if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then
460- just nim-lib
461- exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \
462- --command just nim-ui "$@"
463- fi
464- cd flutter
465- flutter pub get
466- export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
467- runner=()
468- [ -e /run/current-system ] || runner=("$NIXGL")
469- case "{{action}}" in
470- build) exec "${runner[@]}" flutter build linux --debug -t lib/main_nim.dart ;;
471- run) exec "${runner[@]}" flutter run -d linux -t lib/main_nim.dart ;;
472- *) echo "usage: just nim-ui [build|run]" >&2; exit 1 ;;
473- esac
474-
475-# The ClojureDart app, with the Nim core as its transport only.
476-#
477-# This is the wiring that matters: `frq.main-nim` is `frq.main` with one line
478-# changed — `frq.net.nim/install!` where it says `frq.net.dart/install!`. Every
479-# screen, every cell and every action is the one that was already there. Nim
480-# owns the socket, the TLS and the line framing, and nothing else.
481-#
482-# just nim-app build it
483-# just nim-app run open the window
484-nim-app action="build":
197+ cd "{{root}}"
198+ exec "{{tc}}" exec -- bash -euo pipefail -c '
199+ cd nim
200+ if [ -n "${1:-}" ]; then
201+ exec nim c -r --hints:off --path:src "tests/$1.nim"
202+ fi
203+ for t in tests/t*.nim; do
204+ echo "== $t"
205+ nim c -r --hints:off --path:src "$t"
206+ done' _ "$@"
207+
208+# The three Flutter targets, which differ only in what they compile and which
209+# entry point they paint.
210+#
211+# apk ClojureDart, then Gradle. Impure on purpose: Gradle resolves its
212+# own dependencies over the network and has sdkmanager install a
213+# platform and build-tools into ANDROID_HOME as it goes, so the
214+# SDK has to be writable — which is what `just tools android` gets
215+# it. Everything it leaves behind is under `.toolchain/` and
216+# gitignored.
217+# desktop the same `clojure -M:cljd compile`, Flutter's Linux target.
218+# ui no ClojureDart at all: `lib/main_nim.dart` asks the Nim core for
219+# a widget tree and paints it.
220+# app `frq.main-nim` is `frq.main` with one line changed —
221+# `frq.net.nim/install!` where it said `frq.net.dart/install!`.
222+# Every screen, cell and action is the one that was already there.
223+[private]
224+_flutter target action:
485225 #!/usr/bin/env bash
486226 set -euo pipefail
487- cd "{{justfile_directory()}}"
488- if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then
489- just nim-lib
490- exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \
491- --command just nim-app "$@"
492- fi
493- cd flutter
494- flutter pub get
495- export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
496- clojure -M:cljd compile frq.main-nim
497- runner=()
498- [ -e /run/current-system ] || runner=("$NIXGL")
499- case "{{action}}" in
500- build) exec "${runner[@]}" flutter build linux --debug -t lib/main_nim_app.dart ;;
501- run) exec "${runner[@]}" flutter run -d linux -t lib/main_nim_app.dart ;;
502- *) echo "usage: just nim-app [build|run]" >&2; exit 1 ;;
227+ cd "{{root}}"
228+ case "{{target}}" in
229+ apk) "{{tc}}" android ;;
230+ ui|app) just _nim-lib ;;
503231 esac
232+ exec "{{tc}}" exec -- bash -euo pipefail -c '
233+ cd flutter
234+ # Nim resolves OpenSSL through dynlib at run time; without the host
235+ # library on the loader path `newContext` dies in a SIGSEGV that says
236+ # nothing about SSL.
237+ export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
238+ cljd() { clojure -Sdeps "{:mvn/local-repo \"$FRQ_M2\"}" -M:cljd compile "$@"; }
239+
240+ case "$1:$2" in
241+ apk:*)
242+ cljd
243+ # Rewritten every run: it carries absolute paths.
244+ flutter config --android-sdk "$ANDROID_HOME" >/dev/null
245+ flutter build apk --debug
246+ apk=build/app/outputs/flutter-apk/app-debug.apk
247+ [ "$2" = run ] || exit 0
248+ adb install -r "$apk"
249+ exec adb shell monkey -p uk.nandi.frq \
250+ -c android.intent.category.LAUNCHER 1 ;;
251+ desktop:build) cljd; exec flutter build linux --debug ;;
252+ desktop:run) cljd; exec flutter run -d linux ;;
253+ ui:build) flutter pub get
254+ exec flutter build linux --debug -t lib/main_nim.dart ;;
255+ ui:run) flutter pub get
256+ exec flutter run -d linux -t lib/main_nim.dart ;;
257+ app:build) flutter pub get; cljd frq.main-nim
258+ exec flutter build linux --debug -t lib/main_nim_app.dart ;;
259+ app:run) flutter pub get; cljd frq.main-nim
260+ exec flutter run -d linux -t lib/main_nim_app.dart ;;
261+ esac' _ "{{target}}" "{{action}}"
@@ -1,384 +1,171 @@
1-# The work is here now, in recipe bodies, where it used to be in scripts/ —1+# Six verbs over one tree.
2-# babashka scripts run through a scripts/bb that went looking for a babashka.
3-# That indirection bought one thing worth having, a shared way to reach nix on
4-# a host that keeps it in a container, and `nix` below is the whole of it.
5 #2 #
6-# Every recipe that builds has the same shape: outside the dev shell, re-enter3+# There is no nix here any more. `tools/toolchain.sh` fetches Flutter (which
7-# it and come back to this same recipe; inside, do the work. The re-entry test4+# carries Dart), a JDK, the Clojure CLI and Nim as sha256-pinned tarballs into
8-# is an environment variable only the shell sets — no flag to forget, and no5+# `.toolchain/`, and every recipe below runs inside the environment that
9-# second code path for someone who runs `nix develop --command just ...` by6+# script prints. One mechanism, and the same one the containers in `.modal/`
10-# hand.7+# use — which is what lets a plain Debian image build this.
8+#
9+# What the host still brings: a C compiler (Nim shells out to one), OpenSSL
10+# (nim.cfg is `-d:ssl`), git, curl, unzip, python3. For `run desktop` and the
11+# other Linux builds, GTK and the usual CMake/Ninja/pkg-config.
12+#
13+# just build TARGET apk desktop web lib ui app
14+# just run TARGET apk desktop web ui app
15+# just test SUITE all nim dart common live
16+# just modal CONTAINER dev web
17+# just serve [PORT] the Modal-built web bundle, on localhost
18+# just tools ... the toolchain itself
11 19
12 set shell := ["bash", "-euo", "pipefail", "-c"]20 set shell := ["bash", "-euo", "pipefail", "-c"]
13 21
14-# Every recipe below is a `#!` script and passes its arguments on with "$@".22+# Every recipe is a `#!` script. Without this, "$@" is empty in one of them —
15-# Without this that is empty in one — just interpolates into a shebang recipe23+# just interpolates into a shebang recipe rather than handing it argv.
16-# rather than handing it argv — and a recipe silently ignored its flags.
17 set positional-arguments24 set positional-arguments
18 25
19-# nix is not on every host this runs on: on the machine these recipes were26+root := justfile_directory()
20-# written for it lives in an Arch distrobox, at the same path — which is why27+tc := justfile_directory() / "tools/toolchain.sh"
21-# the container is entered rather than the tree copied into it. See CLAUDE.md.
22-nix := `command -v nix >/dev/null 2>&1 && echo nix || echo "distrobox enter arch -- nix"`
23-
24-# --max-jobs 0 is what sends the work to the `builders` entry rather than
25-# compiling it here. Left to the default, nix prefers the local machine, and a
26-# cold Flutter toolchain is a lot of compiling on a laptop — for a derivation a
27-# remote builder has likely built already. FRQ_MAX_JOBS=auto is the way out on
28-# a machine with no builder configured.
29-jobs := env("FRQ_MAX_JOBS", "0")
30 28
31 default:29 default:
32 @just --list30 @just --list
33 31
34-# The APK: ClojureDart compiled to Dart, then Flutter's Gradle build.32+# The toolchain, directly: `just tools versions`, `just tools android`,
35-#33+# `just tools exec -- flutter doctor`.
36-# Impure on purpose, and worth saying why rather than leaving it to be34+[doc('the toolchain itself: versions, android, exec -- CMD')]
37-# discovered. Gradle resolves its own dependencies over the network and35+tools *args:
38-# installs build-tools and a platform into ANDROID_HOME as it goes, so it36+ #!/usr/bin/env bash
39-# cannot run in a sandbox and cannot write to the store. What nix gives here37+ cd "{{root}}"
40-# is the toolchain — clojure, a JDK, Flutter, and an SDK composed by38+ exec "{{tc}}" "$@"
41-# androidenv — and the recipe copies that SDK somewhere writable39+
42-# (flutter/.home) for Gradle to finish off. That copy and everything Gradle40+# Build a target.
43-# leaves behind are gitignored.41+#
44-#42+# apk the Android app, via Gradle
45-# No ndkVersion in android/app/build.gradle.kts, for the same reason: the43+# desktop the ClojureDart app on Flutter's Linux target
46-# Flutter template sets it, setting it makes Gradle fetch that exact NDK, and44+# web the same screens compiled to JavaScript, into build/web
47-# there is no native code here to need one.45+# lib the Nim core as build/nim/libfrqcore.so
48-#46+# ui Nim owning the state and the screens, painted by Flutter
49-# just apk build the debug APK47+# app the ClojureDart app with the Nim core as its transport only
50-# just apk install build it and put it on a connected device48+[doc('build a target: apk desktop web lib ui app')]
51-# just apk run install and launch49+build target="desktop":
52-# just apk log logcat, filtered to this app
53-apk action="build":
54 #!/usr/bin/env bash50 #!/usr/bin/env bash
55 set -euo pipefail51 set -euo pipefail
56- cd "{{justfile_directory()}}/flutter"52+ cd "{{root}}"
57-53+ case "{{target}}" in
58- # The flake's, not an --impure --expr against whatever nixos-unstable is54+ apk) just _flutter apk build ;;
59- # today: the licence config the SDK needs lives in `androidPkgsFor` now,55+ desktop) just _flutter desktop build ;;
60- # so this is an ordinary output at the rev flake.lock pins.56+ web) exec tools/build-web.sh build ;;
61- sdk="$(nix build --no-link --print-out-paths \57+ lib) just _nim-lib ;;
62- "{{justfile_directory()}}#android-sdk")/libexec/android-sdk"58+ ui) just _flutter ui build ;;
63-59+ app) just _flutter app build ;;
64- # adb keeps the key the phone has already trusted under the real HOME, and60+ *) echo "usage: just build [apk|desktop|web|lib|ui|app]" >&2; exit 1 ;;
65- # HOME moves below so Gradle can write into the SDK copy. Told where to
66- # look, adb keeps its identity; left to find $HOME/.android it generates a
67- # new one, the device stops recognising this machine, and the deploy ends
68- # in "no devices/emulators found" while `adb devices` in any other shell
69- # lists it perfectly well.
70- export ANDROID_USER_HOME="${ANDROID_USER_HOME:-$HOME/.android}"
71-
72- export HOME="$PWD/.home"
73- export ANDROID_HOME="$HOME/android-sdk"
74- export ANDROID_SDK_ROOT="$ANDROID_HOME"
75- mkdir -p "$HOME"
76-
77- # Gradle writes into ANDROID_HOME, so it is a copy rather than the store
78- # path. Made once and kept: re-copying would throw away the build-tools
79- # and platform Gradle installed into it on the last run.
80- if [ ! -d "$ANDROID_HOME" ]; then
81- cp -r "$sdk" "$ANDROID_HOME"
82- chmod -R u+w "$ANDROID_HOME"
83- fi
84-
85- # The compile's three caches, same flake-output-and-copy shape as the SDK
86- # and for the same reason: tools.deps and pub both write into theirs, and
87- # the store is read-only. What this buys is the "Resolving dependencies…
88- # Downloading packages…" that used to open every run.
89- deps="$(nix build --no-link --print-out-paths \
90- "{{justfile_directory()}}#cljd-deps")"
91-
92- # Neither of these follows HOME. Both are read off the JVM's user.home,
93- # which comes from /etc/passwd rather than the environment — so moving
94- # HOME below is not enough to move them, and left alone they would be the
95- # real ~/.m2 and ~/.gitlibs, shared with every other project on the box.
96- export GITLIBS="$HOME/gitlibs"
97- m2="$HOME/m2"
98- export PUB_CACHE="$HOME/.pub-cache"
99-
100- # Seeded once each and then left alone, exactly as ANDROID_HOME is: after
101- # the first compile these hold whatever the working tree has asked for
102- # since, and re-copying would throw that away.
103- seed() {
104- [ -e "$2" ] && return 0
105- mkdir -p "$(dirname "$2")"
106- cp -r "$deps/$1" "$2"
107- chmod -R u+w "$2"
108- }
109- seed m2 "$m2"
110- seed gitlibs "$GITLIBS"
111- seed pub-cache "$PUB_CACHE"
112- seed clojuredart/cache "$PWD/.clojuredart/cache"
113-
114- # Also the flake's. `nix shell nixpkgs#...` read the registry, which is a
115- # different and unlocked nixpkgs — the Flutter that built the APK could
116- # move under it without flake.lock changing a line.
117- flutter="nix develop {{justfile_directory()}}#flutter --command"
118-
119- # cljd-deps ships the analyzer project unresolved — a fixed-output
120- # derivation may not name the store, and a resolved pub project is
121- # nothing but store paths. So it is resolved here instead, offline,
122- # against the cache that derivation did fetch. ClojureDart reaches for the
123- # network only when bin/analyzer.dart is missing, and after this it is not.
124- for helper in .clojuredart/cache/*/cljd_helper; do
125- [ -d "$helper" ] || continue
126- [ -e "$helper/.dart_tool/package_config.json" ] && continue
127- ( cd "$helper" && $flutter flutter pub get --offline )
128- done
129-
130- $flutter clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile
131-
132- # Rewritten every run: it carries absolute store paths, and the flutter
133- # one moves whenever nixpkgs does.
134- $flutter flutter config --android-sdk "$ANDROID_HOME" >/dev/null
135-
136- apk=build/app/outputs/flutter-apk/app-debug.apk
137- adb="${ADB:-$ANDROID_HOME/platform-tools/adb}"
138-
139- case "{{action}}" in
140- build) $flutter flutter build apk --debug ;;
141- install) $flutter flutter build apk --debug && "$adb" install -r "$apk" ;;
142- run) $flutter flutter build apk --debug && "$adb" install -r "$apk" \
143- && "$adb" shell monkey -p uk.nandi.frq -c android.intent.category.LAUNCHER 1 ;;
144- log) "$adb" logcat -s flutter ;;
145- *) echo "usage: just apk [build|install|run|log]" >&2; exit 1 ;;
146 esac61 esac
147 62
148-# What may appear in common/, checked. Needs nothing built: it reads the63+# Build a target and start it.
149-# source, so it is the one check that runs anywhere, and CI runs exactly this.
150-check-common:
151- #!/usr/bin/env bash
152- python3 tools/check-common.py common
153-
154-# The desktop GUI: the same screens the APK paints, on Flutter's Linux target.
155-#
156-# This recipe and `just apk` are two targets over one tree, and the split is
157-# the one the APK already draws. Everything under `common/` — the screens, the
158-# cells, `frq.io` — is shared; what differs is who answers the host. So this
159-# is `just apk` with the Android half taken out: the same `clojure -M:cljd compile` over the same flutter/src,
160-# then Flutter's Linux target rather than its Android one. CMake and Ninja
161-# instead of Gradle, `flutter/linux/` as the runner, no SDK and no JDK.
162 #64 #
163-# Still impure, for one of the two reasons `apk` is: pub.dev resolution and65+# apk install on a connected device and launch it
164-# Flutter's own engine artifacts are network. What it does NOT need is the66+# web serve build/web on ARG (default 8080)
165-# writable-ANDROID_HOME dance — nothing here writes into the store — so there67+# the rest open the window
166-# is no `flutter/.home` on this path.
167 #68 #
168-# nixGL because Flutter paints through GL, and off NixOS the driver is the69+# just run apk
169-# host's.70+# just run web 3000
170-#71+[doc('build a target and start it')]
171-# just flutter-desktop build the debug bundle72+run target="desktop" arg="":
172-# just flutter-desktop run build it and open the window
173-flutter-desktop action="build":
174 #!/usr/bin/env bash73 #!/usr/bin/env bash
175 set -euo pipefail74 set -euo pipefail
176- cd "{{justfile_directory()}}"75+ cd "{{root}}"
177- if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then76+ case "{{target}}" in
178- exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \77+ apk) just _flutter apk run ;;
179- --command just flutter-desktop "$@"78+ desktop) just _flutter desktop run ;;
180- fi79+ web) port="${2:-}"; exec tools/build-web.sh serve "${port:-8080}" ;;
181- cd flutter80+ ui) just _flutter ui run ;;
182-81+ app) just _flutter app run ;;
183- # The same three caches `just apk` seeds, in the same place and out of the82+ log) just tools exec -- adb logcat -s flutter ;;
184- # same flake output — one compiler, one set of dependencies, and no reason83+ *) echo "usage: just run [apk|desktop|web|ui|app|log]" >&2; exit 1 ;;
185- # for the two frontends to keep a copy each. `.home/` is `apk`'s directory
186- # by name and this is the only thing put there from here, which is the
187- # point: whichever recipe runs first pays for the copy and the other finds
188- # it warm.
189- #
190- # No `nix build` here, unlike `apk`: this recipe is already inside the
191- # shell that names FRQ_CLJD_DEPS by the time it gets this far, and `apk`
192- # needs the path before it enters anything.
193- #
194- # m2 and gitlibs are set here for the reason they are set there — the JVM
195- # reads user.home out of /etc/passwd, so neither follows HOME and left
196- # alone they are the real ~/.m2 and ~/.gitlibs.
197- export PUB_CACHE="$PWD/.home/.pub-cache"
198- export GITLIBS="$PWD/.home/gitlibs"
199- m2="$PWD/.home/m2"
200-
201- seed() {
202- [ -e "$2" ] && return 0
203- mkdir -p "$(dirname "$2")"
204- cp -r "$FRQ_CLJD_DEPS/$1" "$2"
205- chmod -R u+w "$2"
206- }
207- seed m2 "$m2"
208- seed gitlibs "$GITLIBS"
209- seed pub-cache "$PUB_CACHE"
210- seed clojuredart/cache "$PWD/.clojuredart/cache"
211-
212- # Resolved here rather than in cljd-deps, which was not allowed to name
213- # the store — see the same loop in `apk`.
214- for helper in .clojuredart/cache/*/cljd_helper; do
215- [ -d "$helper" ] || continue
216- [ -e "$helper/.dart_tool/package_config.json" ] && continue
217- ( cd "$helper" && flutter pub get --offline )
218- done
219-
220- clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile
221- flutter build linux --debug
222-
223- # x64/arm64 is Flutter's own name for the host arch, not uname's.
224- case "$(uname -m)" in
225- x86_64) arch=x64 ;;
226- aarch64) arch=arm64 ;;
227- *) echo "unknown arch $(uname -m)" >&2; exit 1 ;;
228- esac
229- bundle="build/linux/$arch/debug/bundle"
230-
231- case "{{action}}" in
232- build) echo "built $PWD/$bundle/frq" ;;
233- run)
234- runner=()
235- [ -e /run/current-system ] || runner=("$NIXGL")
236- exec "${runner[@]}" "$bundle/frq"
237- ;;
238- *) echo "usage: just flutter-desktop [build|run]" >&2; exit 1 ;;
239 esac84 esac
240 85
241-# The third frontend: the same screens again, compiled to JavaScript.86+# Test a suite.
242-#87+#
243-# `flutter-desktop` with the Linux half taken out. One `clojure -M:cljd88+# common what may appear in common/, read off the source. Needs no
244-# compile` over the same flutter/src and common/, then Flutter's web target89+# toolchain at all, which is why CI runs exactly this.
245-# instead of its Linux one — dart2js instead of CMake and Ninja, and a90+# nim the Nim core. No Flutter, no Dart, no SDK — a rule about the IRC
246-# directory of static files instead of a bundle with an executable in it.91+# wire format is checkable in a second.
247-#92+# dart the Dart side of the FFI boundary, on the plain Dart VM. Passing
248-# And the one target with no nix in it. The other two need the host: a JDK93+# `nim` and failing this one is a marshalling bug, which is why the
249-# and the Android SDK for `apk`, GTK and a C++ toolchain and nixGL for94+# two are separate suites.
250-# `flutter-desktop`. This one needs a Dart, a JVM and a browser, and the95+# live the whole stack against a real freeq. Not in `all`: it wants a
251-# browser is not ours — so `tools/toolchain.sh` fetches the first two as96+# network and a running server.
252-# pinned tarballs into `.toolchain/` and there is nothing left for a devShell97+#
253-# to supply. That is what lets the container in `.modal/flutter-web/` drop98+# just test all of them
254-# its image build too: same script, same three pins, no store to populate.99+# just test nim tircparse one Nim file
255-#100+[doc('run a suite: all common nim dart live')]
256-# Impure for the reason the other two are: pub.dev resolution and Flutter's101+test suite="all" *args:
257-# engine artifacts are network, and now the toolchain is as well — pinned by
258-# sha256, which is the reproducibility that was worth having out of the store.
259-#
260-# The entry point is `frq.main-web`, not `frq.main`: path_provider has no web
261-# implementation, so the `getApplicationSupportDirectory` that `frq.main`
262-# awaits throws MissingPluginException before any widget is built. The web
263-# entry installs `frq.io.web` — localStorage behind the same seam — and awaits
264-# nothing. `frq.net.dart` is still the socket half, so connecting will want a
265-# WebSocket before this does more than paint.
266-#
267-# One build and no `--debug` variant, because there is nothing to gain from
268-# one: dart2js at -O1 measured 52.5s against the release build's 49.8s on the
269-# same source change here, so a second, larger bundle would buy noise. See
270-# `tools/build-web.sh`, which writes the numbers down.
271-#
272-# just flutter-web build build/web
273-# just flutter-web serve build it and serve it on $PORT (8080)
274-# just flutter-web serve 3000 ...on another port
275-flutter-web action="build" port="8080":
276 #!/usr/bin/env bash102 #!/usr/bin/env bash
277 set -euo pipefail103 set -euo pipefail
278- # A wrapper and nothing else. The build is a shell script because the104+ cd "{{root}}"
279- # container runs it too, and a container that had to install `just` to105+ shift
280- # start would be one dependency away from the point.106+ case "{{suite}}" in
281- exec "{{justfile_directory()}}/tools/build-web.sh" {{action}} {{port}}107+ all) just test common && just test nim && just test dart ;;
108+ common) exec python3 tools/check-common.py common ;;
109+ nim) just _nim-test "$@" ;;
110+ dart) just _nim-lib
111+ exec "{{tc}}" exec -- bash -c \
112+ 'cd dart/frq_core && dart pub get && dart test -r expanded' ;;
113+ live) just _nim-lib
114+ exec "{{tc}}" exec -- bash -c \
115+ 'cd dart/frq_core && dart pub get >/dev/null && dart run tool/live_ui.dart "$@"' _ "$@" ;;
116+ *) echo "usage: just test [all|common|nim|dart|live]" >&2; exit 1 ;;
117+ esac
282 118
283-# The containers in `.modal/`, run on Modal rather than here. This machine119+# The containers in `.modal/`, run on Modal rather than here: this machine
284-# evaluates and Modal builds — see CLAUDE.md, which says so rather more120+# evaluates and Modal builds. See CLAUDE.md, which says so rather more firmly.
285-# firmly — and these two recipes are the whole interface to that.
286 #121 #
287-# Named for where the work happens, the way `flutter-desktop` is named for122+# `--shell` leaves a sandbox running with the container's own image, volumes
288-# what paints: there is no re-entry test here because nothing123+# and environment, and prints the command to attach to it. It blocks — an
289-# re-enters. There is no `nix` variable either, and that used to be because124+# ephemeral app takes its sandbox down when the entrypoint returns — so attach
290-# nix ran out there — now it is because neither of these containers has any125+# from a second terminal and Ctrl-C here when done. The sandbox bills until
291-# nix in it at all.126+# you do.
292 #127 #
293-# just modal flutter-dev the incremental Flutter loop128+# just modal dev
294-# just modal flutter-web the web bundle129+# just modal web --shell
295-modal container="flutter-dev" *args:130+[doc('run a .modal/ container on Modal')]
131+modal container="dev" *args:
296 #!/usr/bin/env bash132 #!/usr/bin/env bash
297 set -euo pipefail133 set -euo pipefail
298- cd "{{justfile_directory()}}"134+ cd "{{root}}"
299- shift135+ shift || true
300 exec modal run ".modal/{{container}}/container.py" "$@"136 exec modal run ".modal/{{container}}/container.py" "$@"
301 137
302 # The Modal-built web bundle, served from this machine on localhost.138 # The Modal-built web bundle, served from this machine on localhost.
303 #139 #
304-# Not a local build: `modal volume get` pulls what `just modal flutter-web`140+# Not a local build: `modal volume get` pulls down what `just modal
305-# already compiled out of the devshell volume, so this needs no Flutter, no141+# web` already compiled, so this needs only python3.
306-# Dart and no nix — only python, which the flake shell has and so does the142+#
307-# machine.143+# It exists for the auth broker rather than for convenience. freeq's broker
308-#144+# finishes an OAuth login by redirecting to `return_to`, and only to an origin
309-# It exists for one reason, and the reason is the auth broker rather than145+# on its allowlist: its own https hosts, and http://localhost or
310-# convenience. freeq's broker finishes an OAuth login by redirecting the146+# http://127.0.0.1 on ANY port. Served from anywhere else — the Modal URL
311-# browser to `return_to`, and it will only redirect to an origin on its147+# included — a Bluesky sign-in gets `400 Invalid return_to URL` and can never
312-# allowlist: its own https hosts, and `http://localhost` or `http://127.0.0.1`148+# complete. Guest and app-password sign-in work on the deployed URL; neither
313-# on ANY port. A build served from anywhere else — the Modal URL included —149+# goes near the broker.
314-# gets `400 Invalid return_to URL` and can never complete a Bluesky sign-in,150+[doc('serve the Modal-built web bundle on localhost')]
315-# no matter what the client does. localhost is the one allowlisted origin we151+serve port="8080":
316-# can serve from, so this is how Bluesky sign-in is tested.
317-#
318-# Guest and app-password sign-in need none of this; they work on the deployed
319-# URL, because neither goes near the broker.
320-#
321-# just web-local fetch and serve on :8080
322-# just web-local 3000 another port; any port is allowlisted
323-web-local port="8080":
324 #!/usr/bin/env bash152 #!/usr/bin/env bash
325 set -euo pipefail153 set -euo pipefail
326- cd "{{justfile_directory()}}"154+ cd "{{root}}"
327- out="{{justfile_directory()}}/.web-local"155+ out="{{root}}/.web-local"
328 mkdir -p "$out"156 mkdir -p "$out"
329 echo "fetching the Modal-built bundle…"157 echo "fetching the Modal-built bundle…"
330 # --force: this is a mirror of the volume, and a stale file left behind158 # --force: this is a mirror of the volume, and a stale file left behind
331 # would be served in preference to the one just built.159 # would be served in preference to the one just built.
332- modal volume get --force devshell frq-flutter-web/flutter/build/web "$out"160+ modal volume get --force devshell frq-web/flutter/build/web "$out"
333 echo161 echo
334 echo " http://localhost:{{port}}"162 echo " http://localhost:{{port}}"
335 echo163 echo
336- echo "Bluesky sign-in works here and not on the Modal URL: the broker"164+ echo "Bluesky sign-in works here and not on the Modal URL. Put"
337- echo "allowlists localhost on any port. Put wss://irc.freeq.at/irc in the"165+ echo "wss://irc.freeq.at/irc in the Server field — a browser has no TCP."
338- echo "Server field — a browser has no TCP."
339 exec python3 -m http.server {{port}} --bind 127.0.0.1 --directory "$out/web"166 exec python3 -m http.server {{port}} --bind 127.0.0.1 --directory "$out/web"
340 167
341-# A sandbox left running with the container's own image, volumes and168+# --- the work behind the verbs ------------------------------------------
342-# environment, and the command to get into it. `modal shell --image` cannot
343-# be pointed at a published Modal image like arch-nix, so attaching to a
344-# running sandbox is the only way to get a shell that is the container.
345-#
346-# It blocks: an ephemeral app stops when its entrypoint returns and takes the
347-# sandbox with it. Attach from a second terminal, and Ctrl-C here when done —
348-# the sandbox bills until you do.
349-#
350-# just modal-shell flutter-dev, the usual one
351-# just modal-shell flutter-web the web bundle container
352-modal-shell container="flutter-dev":
353- #!/usr/bin/env bash
354- set -euo pipefail
355- cd "{{justfile_directory()}}"
356- exec modal run ".modal/{{container}}/container.py" --shell
357-
358-# The Nim core's test suite.
359-#
360-# It needs no Flutter, no Dart and no Android SDK — which is the point of
361-# having the logic here rather than under `common/`: a rule about the IRC wire
362-# format can be checked in a second, on any machine, without a toolchain that
363-# takes minutes to enter.
364-#
365-# just nim-test the whole suite
366-# just nim-test tircparse one file
367-nim-test file="":
368- #!/usr/bin/env bash
369- set -euo pipefail
370- cd "{{justfile_directory()}}"
371- if [ -z "${FRQ_NIM:-}" ]; then
372- exec {{nix}} develop .#nim --max-jobs {{jobs}} --command just nim-test "$@"
373- fi
374- cd nim
375- if [ -n "{{file}}" ]; then
376- exec nim c -r --hints:off --path:src "tests/{{file}}.nim"
377- fi
378- for t in tests/t*.nim; do
379- echo "== $t"
380- nim c -r --hints:off --path:src "$t"
381- done
382 169
383 # The Nim core as a shared library, into build/nim.170 # The Nim core as a shared library, into build/nim.
384 #171 #
@@ -387,117 +174,88 @@ nim-test file="":
387 # from another heap and dereferencing it segfaults. ORC's heap is shared.174 # from another heap and dereferencing it segfaults. ORC's heap is shared.
388 #175 #
389 # `-d:release` and not `-d:danger`: the bounds checks are what turn a176 # `-d:release` and not `-d:danger`: the bounds checks are what turn a
390-# malformed line off a socket into an exception instead of a read past the end177+# malformed line off a socket into an exception rather than a read past the
391-# of a buffer, and this parses exactly that.178+# end of a buffer, and this parses exactly that.
392-nim-lib:179+[private]
180+_nim-lib:
393 #!/usr/bin/env bash181 #!/usr/bin/env bash
394 set -euo pipefail182 set -euo pipefail
395- cd "{{justfile_directory()}}"183+ cd "{{root}}"
396- if [ -z "${FRQ_NIM:-}" ]; then184+ out="{{root}}/build/nim"
397- exec {{nix}} develop .#nim --max-jobs {{jobs}} --command just nim-lib
398- fi
399- out="{{justfile_directory()}}/build/nim"
400 mkdir -p "$out"185 mkdir -p "$out"
401- cd nim186+ exec "{{tc}}" exec -- bash -euo pipefail -c '
402- nim c --app:lib --mm:orc -d:release --hints:off \187+ cd nim
403- --path:src --out:"$out/libfrqcore.so" src/frq_core.nim188+ nim c --app:lib --mm:orc -d:release --hints:off \
404- echo "built $out/libfrqcore.so"189+ --path:src --out:"'"$out"'/libfrqcore.so" src/frq_core.nim
405- nm -D --defined-only "$out/libfrqcore.so" | grep ' T frq_' || true190+ echo "built '"$out"'/libfrqcore.so"
406-191+ nm -D --defined-only "'"$out"'/libfrqcore.so" | grep " T frq_" || true'
407-# The Dart side of the Nim boundary, on the plain Dart VM.192+
408-#193+[private]
409-# No Flutter, no emulator, no ClojureDart — `dart/frq_core` is ordinary Dart194+_nim-test *args:
410-# over `dart:ffi` and is not a Flutter package, so the test that proves the
411-# marshalling runs in a second. Passing `just nim-test` and failing this one is a
412-# marshalling bug, which is the whole reason the two suites are separate.
413-#
414-# Builds the library first: the test dlopens a real .so and there is no point
415-# reporting that it could not find one.
416-dart-test:
417 #!/usr/bin/env bash195 #!/usr/bin/env bash
418 set -euo pipefail196 set -euo pipefail
419- cd "{{justfile_directory()}}"197+ cd "{{root}}"
420- if [ -z "${FRQ_DART:-}" ]; then198+ exec "{{tc}}" exec -- bash -euo pipefail -c '
421- # nim-lib before the re-entry, not after: it enters a shell of its own199+ cd nim
422- # and doing it on the far side would build the library twice.200+ if [ -n "${1:-}" ]; then
423- just nim-lib201+ exec nim c -r --hints:off --path:src "tests/$1.nim"
424- exec {{nix}} develop .#dart --max-jobs {{jobs}} --command just dart-test202+ fi
425- fi203+ for t in tests/t*.nim; do
426- cd dart/frq_core204+ echo "== $t"
427- dart pub get205+ nim c -r --hints:off --path:src "$t"
428- dart test -r expanded206+ done' _ "$@"
429-207+
430-# The whole Nim stack against a real freeq: socket, state and screens.208+# The three Flutter targets, which differ only in what they compile and which
431-#209+# entry point they paint.
432-# Connects, waits for the room list, opens a room and prints what the tree210+#
433-# actually contains — through the FFI, so it is the path the window uses.211+# apk ClojureDart, then Gradle. Impure on purpose: Gradle resolves its
434-# Not in any suite: it needs a network and a running freeq.212+# own dependencies over the network and has sdkmanager install a
435-nim-live *args:213+# platform and build-tools into ANDROID_HOME as it goes, so the
436- #!/usr/bin/env bash214+# SDK has to be writable — which is what `just tools android` gets
437- set -euo pipefail215+# it. Everything it leaves behind is under `.toolchain/` and
438- cd "{{justfile_directory()}}"216+# gitignored.
439- if [ -z "${FRQ_DART:-}" ]; then217+# desktop the same `clojure -M:cljd compile`, Flutter's Linux target.
440- just nim-lib218+# ui no ClojureDart at all: `lib/main_nim.dart` asks the Nim core for
441- exec {{nix}} develop .#dart --max-jobs {{jobs}} --command just nim-live "$@"219+# a widget tree and paints it.
442- fi220+# app `frq.main-nim` is `frq.main` with one line changed —
443- shift || true221+# `frq.net.nim/install!` where it said `frq.net.dart/install!`.
444- cd dart/frq_core222+# Every screen, cell and action is the one that was already there.
445- dart pub get >/dev/null223+[private]
446- exec dart run tool/live_ui.dart "$@"224+_flutter target action:
447-
448-# frq with Nim owning the state and the screens, rendered by Flutter.
449-#
450-# No ClojureDart on this path. `lib/main_nim.dart` asks the Nim core for a
451-# widget tree and paints it; the screens are ports of `common/frq/screens/`.
452-#
453-# just nim-ui build it
454-# just nim-ui run open the window
455-nim-ui action="build":
456- #!/usr/bin/env bash
457- set -euo pipefail
458- cd "{{justfile_directory()}}"
459- if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then
460- just nim-lib
461- exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \
462- --command just nim-ui "$@"
463- fi
464- cd flutter
465- flutter pub get
466- export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
467- runner=()
468- [ -e /run/current-system ] || runner=("$NIXGL")
469- case "{{action}}" in
470- build) exec "${runner[@]}" flutter build linux --debug -t lib/main_nim.dart ;;
471- run) exec "${runner[@]}" flutter run -d linux -t lib/main_nim.dart ;;
472- *) echo "usage: just nim-ui [build|run]" >&2; exit 1 ;;
473- esac
474-
475-# The ClojureDart app, with the Nim core as its transport only.
476-#
477-# This is the wiring that matters: `frq.main-nim` is `frq.main` with one line
478-# changed — `frq.net.nim/install!` where it says `frq.net.dart/install!`. Every
479-# screen, every cell and every action is the one that was already there. Nim
480-# owns the socket, the TLS and the line framing, and nothing else.
481-#
482-# just nim-app build it
483-# just nim-app run open the window
484-nim-app action="build":
485 #!/usr/bin/env bash225 #!/usr/bin/env bash
486 set -euo pipefail226 set -euo pipefail
487- cd "{{justfile_directory()}}"227+ cd "{{root}}"
488- if [ -z "${FRQ_FLUTTER_DESKTOP:-}" ]; then228+ case "{{target}}" in
489- just nim-lib229+ apk) "{{tc}}" android ;;
490- exec {{nix}} develop .#flutter-desktop --max-jobs {{jobs}} \230+ ui|app) just _nim-lib ;;
491- --command just nim-app "$@"
492- fi
493- cd flutter
494- flutter pub get
495- export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
496- clojure -M:cljd compile frq.main-nim
497- runner=()
498- [ -e /run/current-system ] || runner=("$NIXGL")
499- case "{{action}}" in
500- build) exec "${runner[@]}" flutter build linux --debug -t lib/main_nim_app.dart ;;
501- run) exec "${runner[@]}" flutter run -d linux -t lib/main_nim_app.dart ;;
502- *) echo "usage: just nim-app [build|run]" >&2; exit 1 ;;
503 esac231 esac
232+ exec "{{tc}}" exec -- bash -euo pipefail -c '
233+ cd flutter
234+ # Nim resolves OpenSSL through dynlib at run time; without the host
235+ # library on the loader path `newContext` dies in a SIGSEGV that says
236+ # nothing about SSL.
237+ export LD_LIBRARY_PATH="${FRQ_OPENSSL_LIB:-}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
238+ cljd() { clojure -Sdeps "{:mvn/local-repo \"$FRQ_M2\"}" -M:cljd compile "$@"; }
239+
240+ case "$1:$2" in
241+ apk:*)
242+ cljd
243+ # Rewritten every run: it carries absolute paths.
244+ flutter config --android-sdk "$ANDROID_HOME" >/dev/null
245+ flutter build apk --debug
246+ apk=build/app/outputs/flutter-apk/app-debug.apk
247+ [ "$2" = run ] || exit 0
248+ adb install -r "$apk"
249+ exec adb shell monkey -p uk.nandi.frq \
250+ -c android.intent.category.LAUNCHER 1 ;;
251+ desktop:build) cljd; exec flutter build linux --debug ;;
252+ desktop:run) cljd; exec flutter run -d linux ;;
253+ ui:build) flutter pub get
254+ exec flutter build linux --debug -t lib/main_nim.dart ;;
255+ ui:run) flutter pub get
256+ exec flutter run -d linux -t lib/main_nim.dart ;;
257+ app:build) flutter pub get; cljd frq.main-nim
258+ exec flutter build linux --debug -t lib/main_nim_app.dart ;;
259+ app:run) flutter pub get; cljd frq.main-nim
260+ exec flutter run -d linux -t lib/main_nim_app.dart ;;
261+ esac' _ "{{target}}" "{{action}}"
modified nim/README.md +20 -19
@@ -21,7 +21,7 @@ is proven against it. Nothing is deleted on faith.
2121 ```
2222 src/frq_core.nim the C ABI: every exported symbol, and nothing else
2323 src/frq/ircparse.nim the IRC wire format
24-tests/ one per module, run by `just nim-test`
24+tests/ one per module, run by `just test nim`
2525 ```
2626
2727 `src/frq_core.nim` is the only file that knows about C. Everything under
@@ -57,7 +57,7 @@ deleted as each module lands: they are the web's implementation, not dead code.
5757
5858 ## What is wired up
5959
60-`just nim-app run` is the real client with the Nim core as its transport. Every
60+`just run app` is the real client with the Nim core as its transport. Every
6161 screen, cell and action is the one that was already there; `frq.main-nim` is
6262 `frq.main` with one line changed.
6363
@@ -88,29 +88,30 @@ replies, no images — and the path forward from it was rewriting every screen
8888 in Nim and losing all of that. It is at 1d62d1a if it is ever wanted.
8989
9090 ```bash
91-just nim-app run # the real client, Nim transport
92-just nim-test # the Nim suite
93-just dart-test # the Dart side of the boundary
94-just nim-lib # libfrqcore.so into build/nim
91+just run app # the real client, Nim transport
92+just test nim # the Nim suite
93+just test dart # the Dart side of the boundary
94+just build lib # libfrqcore.so into build/nim
9595
96-FRQ_TRACE=1 just nim-app run # every line in and out, both languages
96+FRQ_TRACE=1 just run app # every line in and out, both languages
9797 ```
9898
99-The GUI needs OpenSSL on its loader path, which the `flutter-desktop` shell
100-provides as `FRQ_OPENSSL_LIB` and the `nim-app` recipe prepends for the app
101-alone. Not set as `LD_LIBRARY_PATH` in the shell itself: that shell also runs
102-Flutter through nixGL, which does its own careful things to the loader path.
99+The GUI needs OpenSSL on its loader path: Nim resolves the entry points
100+through dynlib at run time, and without the library there `newContext` dies in
101+a SIGSEGV that says nothing about SSL. The recipe prepends `FRQ_OPENSSL_LIB`
102+for the app alone, so a host whose libssl is somewhere unusual has one variable
103+to set rather than an `LD_LIBRARY_PATH` to inherit.
103104
104105 ## Status
105106
106-`frq/ircparse.nim` is ported and tested — 29 cases, `just nim-test` — and the
107+`frq/ircparse.nim` is ported and tested — 29 cases, `just test nim` — and the
107108 ABI is exercised from C through `dlopen`, including the allocation contract
108109 under a hundred thousand parse/free cycles. That half is real.
109110
110111 The Dart binding is real too, and is `dart/frq_core` — **plain Dart, not
111112 ClojureDart**. It calls the library over `dart:ffi` and is covered by 20 tests
112113 on the Dart VM, including the UTF-8 round trip and ten thousand calls against
113-the ownership rules. `just dart-test` runs the pair of them in about a second.
114+the ownership rules. `just test dart` runs the pair of them in about a second.
114115
115116 The binding was ClojureDart for one commit and should not have been: the Nim
116117 core exists to have less Clojure in the tree, and `lookupFunction` takes two
@@ -123,8 +124,7 @@ the shipping app is unchanged and `frq.main-nim` is a second entry point
123124 beside it. `common/frq/irc/parse.cljc` is still what does the parsing on every
124125 target, including this one. That step is its
125126 own piece of work — the Flutter app takes the package as a path dependency
126-(which means a `pubspec.lock` regeneration and widening the nix build's source
127-root), the library has to reach each target (`jniLibs` for the APK, beside the
127+(which means a `pubspec.lock` regeneration), the library has to reach each target (`jniLibs` for the APK, beside the
128128 executable for the desktop bundle), and only then can a call site choose Nim
129129 on native and the ClojureDart original on the web.
130130
@@ -135,9 +135,10 @@ Modules still in `common/` and not yet here: `rooms`, `msgsig`, `crypto`,
135135 ## Building
136136
137137 ```bash
138-just nim-test # the Nim test suite
139-just nim-lib # libfrqcore.so into build/nim
138+just test nim # the Nim test suite
139+just build lib # libfrqcore.so into build/nim
140140 ```
141141
142-Both want `nix develop .#nim`, and re-enter it themselves if they are not
143-already inside.
142+Both run out of `.toolchain/`, which `tools/toolchain.sh` fills on first
143+use. What they want from the host is a C compiler -- `nim c` shells out to one
144+-- and OpenSSL.
@@ -21,7 +21,7 @@ is proven against it. Nothing is deleted on faith.
21 ```21 ```
22 src/frq_core.nim the C ABI: every exported symbol, and nothing else22 src/frq_core.nim the C ABI: every exported symbol, and nothing else
23 src/frq/ircparse.nim the IRC wire format23 src/frq/ircparse.nim the IRC wire format
24-tests/ one per module, run by `just nim-test`24+tests/ one per module, run by `just test nim`
25 ```25 ```
26 26
27 `src/frq_core.nim` is the only file that knows about C. Everything under27 `src/frq_core.nim` is the only file that knows about C. Everything under
@@ -57,7 +57,7 @@ deleted as each module lands: they are the web's implementation, not dead code.
57 57
58 ## What is wired up58 ## What is wired up
59 59
60-`just nim-app run` is the real client with the Nim core as its transport. Every60+`just run app` is the real client with the Nim core as its transport. Every
61 screen, cell and action is the one that was already there; `frq.main-nim` is61 screen, cell and action is the one that was already there; `frq.main-nim` is
62 `frq.main` with one line changed.62 `frq.main` with one line changed.
63 63
@@ -88,29 +88,30 @@ replies, no images — and the path forward from it was rewriting every screen
88 in Nim and losing all of that. It is at 1d62d1a if it is ever wanted.88 in Nim and losing all of that. It is at 1d62d1a if it is ever wanted.
89 89
90 ```bash90 ```bash
91-just nim-app run # the real client, Nim transport91+just run app # the real client, Nim transport
92-just nim-test # the Nim suite92+just test nim # the Nim suite
93-just dart-test # the Dart side of the boundary93+just test dart # the Dart side of the boundary
94-just nim-lib # libfrqcore.so into build/nim94+just build lib # libfrqcore.so into build/nim
95 95
96-FRQ_TRACE=1 just nim-app run # every line in and out, both languages96+FRQ_TRACE=1 just run app # every line in and out, both languages
97 ```97 ```
98 98
99-The GUI needs OpenSSL on its loader path, which the `flutter-desktop` shell99+The GUI needs OpenSSL on its loader path: Nim resolves the entry points
100-provides as `FRQ_OPENSSL_LIB` and the `nim-app` recipe prepends for the app100+through dynlib at run time, and without the library there `newContext` dies in
101-alone. Not set as `LD_LIBRARY_PATH` in the shell itself: that shell also runs101+a SIGSEGV that says nothing about SSL. The recipe prepends `FRQ_OPENSSL_LIB`
102-Flutter through nixGL, which does its own careful things to the loader path.102+for the app alone, so a host whose libssl is somewhere unusual has one variable
103+to set rather than an `LD_LIBRARY_PATH` to inherit.
103 104
104 ## Status105 ## Status
105 106
106-`frq/ircparse.nim` is ported and tested — 29 cases, `just nim-test` — and the107+`frq/ircparse.nim` is ported and tested — 29 cases, `just test nim` — and the
107 ABI is exercised from C through `dlopen`, including the allocation contract108 ABI is exercised from C through `dlopen`, including the allocation contract
108 under a hundred thousand parse/free cycles. That half is real.109 under a hundred thousand parse/free cycles. That half is real.
109 110
110 The Dart binding is real too, and is `dart/frq_core` — **plain Dart, not111 The Dart binding is real too, and is `dart/frq_core` — **plain Dart, not
111 ClojureDart**. It calls the library over `dart:ffi` and is covered by 20 tests112 ClojureDart**. It calls the library over `dart:ffi` and is covered by 20 tests
112 on the Dart VM, including the UTF-8 round trip and ten thousand calls against113 on the Dart VM, including the UTF-8 round trip and ten thousand calls against
113-the ownership rules. `just dart-test` runs the pair of them in about a second.114+the ownership rules. `just test dart` runs the pair of them in about a second.
114 115
115 The binding was ClojureDart for one commit and should not have been: the Nim116 The binding was ClojureDart for one commit and should not have been: the Nim
116 core exists to have less Clojure in the tree, and `lookupFunction` takes two117 core exists to have less Clojure in the tree, and `lookupFunction` takes two
@@ -123,8 +124,7 @@ the shipping app is unchanged and `frq.main-nim` is a second entry point
123 beside it. `common/frq/irc/parse.cljc` is still what does the parsing on every124 beside it. `common/frq/irc/parse.cljc` is still what does the parsing on every
124 target, including this one. That step is its125 target, including this one. That step is its
125 own piece of work — the Flutter app takes the package as a path dependency126 own piece of work — the Flutter app takes the package as a path dependency
126-(which means a `pubspec.lock` regeneration and widening the nix build's source127+(which means a `pubspec.lock` regeneration), the library has to reach each target (`jniLibs` for the APK, beside the
127-root), the library has to reach each target (`jniLibs` for the APK, beside the
128 executable for the desktop bundle), and only then can a call site choose Nim128 executable for the desktop bundle), and only then can a call site choose Nim
129 on native and the ClojureDart original on the web.129 on native and the ClojureDart original on the web.
130 130
@@ -135,9 +135,10 @@ Modules still in `common/` and not yet here: `rooms`, `msgsig`, `crypto`,
135 ## Building135 ## Building
136 136
137 ```bash137 ```bash
138-just nim-test # the Nim test suite138+just test nim # the Nim test suite
139-just nim-lib # libfrqcore.so into build/nim139+just build lib # libfrqcore.so into build/nim
140 ```140 ```
141 141
142-Both want `nix develop .#nim`, and re-enter it themselves if they are not142+Both run out of `.toolchain/`, which `tools/toolchain.sh` fills on first
143-already inside.143+use. What they want from the host is a C compiler -- `nim c` shells out to one
144+-- and OpenSSL.
modified tools/build-web.sh +1 -1
@@ -6,7 +6,7 @@
66 # one target that needs nothing from the host — no JDK of the machine's, no
77 # GTK, no Android SDK, no nix. `tools/toolchain.sh` fetches the three tarballs
88 # it does need, and everything below runs out of `.toolchain/`. The container
9-# in `.modal/flutter-web/` runs this same file; `just flutter-web` is a
9+# in `.modal/web/` runs this same file; `just build web` is a
1010 # wrapper around it.
1111 #
1212 # tools/build-web.sh build build/web
@@ -6,7 +6,7 @@
6 # one target that needs nothing from the host — no JDK of the machine's, no6 # one target that needs nothing from the host — no JDK of the machine's, no
7 # GTK, no Android SDK, no nix. `tools/toolchain.sh` fetches the three tarballs7 # GTK, no Android SDK, no nix. `tools/toolchain.sh` fetches the three tarballs
8 # it does need, and everything below runs out of `.toolchain/`. The container8 # it does need, and everything below runs out of `.toolchain/`. The container
9-# in `.modal/flutter-web/` runs this same file; `just flutter-web` is a9+# in `.modal/web/` runs this same file; `just build web` is a
10 # wrapper around it.10 # wrapper around it.
11 #11 #
12 # tools/build-web.sh build build/web12 # tools/build-web.sh build build/web
modified tools/toolchain.sh +78 -9
@@ -1,22 +1,27 @@
11 #!/usr/bin/env bash
22 # The toolchain the web build needs, fetched by hand.
33 #
4-# Three tarballs — Flutter (which carries Dart), a JDK, and the Clojure CLI —
4+# Archives — Flutter (which carries Dart), a JDK, the Clojure CLI and Nim —
55 # pinned by version and by sha256, unpacked into `.toolchain/`, and put on a
66 # PATH. That is the whole of it. No nix, no image, no devShell: a checkout
7-# plus this script is a machine that can build the web bundle, and the same
8-# script is what the Modal container runs.
7+# plus this script is a machine that can build any target here, and the same
8+# script is what the Modal containers run.
9+#
10+# The Android SDK is the one thing not fetched by default, because only
11+# `just build apk` wants it and it is large: `tools/toolchain.sh android`
12+# fetches Google's command-line tools and lets sdkmanager finish the job.
913 #
1014 # Why not DotSlash, which the rest of the repo uses for its native libraries:
1115 # DotSlash hands out an *immutable* cached artifact, and Flutter is not one.
1216 # `flutter build web` downloads its engine artifacts into `bin/cache/` inside
1317 # its own SDK directory the first time it runs, so the SDK has to be writable
14-# — which is the same thing `just apk` learned when it had to copy the
15-# store's Android SDK out to `flutter/.home` before Gradle would touch it.
18+# — which is the same reason the Android SDK below is fetched into a
19+# directory of ours rather than used read-only from anywhere.
1620 # A pinned URL and a checked hash give the reproducibility DotSlash is for;
1721 # the writability is what it cannot give.
1822 #
1923 # tools/toolchain.sh fetch whatever is missing
24+# tools/toolchain.sh android ...and the Android SDK as well
2025 # eval "$(tools/toolchain.sh env)" ...and put it on this shell's PATH
2126 # tools/toolchain.sh exec -- flutter --version
2227 #
@@ -52,13 +57,34 @@ CLOJURE_VERSION="1.12.6.1673"
5257 CLOJURE_URL="https://github.com/clojure/brew-install/releases/download/${CLOJURE_VERSION}/clojure-tools-${CLOJURE_VERSION}.tar.gz"
5358 CLOJURE_SHA="fe9194858e75d5af13c2e2aff92d710674d5bc5105f2b42f90a7d94d82ec023c"
5459
60+# Nim, for `nim/` — the portable core. Upstream's prebuilt linux_x64 tarball,
61+# so there is no bootstrap compile here. The core deliberately has no
62+# dependencies outside Nim's standard library, so the compiler is all of it.
63+#
64+# Two things Nim wants from the host rather than from here: a C compiler,
65+# because `nim c` shells out to one, and OpenSSL, because `-d:ssl` in
66+# nim/nim.cfg makes std/net resolve -lssl and -lcrypto through dynlib at run
67+# time. A missing libssl is a SIGSEGV in `newContext` that says nothing about
68+# SSL, which is why require_host_tools checks for cc up front.
69+NIM_VERSION="2.2.4"
70+NIM_URL="https://nim-lang.org/download/nim-${NIM_VERSION}-linux_x64.tar.xz"
71+NIM_SHA="791802138aaf19c8579232c50b4998ce2ae2928b791127ce5b4ef3c7af53fb46"
72+
73+# Google's command-line tools, which is the smallest thing that can install an
74+# Android SDK. The platform and build-tools are not pinned here: the versions
75+# come from whatever Flutter asks Gradle for, and sdkmanager fetches them into
76+# the same writable directory on first use. `install_android` below.
77+ANDROID_TOOLS_VERSION="11076708"
78+ANDROID_TOOLS_URL="https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_TOOLS_VERSION}_latest.zip"
79+ANDROID_TOOLS_SHA="2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258"
80+
5581 # What the host still has to bring. Small, boring, and on every machine and
5682 # in every base image that is not deliberately empty — but Flutter shells out
5783 # to `git` on its own SDK and to `unzip` on its downloads, so a missing one
5884 # fails somewhere far from here with a much worse message than this.
5985 require_host_tools() {
6086 local missing=()
61- for t in curl tar git unzip; do
87+ for t in curl tar git unzip cc; do
6288 command -v "$t" >/dev/null 2>&1 || missing+=("$t")
6389 done
6490 if [ ${#missing[@]} -gt 0 ]; then
@@ -102,6 +128,38 @@ install_clojure() {
102128 chmod +x "$dest/bin/clojure" "$dest/bin/clj"
103129 }
104130
131+# The Android SDK, which is a zip rather than a tarball and has to land in a
132+# layout sdkmanager recognises: cmdline-tools/latest/, with the tools' own
133+# top-level directory renamed. Everything after that — platform-tools, the
134+# platform, the build-tools — Gradle asks sdkmanager for as it goes, which is
135+# why this directory is ours and writable rather than a read-only artifact.
136+#
137+# No ndkVersion in android/app/build.gradle.kts, and nothing here installs an
138+# NDK: there is no native code in the app to need one.
139+install_android() {
140+ local dest="$TC/android-sdk"
141+ local stamp="$dest/.cmdline-tools.sha256"
142+ if [ ! -d "$dest/cmdline-tools/latest" ] \
143+ || [ "$(cat "$stamp" 2>/dev/null || true)" != "$ANDROID_TOOLS_SHA" ]; then
144+ echo "toolchain: fetching the Android command-line tools" >&2
145+ local dl="$TC/.download.android"
146+ rm -rf "$dest/cmdline-tools" "$dl"
147+ mkdir -p "$dest/cmdline-tools"
148+ curl -fsSL --retry 3 -o "$dl" "$ANDROID_TOOLS_URL"
149+ echo "$ANDROID_TOOLS_SHA $dl" | sha256sum -c - >/dev/null
150+ unzip -q "$dl" -d "$dest/cmdline-tools"
151+ rm -f "$dl"
152+ mv "$dest/cmdline-tools/cmdline-tools" "$dest/cmdline-tools/latest"
153+ echo "$ANDROID_TOOLS_SHA" > "$stamp"
154+ fi
155+ # Gradle will not install anything into an SDK whose licences are
156+ # unaccepted, and it fails late and obscurely when they are not.
157+ if [ ! -d "$dest/licenses" ]; then
158+ JAVA_HOME="$TC/jdk" ANDROID_HOME="$dest" \
159+ yes | "$dest/cmdline-tools/latest/bin/sdkmanager" --licenses >/dev/null
160+ fi
161+}
162+
105163 install_all() {
106164 require_host_tools
107165 mkdir -p "$TC"
@@ -109,6 +167,7 @@ install_all() {
109167 install_archive jdk "$JDK_URL" "$JDK_SHA" 1
110168 install_archive clojure "$CLOJURE_URL" "$CLOJURE_SHA" 1
111169 install_clojure
170+ install_archive nim "$NIM_URL" "$NIM_SHA" 1
112171 }
113172
114173 # The environment, as shell. Everything that would otherwise land in a home
@@ -116,7 +175,7 @@ install_all() {
116175 # cache, the git dependencies tools.deps clones, the local maven repo. One
117176 # directory to keep on a volume, one directory to delete when it goes wrong.
118177 #
119-# GITLIBS and the maven repo are set for the reason `just apk` sets them —
178+# GITLIBS and the maven repo are named here because
120179 # the JVM reads user.home out of /etc/passwd, so neither of them follows HOME.
121180 print_env() {
122181 cat <<ENV
@@ -137,12 +196,20 @@ export JAVA_HOME="$TC/jdk"
137196 export PUB_CACHE="$TC/pub-cache"
138197 export GITLIBS="$TC/gitlibs"
139198 export FRQ_M2="$TC/m2"
140-export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:\$PATH"
199+export ANDROID_HOME="$TC/android-sdk"
200+export ANDROID_SDK_ROOT="$TC/android-sdk"
201+# adb keeps the key the phone has already trusted here. Left to its default
202+# it would follow HOME, and a build that moved HOME would hand the device a
203+# new identity -- after which the deploy ends in "no devices/emulators found"
204+# while \`adb devices\` in any other shell lists the phone perfectly well.
205+export ANDROID_USER_HOME="\${ANDROID_USER_HOME:-\$HOME/.android}"
206+export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:$TC/nim/bin:$TC/android-sdk/platform-tools:\$PATH"
141207 ENV
142208 }
143209
144210 case "${1:-install}" in
145211 install) install_all ;;
212+ android) install_all; install_android ;;
146213 env) install_all; print_env ;;
147214 exec)
148215 install_all
@@ -155,6 +222,8 @@ case "${1:-install}" in
155222 echo "flutter $FLUTTER_VERSION"
156223 echo "jdk $JDK_VERSION"
157224 echo "clojure $CLOJURE_VERSION"
225+ echo "nim $NIM_VERSION"
226+ echo "android-tools $ANDROID_TOOLS_VERSION"
158227 ;;
159- *) echo "usage: toolchain.sh [install|env|exec -- cmd...|versions]" >&2; exit 1 ;;
228+ *) echo "usage: toolchain.sh [install|android|env|exec -- cmd...|versions]" >&2; exit 1 ;;
160229 esac
@@ -1,22 +1,27 @@
1 #!/usr/bin/env bash1 #!/usr/bin/env bash
2 # The toolchain the web build needs, fetched by hand.2 # The toolchain the web build needs, fetched by hand.
3 #3 #
4-# Three tarballs — Flutter (which carries Dart), a JDK, and the Clojure CLI —4+# Archives — Flutter (which carries Dart), a JDK, the Clojure CLI and Nim —
5 # pinned by version and by sha256, unpacked into `.toolchain/`, and put on a5 # pinned by version and by sha256, unpacked into `.toolchain/`, and put on a
6 # PATH. That is the whole of it. No nix, no image, no devShell: a checkout6 # PATH. That is the whole of it. No nix, no image, no devShell: a checkout
7-# plus this script is a machine that can build the web bundle, and the same7+# plus this script is a machine that can build any target here, and the same
8-# script is what the Modal container runs.8+# script is what the Modal containers run.
9+#
10+# The Android SDK is the one thing not fetched by default, because only
11+# `just build apk` wants it and it is large: `tools/toolchain.sh android`
12+# fetches Google's command-line tools and lets sdkmanager finish the job.
9 #13 #
10 # Why not DotSlash, which the rest of the repo uses for its native libraries:14 # Why not DotSlash, which the rest of the repo uses for its native libraries:
11 # DotSlash hands out an *immutable* cached artifact, and Flutter is not one.15 # DotSlash hands out an *immutable* cached artifact, and Flutter is not one.
12 # `flutter build web` downloads its engine artifacts into `bin/cache/` inside16 # `flutter build web` downloads its engine artifacts into `bin/cache/` inside
13 # its own SDK directory the first time it runs, so the SDK has to be writable17 # its own SDK directory the first time it runs, so the SDK has to be writable
14-# — which is the same thing `just apk` learned when it had to copy the18+# — which is the same reason the Android SDK below is fetched into a
15-# store's Android SDK out to `flutter/.home` before Gradle would touch it.19+# directory of ours rather than used read-only from anywhere.
16 # A pinned URL and a checked hash give the reproducibility DotSlash is for;20 # A pinned URL and a checked hash give the reproducibility DotSlash is for;
17 # the writability is what it cannot give.21 # the writability is what it cannot give.
18 #22 #
19 # tools/toolchain.sh fetch whatever is missing23 # tools/toolchain.sh fetch whatever is missing
24+# tools/toolchain.sh android ...and the Android SDK as well
20 # eval "$(tools/toolchain.sh env)" ...and put it on this shell's PATH25 # eval "$(tools/toolchain.sh env)" ...and put it on this shell's PATH
21 # tools/toolchain.sh exec -- flutter --version26 # tools/toolchain.sh exec -- flutter --version
22 #27 #
@@ -52,13 +57,34 @@ CLOJURE_VERSION="1.12.6.1673"
52 CLOJURE_URL="https://github.com/clojure/brew-install/releases/download/${CLOJURE_VERSION}/clojure-tools-${CLOJURE_VERSION}.tar.gz"57 CLOJURE_URL="https://github.com/clojure/brew-install/releases/download/${CLOJURE_VERSION}/clojure-tools-${CLOJURE_VERSION}.tar.gz"
53 CLOJURE_SHA="fe9194858e75d5af13c2e2aff92d710674d5bc5105f2b42f90a7d94d82ec023c"58 CLOJURE_SHA="fe9194858e75d5af13c2e2aff92d710674d5bc5105f2b42f90a7d94d82ec023c"
54 59
60+# Nim, for `nim/` — the portable core. Upstream's prebuilt linux_x64 tarball,
61+# so there is no bootstrap compile here. The core deliberately has no
62+# dependencies outside Nim's standard library, so the compiler is all of it.
63+#
64+# Two things Nim wants from the host rather than from here: a C compiler,
65+# because `nim c` shells out to one, and OpenSSL, because `-d:ssl` in
66+# nim/nim.cfg makes std/net resolve -lssl and -lcrypto through dynlib at run
67+# time. A missing libssl is a SIGSEGV in `newContext` that says nothing about
68+# SSL, which is why require_host_tools checks for cc up front.
69+NIM_VERSION="2.2.4"
70+NIM_URL="https://nim-lang.org/download/nim-${NIM_VERSION}-linux_x64.tar.xz"
71+NIM_SHA="791802138aaf19c8579232c50b4998ce2ae2928b791127ce5b4ef3c7af53fb46"
72+
73+# Google's command-line tools, which is the smallest thing that can install an
74+# Android SDK. The platform and build-tools are not pinned here: the versions
75+# come from whatever Flutter asks Gradle for, and sdkmanager fetches them into
76+# the same writable directory on first use. `install_android` below.
77+ANDROID_TOOLS_VERSION="11076708"
78+ANDROID_TOOLS_URL="https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_TOOLS_VERSION}_latest.zip"
79+ANDROID_TOOLS_SHA="2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258"
80+
55 # What the host still has to bring. Small, boring, and on every machine and81 # What the host still has to bring. Small, boring, and on every machine and
56 # in every base image that is not deliberately empty — but Flutter shells out82 # in every base image that is not deliberately empty — but Flutter shells out
57 # to `git` on its own SDK and to `unzip` on its downloads, so a missing one83 # to `git` on its own SDK and to `unzip` on its downloads, so a missing one
58 # fails somewhere far from here with a much worse message than this.84 # fails somewhere far from here with a much worse message than this.
59 require_host_tools() {85 require_host_tools() {
60 local missing=()86 local missing=()
61- for t in curl tar git unzip; do87+ for t in curl tar git unzip cc; do
62 command -v "$t" >/dev/null 2>&1 || missing+=("$t")88 command -v "$t" >/dev/null 2>&1 || missing+=("$t")
63 done89 done
64 if [ ${#missing[@]} -gt 0 ]; then90 if [ ${#missing[@]} -gt 0 ]; then
@@ -102,6 +128,38 @@ install_clojure() {
102 chmod +x "$dest/bin/clojure" "$dest/bin/clj"128 chmod +x "$dest/bin/clojure" "$dest/bin/clj"
103 }129 }
104 130
131+# The Android SDK, which is a zip rather than a tarball and has to land in a
132+# layout sdkmanager recognises: cmdline-tools/latest/, with the tools' own
133+# top-level directory renamed. Everything after that — platform-tools, the
134+# platform, the build-tools — Gradle asks sdkmanager for as it goes, which is
135+# why this directory is ours and writable rather than a read-only artifact.
136+#
137+# No ndkVersion in android/app/build.gradle.kts, and nothing here installs an
138+# NDK: there is no native code in the app to need one.
139+install_android() {
140+ local dest="$TC/android-sdk"
141+ local stamp="$dest/.cmdline-tools.sha256"
142+ if [ ! -d "$dest/cmdline-tools/latest" ] \
143+ || [ "$(cat "$stamp" 2>/dev/null || true)" != "$ANDROID_TOOLS_SHA" ]; then
144+ echo "toolchain: fetching the Android command-line tools" >&2
145+ local dl="$TC/.download.android"
146+ rm -rf "$dest/cmdline-tools" "$dl"
147+ mkdir -p "$dest/cmdline-tools"
148+ curl -fsSL --retry 3 -o "$dl" "$ANDROID_TOOLS_URL"
149+ echo "$ANDROID_TOOLS_SHA $dl" | sha256sum -c - >/dev/null
150+ unzip -q "$dl" -d "$dest/cmdline-tools"
151+ rm -f "$dl"
152+ mv "$dest/cmdline-tools/cmdline-tools" "$dest/cmdline-tools/latest"
153+ echo "$ANDROID_TOOLS_SHA" > "$stamp"
154+ fi
155+ # Gradle will not install anything into an SDK whose licences are
156+ # unaccepted, and it fails late and obscurely when they are not.
157+ if [ ! -d "$dest/licenses" ]; then
158+ JAVA_HOME="$TC/jdk" ANDROID_HOME="$dest" \
159+ yes | "$dest/cmdline-tools/latest/bin/sdkmanager" --licenses >/dev/null
160+ fi
161+}
162+
105 install_all() {163 install_all() {
106 require_host_tools164 require_host_tools
107 mkdir -p "$TC"165 mkdir -p "$TC"
@@ -109,6 +167,7 @@ install_all() {
109 install_archive jdk "$JDK_URL" "$JDK_SHA" 1167 install_archive jdk "$JDK_URL" "$JDK_SHA" 1
110 install_archive clojure "$CLOJURE_URL" "$CLOJURE_SHA" 1168 install_archive clojure "$CLOJURE_URL" "$CLOJURE_SHA" 1
111 install_clojure169 install_clojure
170+ install_archive nim "$NIM_URL" "$NIM_SHA" 1
112 }171 }
113 172
114 # The environment, as shell. Everything that would otherwise land in a home173 # The environment, as shell. Everything that would otherwise land in a home
@@ -116,7 +175,7 @@ install_all() {
116 # cache, the git dependencies tools.deps clones, the local maven repo. One175 # cache, the git dependencies tools.deps clones, the local maven repo. One
117 # directory to keep on a volume, one directory to delete when it goes wrong.176 # directory to keep on a volume, one directory to delete when it goes wrong.
118 #177 #
119-# GITLIBS and the maven repo are set for the reason `just apk` sets them —178+# GITLIBS and the maven repo are named here because
120 # the JVM reads user.home out of /etc/passwd, so neither of them follows HOME.179 # the JVM reads user.home out of /etc/passwd, so neither of them follows HOME.
121 print_env() {180 print_env() {
122 cat <<ENV181 cat <<ENV
@@ -137,12 +196,20 @@ export JAVA_HOME="$TC/jdk"
137 export PUB_CACHE="$TC/pub-cache"196 export PUB_CACHE="$TC/pub-cache"
138 export GITLIBS="$TC/gitlibs"197 export GITLIBS="$TC/gitlibs"
139 export FRQ_M2="$TC/m2"198 export FRQ_M2="$TC/m2"
140-export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:\$PATH"199+export ANDROID_HOME="$TC/android-sdk"
200+export ANDROID_SDK_ROOT="$TC/android-sdk"
201+# adb keeps the key the phone has already trusted here. Left to its default
202+# it would follow HOME, and a build that moved HOME would hand the device a
203+# new identity -- after which the deploy ends in "no devices/emulators found"
204+# while \`adb devices\` in any other shell lists the phone perfectly well.
205+export ANDROID_USER_HOME="\${ANDROID_USER_HOME:-\$HOME/.android}"
206+export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:$TC/nim/bin:$TC/android-sdk/platform-tools:\$PATH"
141 ENV207 ENV
142 }208 }
143 209
144 case "${1:-install}" in210 case "${1:-install}" in
145 install) install_all ;;211 install) install_all ;;
212+ android) install_all; install_android ;;
146 env) install_all; print_env ;;213 env) install_all; print_env ;;
147 exec)214 exec)
148 install_all215 install_all
@@ -155,6 +222,8 @@ case "${1:-install}" in
155 echo "flutter $FLUTTER_VERSION"222 echo "flutter $FLUTTER_VERSION"
156 echo "jdk $JDK_VERSION"223 echo "jdk $JDK_VERSION"
157 echo "clojure $CLOJURE_VERSION"224 echo "clojure $CLOJURE_VERSION"
225+ echo "nim $NIM_VERSION"
226+ echo "android-tools $ANDROID_TOOLS_VERSION"
158 ;;227 ;;
159- *) echo "usage: toolchain.sh [install|env|exec -- cmd...|versions]" >&2; exit 1 ;;228+ *) echo "usage: toolchain.sh [install|android|env|exec -- cmd...|versions]" >&2; exit 1 ;;
160 esac229 esac