Three tarballs where a devShell was
The web build is the one Flutter target that wants nothing of the host -- no Android SDK, no GTK, no C++, no nixGL -- so what its devShell was supplying was a Dart and a JVM. `tools/toolchain.sh` fetches those as tarballs pinned by sha256, `tools/build-web.sh` builds out of them, and `just flutter-web` is a wrapper around that script rather than a nix re-entry. The container in `.modal/flutter-web/` runs the same file on `debian:13-slim`: its image build is one apt line, and warming a shell, printing its environment, caching that against flake.lock and copying a closure back to a volume afterwards are all gone with the nix they were for. A warm run is ~3m14 end to end, of which 49s is ClojureDart and 20s is dart2js. `--no-wasm-dry-run`, measured at 52.0s against 55.7s: every build was dry-run compiling the whole program against a backend it can never use, since `frq.io.web` and five more are `dart:html` on purpose. No fast mode, though, and that is the measurement and not an omission: dart2js at -O1 came out at 52.5s against release's 49.8s on the same change, because what it spends its time on here is linking rather than optimising. A second, larger bundle for noise. The numbers are in `tools/build-web.sh` so the idea is not had twice. Two things found on the way. `flutter/lib/cljd-out` was being uploaded from the laptop and rsynced over the volume's copy -- the compiler's own output, overwritten with a stale one, which is the incremental build undone by the thing meant to feed it. And Flutter's SDK is a git checkout, so on a volume it trips "detected dubious ownership"; the dart process the live analyzer talks to dies of it, and the compile ends at `EOF while reading` with nothing about git in the message. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
5ce66d5 parent: 562900f modified
.gitignore +5 -0 | @@ -1,6 +1,11 @@ | ||
| 1 | 1 | # The native libraries `just lib` links out of the DotSlash cache. |
| 2 | 2 | /build/ |
| 3 | 3 | |
| 4 | +# The pinned Flutter/JDK/Clojure tarballs `tools/toolchain.sh` fetches, plus | |
| 5 | +# the pub cache, gitlibs and maven repo it keeps beside them. A gigabyte of | |
| 6 | +# SDK, reproducible from the hashes in that script. | |
| 7 | +/.toolchain/ | |
| 8 | + | |
| 4 | 9 | # Tool caches. |
| 5 | 10 | /.cpcache/ |
| 6 | 11 | /.jolt/ |
| @@ -1,6 +1,11 @@ | |||
| 1 | # The native libraries `just lib` links out of the DotSlash cache. | 1 | # The native libraries `just lib` links out of the DotSlash cache. |
| 2 | /build/ | 2 | /build/ |
| 3 | 3 | ||
| 4 | +# The pinned Flutter/JDK/Clojure tarballs `tools/toolchain.sh` fetches, plus | ||
| 5 | +# the pub cache, gitlibs and maven repo it keeps beside them. A gigabyte of | ||
| 6 | +# SDK, reproducible from the hashes in that script. | ||
| 7 | +/.toolchain/ | ||
| 8 | + | ||
| 4 | # Tool caches. | 9 | # Tool caches. |
| 5 | /.cpcache/ | 10 | /.cpcache/ |
| 6 | /.jolt/ | 11 | /.jolt/ |
modified
.modal/flutter-web/README.md +32 -8 | @@ -4,14 +4,38 @@ | ||
| 4 | 4 | just modal flutter-web |
| 5 | 5 | |
| 6 | 6 | Defined by `container.toml`; see `../spec.md` for the keys. |
| 7 | -Built on the published `arch-nix` image. | |
| 7 | +Built on `debian:13-slim`. | |
| 8 | 8 | |
| 9 | -`flutter-dev` with the Linux target swapped for the web one: the same | |
| 10 | -`clojure -M:cljd compile` over the same `flutter/src` and `common/`, | |
| 11 | -then dart2js instead of CMake and Ninja. Same incremental shape -- | |
| 12 | -the working tree and Flutter's caches live on the `devshell` volume, | |
| 13 | -under `frq-flutter-web/` so the desktop container's directory beside | |
| 14 | -it is untouched. | |
| 9 | +No nix, and that is the point of this container rather than an | |
| 10 | +incidental fact about it. The build is `tools/build-web.sh`, which | |
| 11 | +gets its Flutter, its JDK and its Clojure CLI from | |
| 12 | +`tools/toolchain.sh` -- three tarballs pinned by sha256 and unpacked | |
| 13 | +into a directory. So the image build is one `apt-get install` of | |
| 14 | +curl, git, rsync, tar and the two unarchivers, and everything that | |
| 15 | +used to happen before a line of Dart was compiled -- warming a | |
| 16 | +devShell, printing its environment, caching that against flake.lock, | |
| 17 | +copying a nix closure back to a volume afterwards -- does not happen | |
| 18 | +at all. The toolchain lands on the volume and the second run finds | |
| 19 | +it there. | |
| 20 | + | |
| 21 | +The other two Flutter targets keep their devShells: `apk` needs the | |
| 22 | +Android SDK and `flutter-desktop` needs GTK and a C++ toolchain, and | |
| 23 | +a host toolchain is what nix is better at than a tarball. The web | |
| 24 | +target needs a Dart and a JVM, which is what a tarball is for. | |
| 25 | + | |
| 26 | +Same incremental shape as before -- the working tree, the generated | |
| 27 | +Dart under `flutter/lib/cljd-out` and Flutter's caches live on the | |
| 28 | +`devshell` volume, under `frq-flutter-web/` so the desktop | |
| 29 | +container's directory beside it is untouched. None of them is copied | |
| 30 | +in from the laptop: a checkout's copy of the compiler's output is | |
| 31 | +not this container's, and overwriting the volume's with it is how an | |
| 32 | +incremental build stops being one. | |
| 33 | + | |
| 34 | +One build mode, not two. A `fast` mode (dart2js -O1, no icon | |
| 35 | +tree-shaking, no service worker) measured 52.5s against the release | |
| 36 | +build's 49.8s on the same source change, so what it bought was a | |
| 37 | +bigger bundle. `--no-wasm-dry-run` is the flag that did pay, and it | |
| 38 | +is in the one build there is. | |
| 15 | 39 | |
| 16 | 40 | Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command |
| 17 | 41 | is the sandbox's own process, so it dies when the command exits. |
| @@ -21,7 +45,7 @@ ports` tunnels 8080 out, and the URL is printed once the sandbox is | ||
| 21 | 45 | scheduled: |
| 22 | 46 | |
| 23 | 47 | modal run .modal/flutter-web/container.py \ |
| 24 | - --command 'cd /devshell/frq-flutter-web && nix develop /app#flutter-web --command just -f /devshell/frq-flutter-web/justfile flutter-web serve' | |
| 48 | + --command 'cd /devshell/frq-flutter-web && tools/build-web.sh serve 8080' | |
| 25 | 49 | |
| 26 | 50 | That blocks until you Ctrl-C it, and it bills until you do. |
| 27 | 51 | |
| @@ -4,14 +4,38 @@ | |||
| 4 | just modal flutter-web | 4 | just modal flutter-web |
| 5 | 5 | ||
| 6 | Defined by `container.toml`; see `../spec.md` for the keys. | 6 | Defined by `container.toml`; see `../spec.md` for the keys. |
| 7 | -Built on the published `arch-nix` image. | 7 | +Built on `debian:13-slim`. |
| 8 | 8 | ||
| 9 | -`flutter-dev` with the Linux target swapped for the web one: the same | 9 | +No nix, and that is the point of this container rather than an |
| 10 | -`clojure -M:cljd compile` over the same `flutter/src` and `common/`, | 10 | +incidental fact about it. The build is `tools/build-web.sh`, which |
| 11 | -then dart2js instead of CMake and Ninja. Same incremental shape -- | 11 | +gets its Flutter, its JDK and its Clojure CLI from |
| 12 | -the working tree and Flutter's caches live on the `devshell` volume, | 12 | +`tools/toolchain.sh` -- three tarballs pinned by sha256 and unpacked |
| 13 | -under `frq-flutter-web/` so the desktop container's directory beside | 13 | +into a directory. So the image build is one `apt-get install` of |
| 14 | -it is untouched. | 14 | +curl, git, rsync, tar and the two unarchivers, and everything that |
| 15 | +used to happen before a line of Dart was compiled -- warming a | ||
| 16 | +devShell, printing its environment, caching that against flake.lock, | ||
| 17 | +copying a nix closure back to a volume afterwards -- does not happen | ||
| 18 | +at all. The toolchain lands on the volume and the second run finds | ||
| 19 | +it there. | ||
| 20 | + | ||
| 21 | +The other two Flutter targets keep their devShells: `apk` needs the | ||
| 22 | +Android SDK and `flutter-desktop` needs GTK and a C++ toolchain, and | ||
| 23 | +a host toolchain is what nix is better at than a tarball. The web | ||
| 24 | +target needs a Dart and a JVM, which is what a tarball is for. | ||
| 25 | + | ||
| 26 | +Same incremental shape as before -- the working tree, the generated | ||
| 27 | +Dart under `flutter/lib/cljd-out` and Flutter's caches live on the | ||
| 28 | +`devshell` volume, under `frq-flutter-web/` so the desktop | ||
| 29 | +container's directory beside it is untouched. None of them is copied | ||
| 30 | +in from the laptop: a checkout's copy of the compiler's output is | ||
| 31 | +not this container's, and overwriting the volume's with it is how an | ||
| 32 | +incremental build stops being one. | ||
| 33 | + | ||
| 34 | +One build mode, not two. A `fast` mode (dart2js -O1, no icon | ||
| 35 | +tree-shaking, no service worker) measured 52.5s against the release | ||
| 36 | +build's 49.8s on the same source change, so what it bought was a | ||
| 37 | +bigger bundle. `--no-wasm-dry-run` is the flag that did pay, and it | ||
| 38 | +is in the one build there is. | ||
| 15 | 39 | ||
| 16 | Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command | 40 | Runs as a Sandbox on a real VM (kernel 6.x, not gVisor). The command |
| 17 | is the sandbox's own process, so it dies when the command exits. | 41 | is the sandbox's own process, so it dies when the command exits. |
| @@ -21,7 +45,7 @@ ports` tunnels 8080 out, and the URL is printed once the sandbox is | |||
| 21 | scheduled: | 45 | scheduled: |
| 22 | 46 | ||
| 23 | modal run .modal/flutter-web/container.py \ | 47 | modal run .modal/flutter-web/container.py \ |
| 24 | - --command 'cd /devshell/frq-flutter-web && nix develop /app#flutter-web --command just -f /devshell/frq-flutter-web/justfile flutter-web serve' | 48 | + --command 'cd /devshell/frq-flutter-web && tools/build-web.sh serve 8080' |
| 25 | 49 | ||
| 26 | That blocks until you Ctrl-C it, and it bills until you do. | 50 | That blocks until you Ctrl-C it, and it bills until you do. |
| 27 | 51 | ||
modified
.modal/flutter-web/container.toml +68 -169 | @@ -1,7 +1,11 @@ | ||
| 1 | 1 | [container] |
| 2 | 2 | name = "frq-flutter-web" |
| 3 | -description = "the Flutter web build, incremental, in a nix devShell" | |
| 4 | -base = "arch-nix" | |
| 3 | +description = "the Flutter web build, incremental, from a pinned toolchain" | |
| 4 | +# `debian:13-slim` and not `arch-nix`: there is no nix in this container any | |
| 5 | +# more. The build is `tools/build-web.sh`, which fetches its own Flutter, JDK | |
| 6 | +# and Clojure CLI by pinned sha256, so what the image owes it is curl, git, | |
| 7 | +# tar and a C runtime — and the smallest image that has them is the right one. | |
| 8 | +registry = "debian:13-slim" | |
| 5 | 9 | # A Sandbox, not a Function: it runs on a real VM, the command is the |
| 6 | 10 | # sandbox's own process so it dies when the command does, and only a Sandbox |
| 7 | 11 | # can hold open a tunnel -- which is the whole of `serve`. |
| @@ -12,48 +16,18 @@ runtime = "sandbox" | ||
| 12 | 16 | # levels up and `.` is the whole tree. |
| 13 | 17 | context = "../.." |
| 14 | 18 | include = ["."] |
| 15 | -# The devShell, baked in rather than entered -- `flutter-dev`'s trick and for | |
| 16 | -# its reasons. `print-dev-env` writes the whole environment out as shell and | |
| 17 | -# realises its inputs on the way, so the closure becomes an image layer | |
| 18 | -# instead of a fetch every container pays for; sourcing it from .bashrc means | |
| 19 | -# a shell attached to this container *is* the devShell. | |
| 19 | +# The whole image build, and it is one apt line. What used to be here -- a | |
| 20 | +# nix store to populate, a devShell to print, a closure to warm before the | |
| 21 | +# source arrived so an edit would not invalidate it -- is gone with the nix | |
| 22 | +# it was for. There is no `warm` list any more either: this step reads | |
| 23 | +# nothing out of the tree, so nothing in the tree can invalidate it. | |
| 20 | 24 | # |
| 21 | -# `dev` stays for the case where the baked env is stale against a flake edit. | |
| 22 | -# The toolchain layer, and what it is allowed to depend on. | |
| 23 | -# | |
| 24 | -# `warm` is copied before `setup` runs and is deliberately six files: the | |
| 25 | -# flake and its lock, plus the four `cljd-deps` actually reads -- it does | |
| 26 | -# `cp ${./common/deps.edn}`, `${./flutter/deps.edn}`, `${./flutter/pubspec.yaml}` | |
| 27 | -# and `${./flutter/pubspec.lock}` and nothing else. That is the whole of what | |
| 28 | -# `nix develop .#flutter-web` needs to evaluate, so this layer moves when a | |
| 29 | -# dependency moves and not when a line of ClojureDart does. | |
| 30 | -# | |
| 31 | -# The point of the split: `[build] commands` run after the full source copy, | |
| 32 | -# so editing `flutter/src/frq/net/web.cljd` used to invalidate them and spend | |
| 33 | -# minutes re-warming a devShell that had not changed. Here the image is built | |
| 34 | -# from the toolchain and the program is built in the sandbox, which is where | |
| 35 | -# it was always going to happen anyway. | |
| 36 | -warm = [ | |
| 37 | - "flake.nix", "flake.lock", | |
| 38 | - "common/deps.edn", | |
| 39 | - "flutter/deps.edn", "flutter/pubspec.yaml", "flutter/pubspec.lock", | |
| 40 | -] | |
| 41 | -# The devShell, baked in rather than entered. `print-dev-env` writes the whole | |
| 42 | -# environment out as shell and realises its inputs on the way, so the closure | |
| 43 | -# becomes an image layer instead of a fetch every container pays for; sourcing | |
| 44 | -# it from .bashrc means a shell attached to this container *is* the devShell. | |
| 45 | -# | |
| 46 | -# Best-effort, and the reason is gVisor. An image build is a Function | |
| 47 | -# underneath, so it cannot *build* a derivation -- and `print-dev-env` realises | |
| 48 | -# the shell's inputs, which here includes `flutter-wrapped-…-sdk-links.drv`, | |
| 49 | -# in no binary cache and therefore built. Under gVisor that dies with | |
| 50 | -# `unexpected EOF reading a line`, and the ptyshim that papered over it is | |
| 51 | -# deprecated. Nothing here is load-bearing: [run] enters `nix develop` itself, | |
| 52 | -# on the VM, where a real pty makes the same build work. | |
| 25 | +# git, because Flutter shells out to it against its own SDK checkout and | |
| 26 | +# refuses to run without one; unzip and xz-utils, because that is what the | |
| 27 | +# SDK and the toolchain tarballs arrive as; rsync for the sync below; | |
| 28 | +# ca-certificates so curl can verify what it fetches. | |
| 53 | 29 | setup = [ |
| 54 | - "nix print-dev-env /app#flutter-web --accept-flake-config --extra-substituters file:///nix-cache > /etc/devshell.sh || rm -f /etc/devshell.sh", | |
| 55 | - "echo '[ -s /etc/devshell.sh ] && . /etc/devshell.sh' >> /root/.bashrc", | |
| 56 | - "printf '#!/bin/sh\\nexec nix develop /app#flutter-web \"$@\"\\n' > /usr/local/bin/dev && chmod +x /usr/local/bin/dev", | |
| 30 | + "apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git rsync tar unzip xz-utils && rm -rf /var/lib/apt/lists/*", | |
| 57 | 31 | ] |
| 58 | 32 | # The build state a local checkout carries, wanted by nothing out here: this |
| 59 | 33 | # container builds into a volume of its own, and the jolt and clojure caches |
| @@ -61,6 +35,18 @@ setup = [ | ||
| 61 | 35 | ignore = [ |
| 62 | 36 | "flutter/build", "flutter/.home", "flutter/.dart_tool", |
| 63 | 37 | "flutter/.clojuredart", "flutter/.cpcache", |
| 38 | + # The ClojureDart compiler's output, gitignored and -- until now -- | |
| 39 | + # uploaded anyway, because this list is explicit and does not read | |
| 40 | + # .gitignore. It is megabytes of generated Dart per run, and worse than | |
| 41 | + # the upload is what happened on arrival: the rsync below overwrote the | |
| 42 | + # volume's copy, the one the last container compiled, with a laptop's. | |
| 43 | + # Every file whose content differed then looked new to the compiler and | |
| 44 | + # to Flutter, which is the incremental build undone by the thing that was | |
| 45 | + # meant to feed it. It belongs to the volume, like flutter/build. | |
| 46 | + "flutter/lib/cljd-out", | |
| 47 | + # The toolchain, which is a gigabyte of Flutter SDK and lives on the | |
| 48 | + # volume out here. | |
| 49 | + ".toolchain", | |
| 64 | 50 | ".jolt", ".cpcache", "result", "build", ".git", |
| 65 | 51 | # An editor's linter rewrites this while the upload is reading it, and |
| 66 | 52 | # Modal fails the whole run with "was modified during build process". |
| @@ -68,23 +54,22 @@ ignore = [ | ||
| 68 | 54 | ".clj-kondo", |
| 69 | 55 | ] |
| 70 | 56 | |
| 71 | -# `nix-cache` is the binary cache every container here reads from and writes | |
| 72 | -# back to. `devshell` is the working state of a `nix develop` loop, shared by | |
| 73 | -# every container that has one -- each gets its own directory under it, named | |
| 74 | -# for the devShell it belongs to, so `flutter-web` and `flutter-desktop` never | |
| 75 | -# write the same tree even though they share a `.home`-shaped cache layout. | |
| 76 | -# Modal Volumes have no locking, so those directory names are the only thing | |
| 77 | -# keeping them apart, and two runs of the *same* devshell must not overlap. | |
| 57 | +# One volume now, where there were two: the nix binary cache went with nix. | |
| 58 | +# `devshell` is the working state of an incremental loop, shared by every | |
| 59 | +# container that has one -- each gets its own directory under it, named for | |
| 60 | +# what it belongs to, so `flutter-web` and `flutter-desktop` never write the | |
| 61 | +# same tree. Modal Volumes have no locking, so those directory names are the | |
| 62 | +# only thing keeping them apart, and two runs of the *same* container must | |
| 63 | +# not overlap. | |
| 78 | 64 | [volumes] |
| 79 | -nix-cache = "/nix-cache" | |
| 80 | -devshell = "/devshell" | |
| 65 | +devshell = "/devshell" | |
| 81 | 66 | |
| 82 | 67 | [resources] |
| 83 | 68 | cpu = 8 |
| 84 | 69 | memory = 16384 |
| 85 | 70 | timeout = 3600 |
| 86 | 71 | |
| 87 | -# The port `just flutter-web serve` listens on, tunnelled out. Nothing is | |
| 72 | +# The port `tools/build-web.sh serve` listens on, tunnelled out. Nothing is | |
| 88 | 73 | # served unless the command asks for it -- a plain build exits and the tunnel |
| 89 | 74 | # closes with the sandbox -- but the port has to be declared at create time, |
| 90 | 75 | # so it is declared once here and `modal run --command` decides whether |
| @@ -94,28 +79,24 @@ ports = [8080] | ||
| 94 | 79 | |
| 95 | 80 | [run] |
| 96 | 81 | workdir = "/app" |
| 97 | -# Nix for the dependencies, the ordinary toolchain for the build -- the | |
| 98 | -# `flutter-dev` argument, unchanged: a derivation is all-or-nothing, so any | |
| 99 | -# edit under `nix build` is a fresh sandbox and a fresh compile of everything. | |
| 100 | -# Here the devShell supplies dart2js and the engine artifacts and `flutter | |
| 101 | -# build web` decides what is stale. | |
| 82 | +# Source in, toolchain out of the volume, build in place. Three steps, and | |
| 83 | +# none of them evaluates anything: the old command spent its first minutes | |
| 84 | +# entering a devShell, printing an environment, caching that environment | |
| 85 | +# against flake.lock and copying a nix closure back afterwards, all to arrive | |
| 86 | +# at a PATH. A PATH is what `tools/toolchain.sh env` prints, out of a | |
| 87 | +# directory that is already on the volume. | |
| 102 | 88 | command = """ |
| 103 | 89 | set -e |
| 104 | -# This container's own directory on the shared devshell volume, named for the | |
| 105 | -# devShell whose state it keeps. `flutter-desktop` has its own beside it. | |
| 106 | 90 | SHELL_DIR=/devshell/frq-flutter-web |
| 107 | -mkdir -p "$SHELL_DIR" "$SHELL_DIR/.cache" | |
| 108 | 91 | |
| 109 | -# A worktree's `.git` is a *file* naming a gitdir back on the machine that | |
| 110 | -# copied it in, and nix believes it and goes looking for a path that is not | |
| 111 | -# here. It has to go before any flake reference to /app. | |
| 112 | -rm -rf /app/.git | |
| 92 | +# Beside the working tree and NOT inside it: the rsync below runs with | |
| 93 | +# --delete, so anything under $SHELL_DIR that is not in /app is removed on | |
| 94 | +# every run. A cache kept in there would be deleted moments before it was | |
| 95 | +# consulted -- which is what happened to the last one that tried. | |
| 96 | +export FRQ_TOOLCHAIN=/devshell/frq-flutter-web.toolchain | |
| 97 | +mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN" | |
| 113 | 98 | |
| 114 | 99 | echo "sync: /app -> $SHELL_DIR" |
| 115 | -# `nix shell --command` and not `nix profile install`: a profile install puts | |
| 116 | -# rsync in ~/.nix-profile/bin, which is not on the PATH of the shell already | |
| 117 | -# running. | |
| 118 | -# | |
| 119 | 100 | # rsync and not cp, with --checksum and not mtimes: Modal copies the source in |
| 120 | 101 | # with fresh timestamps every run, so a plain copy looks entirely new to |
| 121 | 102 | # Flutter and rebuilds the lot. --checksum compares content and leaves the |
| @@ -123,115 +104,33 @@ echo "sync: /app -> $SHELL_DIR" | ||
| 123 | 104 | # incremental build. |
| 124 | 105 | # |
| 125 | 106 | # The excludes are the state we are here to keep -- overwriting them from /app |
| 126 | -# would defeat the volume. `flutter/web/` is NOT on the list: it is committed | |
| 127 | -# now, because the OAuth client keeps a script there, so it has to arrive from | |
| 107 | +# would defeat the volume. `flutter/web/` is NOT on the list: it is committed, | |
| 108 | +# because the OAuth client keeps a script there, so it has to arrive from | |
| 128 | 109 | # /app like any other source. |
| 129 | -nix shell nixpkgs#rsync --accept-flake-config \ | |
| 130 | - --extra-substituters file:///nix-cache --command \ | |
| 131 | - rsync -a --checksum --delete \ | |
| 110 | +rsync -a --checksum --delete \ | |
| 132 | 111 | --exclude 'flutter/.home/' \ |
| 133 | 112 | --exclude 'flutter/.clojuredart/' \ |
| 134 | 113 | --exclude 'flutter/build/' \ |
| 114 | + --exclude 'flutter/lib/cljd-out/' \ | |
| 135 | 115 | --exclude 'flutter/.dart_tool/' \ |
| 116 | + --exclude '.toolchain/' \ | |
| 136 | 117 | --exclude '.git' \ |
| 137 | 118 | /app/ "$SHELL_DIR/" |
| 138 | 119 | |
| 139 | 120 | cd "$SHELL_DIR" |
| 140 | -echo "state carried over:" | |
| 141 | -du -sh flutter/.home flutter/.clojuredart flutter/build 2>/dev/null \ | |
| 142 | - || echo " (none yet -- first run)" | |
| 121 | +# What survived from the last run, by presence and not by size: `du` here | |
| 122 | +# walked the pub cache, the toolchain and every object of the last build over | |
| 123 | +# a network volume, for numbers nobody acts on. | |
| 124 | +for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/build; do | |
| 125 | + [ -d "$d" ] && echo " carried over: $d" | |
| 126 | +done | |
| 143 | 127 | |
| 144 | -# Evaluated from /app and built in the volume, as `flutter-dev` does and for | |
| 145 | -# the same two reasons: /app is the pristine copy, so nix stores a source tree | |
| 146 | -# of the repo rather than one carrying gigabytes of flutter/build, while the | |
| 147 | -# recipe still runs where the state it reuses lives. `just -f` is what puts it | |
| 148 | -# there, since the recipe cds to its own justfile's directory. | |
| 149 | -# The baked devShell if there is one, and `nix develop` if there is not. | |
| 150 | -# | |
| 151 | -# This is the difference between a two-minute rebuild and a three-minute one. | |
| 152 | -# `nix develop /app#flutter-web` re-copies the whole repo into the nix store | |
| 153 | -# and re-evaluates the flake on every run -- the flake's source is the tree, | |
| 154 | -# so any edit makes it a new source -- and all of that to arrive at an | |
| 155 | -# environment the image already computed with `print-dev-env` and wrote to | |
| 156 | -# /etc/devshell.sh. Sourcing it is the same PATH and the same variables with | |
| 157 | -# no evaluation at all. | |
| 158 | -# | |
| 159 | -# The fallback is not decoration: that setup step is best-effort, because | |
| 160 | -# realising the devShell under gVisor can fail (see [build] setup), and a | |
| 161 | -# container whose bake did not happen still has to build. | |
| 162 | -# The devShell environment, computed once and kept on the volume. | |
| 163 | -# | |
| 164 | -# `nix develop /app#flutter-web` re-copies the whole repo into the nix store | |
| 165 | -# and re-evaluates the flake on every run -- the flake's source IS the tree, | |
| 166 | -# so any edit makes it a new source -- to arrive at an environment that has | |
| 167 | -# not changed. `print-dev-env` writes that environment out as shell, and | |
| 168 | -# sourcing it is the same PATH and the same variables with no evaluation. | |
| 169 | -# | |
| 170 | -# Here and not in [build] setup, which is where it used to be: an image build | |
| 171 | -# is a Function under gVisor, where nix cannot realise a derivation, and this | |
| 172 | -# devShell's closure contains one no cache can answer for | |
| 173 | -# (flutter-wrapped-...-sdk-links). That bake failed every time and was made | |
| 174 | -# non-fatal, which meant it silently never happened. On the VM it works, and | |
| 175 | -# the volume is what makes it worth doing once. | |
| 176 | -# | |
| 177 | -# Regenerated when flake.lock's CONTENT changes -- by hash and never by mtime. | |
| 178 | -# Modal copies /app in with fresh timestamps on every run, so `-nt` says the | |
| 179 | -# lock is newer every single time and the cache never hits. That is the same | |
| 180 | -# trap the rsync above documents and works around with --checksum; it catches | |
| 181 | -# anything here that asks a file when it changed. | |
| 182 | -# Beside the working tree and NOT inside it: the rsync above runs with | |
| 183 | -# --delete, so anything under $SHELL_DIR that is not in /app is removed on | |
| 184 | -# every run. Kept in there, this cache was deleted moments before it was | |
| 185 | -# consulted, which is why it recomputed every time while claiming to be a | |
| 186 | -# cache. | |
| 187 | -ENV_DIR="$SHELL_DIR.env" | |
| 188 | -mkdir -p "$ENV_DIR" | |
| 189 | -ENV_SH="$ENV_DIR/devshell.sh" | |
| 190 | -STAMP="$ENV_DIR/devshell.lock" | |
| 191 | -WANT="$(sha256sum /app/flake.lock | cut -d' ' -f1)" | |
| 192 | -if [ ! -s "$ENV_SH" ] || [ "$(cat "$STAMP" 2>/dev/null)" != "$WANT" ]; then | |
| 193 | - echo "devshell: computing (first run, or flake.lock changed)" | |
| 194 | - nix print-dev-env /app#flutter-web --accept-flake-config \ | |
| 195 | - --extra-substituters file:///nix-cache > "$ENV_SH.tmp" | |
| 196 | - mv "$ENV_SH.tmp" "$ENV_SH" | |
| 197 | - echo "$WANT" > "$STAMP" | |
| 198 | -else | |
| 199 | - echo "devshell: reusing the computed env" | |
| 200 | -fi | |
| 201 | - | |
| 202 | -# A subshell, so what the env sets does not leak into the `nix copy` below -- | |
| 203 | -# that wants the container's own nix, not the shell's. `set +u` because a | |
| 204 | -# printed dev env references variables that need not be set. | |
| 205 | -( set +u; . "$ENV_SH"; set -u | |
| 206 | - just -f "$SHELL_DIR/justfile" flutter-web ) | |
| 207 | - | |
| 208 | -echo "built:" | |
| 209 | -du -sh flutter/build/web | |
| 210 | - | |
| 211 | -# The devShell's closure is gigabytes of Flutter and Dart, and the store it | |
| 212 | -# landed in belongs to the image rather than to a volume -- so without this | |
| 213 | -# every run re-fetches it from upstream. Written back, the next run | |
| 214 | -# substitutes it from file:///nix-cache instead. | |
| 215 | -if [ -f /nix-cache/nix-cache-info ]; then | |
| 216 | - echo "cache: writing the devShell closure back" | |
| 217 | - nix copy --no-check-sigs --all --to file:///nix-cache | |
| 218 | -fi | |
| 128 | +# The same script `just flutter-web` runs, with the same single build mode: | |
| 129 | +# what is served and what a laptop compiles are the same bundle. | |
| 130 | +tools/build-web.sh build | |
| 219 | 131 | """ |
| 220 | -# Nix's own cache, on the volume rather than in the container. Without it | |
| 221 | -# every Sandbox starts empty and `nix develop` re-clones the flake's git | |
| 222 | -# inputs, because flake.lock pins which revision to fetch and not whether it | |
| 223 | -# is already on disk. Set here rather than in the command so an interactive | |
| 224 | -# shell into this container gets it too. | |
| 225 | -env = { XDG_CACHE_HOME = "/devshell/frq-flutter-web/.cache" } | |
| 226 | - | |
| 227 | -[nix] | |
| 228 | -# Every nix command in the container reads the mounted cache, including one | |
| 229 | -# typed by hand in a shell. | |
| 230 | -substituters = ["file:///nix-cache"] | |
| 231 | -# No devShell warming at image build time: this enters `nix develop` at run | |
| 232 | -# time, on the VM, where the cache answers for its closure. The ptyshim that | |
| 233 | -# warming would need under gVisor is deprecated and does not come back. | |
| 234 | -flake = false | |
| 235 | -shim = false | |
| 236 | - | |
| 237 | -# [experimental] overrides the sandbox default of vm_runtime = true. | |
| 132 | +# Flutter keeps its settings -- `--enable-web` among them -- under | |
| 133 | +# XDG_CONFIG_HOME, and its own caches under XDG_CACHE_HOME. Both point into | |
| 134 | +# the volume so a second run finds what the first one decided. Set here | |
| 135 | +# rather than in the command so a shell into this container gets them too. | |
| 136 | +env = { XDG_CACHE_HOME = "/devshell/frq-flutter-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-flutter-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-flutter-web.toolchain" } | |
| @@ -1,7 +1,11 @@ | |||
| 1 | [container] | 1 | [container] |
| 2 | name = "frq-flutter-web" | 2 | name = "frq-flutter-web" |
| 3 | -description = "the Flutter web build, incremental, in a nix devShell" | 3 | +description = "the Flutter web build, incremental, from a pinned toolchain" |
| 4 | -base = "arch-nix" | 4 | +# `debian:13-slim` and not `arch-nix`: there is no nix in this container any |
| 5 | +# more. The build is `tools/build-web.sh`, which fetches its own Flutter, JDK | ||
| 6 | +# and Clojure CLI by pinned sha256, so what the image owes it is curl, git, | ||
| 7 | +# tar and a C runtime — and the smallest image that has them is the right one. | ||
| 8 | +registry = "debian:13-slim" | ||
| 5 | # A Sandbox, not a Function: it runs on a real VM, the command is the | 9 | # A Sandbox, not a Function: it runs on a real VM, the command is the |
| 6 | # sandbox's own process so it dies when the command does, and only a Sandbox | 10 | # sandbox's own process so it dies when the command does, and only a Sandbox |
| 7 | # can hold open a tunnel -- which is the whole of `serve`. | 11 | # can hold open a tunnel -- which is the whole of `serve`. |
| @@ -12,48 +16,18 @@ runtime = "sandbox" | |||
| 12 | # levels up and `.` is the whole tree. | 16 | # levels up and `.` is the whole tree. |
| 13 | context = "../.." | 17 | context = "../.." |
| 14 | include = ["."] | 18 | include = ["."] |
| 15 | -# The devShell, baked in rather than entered -- `flutter-dev`'s trick and for | 19 | +# The whole image build, and it is one apt line. What used to be here -- a |
| 16 | -# its reasons. `print-dev-env` writes the whole environment out as shell and | 20 | +# nix store to populate, a devShell to print, a closure to warm before the |
| 17 | -# realises its inputs on the way, so the closure becomes an image layer | 21 | +# source arrived so an edit would not invalidate it -- is gone with the nix |
| 18 | -# instead of a fetch every container pays for; sourcing it from .bashrc means | 22 | +# it was for. There is no `warm` list any more either: this step reads |
| 19 | -# a shell attached to this container *is* the devShell. | 23 | +# nothing out of the tree, so nothing in the tree can invalidate it. |
| 20 | # | 24 | # |
| 21 | -# `dev` stays for the case where the baked env is stale against a flake edit. | 25 | +# git, because Flutter shells out to it against its own SDK checkout and |
| 22 | -# The toolchain layer, and what it is allowed to depend on. | 26 | +# refuses to run without one; unzip and xz-utils, because that is what the |
| 23 | -# | 27 | +# SDK and the toolchain tarballs arrive as; rsync for the sync below; |
| 24 | -# `warm` is copied before `setup` runs and is deliberately six files: the | 28 | +# ca-certificates so curl can verify what it fetches. |
| 25 | -# flake and its lock, plus the four `cljd-deps` actually reads -- it does | ||
| 26 | -# `cp ${./common/deps.edn}`, `${./flutter/deps.edn}`, `${./flutter/pubspec.yaml}` | ||
| 27 | -# and `${./flutter/pubspec.lock}` and nothing else. That is the whole of what | ||
| 28 | -# `nix develop .#flutter-web` needs to evaluate, so this layer moves when a | ||
| 29 | -# dependency moves and not when a line of ClojureDart does. | ||
| 30 | -# | ||
| 31 | -# The point of the split: `[build] commands` run after the full source copy, | ||
| 32 | -# so editing `flutter/src/frq/net/web.cljd` used to invalidate them and spend | ||
| 33 | -# minutes re-warming a devShell that had not changed. Here the image is built | ||
| 34 | -# from the toolchain and the program is built in the sandbox, which is where | ||
| 35 | -# it was always going to happen anyway. | ||
| 36 | -warm = [ | ||
| 37 | - "flake.nix", "flake.lock", | ||
| 38 | - "common/deps.edn", | ||
| 39 | - "flutter/deps.edn", "flutter/pubspec.yaml", "flutter/pubspec.lock", | ||
| 40 | -] | ||
| 41 | -# The devShell, baked in rather than entered. `print-dev-env` writes the whole | ||
| 42 | -# environment out as shell and realises its inputs on the way, so the closure | ||
| 43 | -# becomes an image layer instead of a fetch every container pays for; sourcing | ||
| 44 | -# it from .bashrc means a shell attached to this container *is* the devShell. | ||
| 45 | -# | ||
| 46 | -# Best-effort, and the reason is gVisor. An image build is a Function | ||
| 47 | -# underneath, so it cannot *build* a derivation -- and `print-dev-env` realises | ||
| 48 | -# the shell's inputs, which here includes `flutter-wrapped-…-sdk-links.drv`, | ||
| 49 | -# in no binary cache and therefore built. Under gVisor that dies with | ||
| 50 | -# `unexpected EOF reading a line`, and the ptyshim that papered over it is | ||
| 51 | -# deprecated. Nothing here is load-bearing: [run] enters `nix develop` itself, | ||
| 52 | -# on the VM, where a real pty makes the same build work. | ||
| 53 | setup = [ | 29 | setup = [ |
| 54 | - "nix print-dev-env /app#flutter-web --accept-flake-config --extra-substituters file:///nix-cache > /etc/devshell.sh || rm -f /etc/devshell.sh", | 30 | + "apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git rsync tar unzip xz-utils && rm -rf /var/lib/apt/lists/*", |
| 55 | - "echo '[ -s /etc/devshell.sh ] && . /etc/devshell.sh' >> /root/.bashrc", | ||
| 56 | - "printf '#!/bin/sh\\nexec nix develop /app#flutter-web \"$@\"\\n' > /usr/local/bin/dev && chmod +x /usr/local/bin/dev", | ||
| 57 | ] | 31 | ] |
| 58 | # The build state a local checkout carries, wanted by nothing out here: this | 32 | # The build state a local checkout carries, wanted by nothing out here: this |
| 59 | # container builds into a volume of its own, and the jolt and clojure caches | 33 | # container builds into a volume of its own, and the jolt and clojure caches |
| @@ -61,6 +35,18 @@ setup = [ | |||
| 61 | ignore = [ | 35 | ignore = [ |
| 62 | "flutter/build", "flutter/.home", "flutter/.dart_tool", | 36 | "flutter/build", "flutter/.home", "flutter/.dart_tool", |
| 63 | "flutter/.clojuredart", "flutter/.cpcache", | 37 | "flutter/.clojuredart", "flutter/.cpcache", |
| 38 | + # The ClojureDart compiler's output, gitignored and -- until now -- | ||
| 39 | + # uploaded anyway, because this list is explicit and does not read | ||
| 40 | + # .gitignore. It is megabytes of generated Dart per run, and worse than | ||
| 41 | + # the upload is what happened on arrival: the rsync below overwrote the | ||
| 42 | + # volume's copy, the one the last container compiled, with a laptop's. | ||
| 43 | + # Every file whose content differed then looked new to the compiler and | ||
| 44 | + # to Flutter, which is the incremental build undone by the thing that was | ||
| 45 | + # meant to feed it. It belongs to the volume, like flutter/build. | ||
| 46 | + "flutter/lib/cljd-out", | ||
| 47 | + # The toolchain, which is a gigabyte of Flutter SDK and lives on the | ||
| 48 | + # volume out here. | ||
| 49 | + ".toolchain", | ||
| 64 | ".jolt", ".cpcache", "result", "build", ".git", | 50 | ".jolt", ".cpcache", "result", "build", ".git", |
| 65 | # An editor's linter rewrites this while the upload is reading it, and | 51 | # An editor's linter rewrites this while the upload is reading it, and |
| 66 | # Modal fails the whole run with "was modified during build process". | 52 | # Modal fails the whole run with "was modified during build process". |
| @@ -68,23 +54,22 @@ ignore = [ | |||
| 68 | ".clj-kondo", | 54 | ".clj-kondo", |
| 69 | ] | 55 | ] |
| 70 | 56 | ||
| 71 | -# `nix-cache` is the binary cache every container here reads from and writes | 57 | +# One volume now, where there were two: the nix binary cache went with nix. |
| 72 | -# back to. `devshell` is the working state of a `nix develop` loop, shared by | 58 | +# `devshell` is the working state of an incremental loop, shared by every |
| 73 | -# every container that has one -- each gets its own directory under it, named | 59 | +# container that has one -- each gets its own directory under it, named for |
| 74 | -# for the devShell it belongs to, so `flutter-web` and `flutter-desktop` never | 60 | +# what it belongs to, so `flutter-web` and `flutter-desktop` never write the |
| 75 | -# write the same tree even though they share a `.home`-shaped cache layout. | 61 | +# same tree. Modal Volumes have no locking, so those directory names are the |
| 76 | -# Modal Volumes have no locking, so those directory names are the only thing | 62 | +# only thing keeping them apart, and two runs of the *same* container must |
| 77 | -# keeping them apart, and two runs of the *same* devshell must not overlap. | 63 | +# not overlap. |
| 78 | [volumes] | 64 | [volumes] |
| 79 | -nix-cache = "/nix-cache" | 65 | +devshell = "/devshell" |
| 80 | -devshell = "/devshell" | ||
| 81 | 66 | ||
| 82 | [resources] | 67 | [resources] |
| 83 | cpu = 8 | 68 | cpu = 8 |
| 84 | memory = 16384 | 69 | memory = 16384 |
| 85 | timeout = 3600 | 70 | timeout = 3600 |
| 86 | 71 | ||
| 87 | -# The port `just flutter-web serve` listens on, tunnelled out. Nothing is | 72 | +# The port `tools/build-web.sh serve` listens on, tunnelled out. Nothing is |
| 88 | # served unless the command asks for it -- a plain build exits and the tunnel | 73 | # served unless the command asks for it -- a plain build exits and the tunnel |
| 89 | # closes with the sandbox -- but the port has to be declared at create time, | 74 | # closes with the sandbox -- but the port has to be declared at create time, |
| 90 | # so it is declared once here and `modal run --command` decides whether | 75 | # so it is declared once here and `modal run --command` decides whether |
| @@ -94,28 +79,24 @@ ports = [8080] | |||
| 94 | 79 | ||
| 95 | [run] | 80 | [run] |
| 96 | workdir = "/app" | 81 | workdir = "/app" |
| 97 | -# Nix for the dependencies, the ordinary toolchain for the build -- the | 82 | +# Source in, toolchain out of the volume, build in place. Three steps, and |
| 98 | -# `flutter-dev` argument, unchanged: a derivation is all-or-nothing, so any | 83 | +# none of them evaluates anything: the old command spent its first minutes |
| 99 | -# edit under `nix build` is a fresh sandbox and a fresh compile of everything. | 84 | +# entering a devShell, printing an environment, caching that environment |
| 100 | -# Here the devShell supplies dart2js and the engine artifacts and `flutter | 85 | +# against flake.lock and copying a nix closure back afterwards, all to arrive |
| 101 | -# build web` decides what is stale. | 86 | +# at a PATH. A PATH is what `tools/toolchain.sh env` prints, out of a |
| 87 | +# directory that is already on the volume. | ||
| 102 | command = """ | 88 | command = """ |
| 103 | set -e | 89 | set -e |
| 104 | -# This container's own directory on the shared devshell volume, named for the | ||
| 105 | -# devShell whose state it keeps. `flutter-desktop` has its own beside it. | ||
| 106 | SHELL_DIR=/devshell/frq-flutter-web | 90 | SHELL_DIR=/devshell/frq-flutter-web |
| 107 | -mkdir -p "$SHELL_DIR" "$SHELL_DIR/.cache" | ||
| 108 | 91 | ||
| 109 | -# A worktree's `.git` is a *file* naming a gitdir back on the machine that | 92 | +# Beside the working tree and NOT inside it: the rsync below runs with |
| 110 | -# copied it in, and nix believes it and goes looking for a path that is not | 93 | +# --delete, so anything under $SHELL_DIR that is not in /app is removed on |
| 111 | -# here. It has to go before any flake reference to /app. | 94 | +# every run. A cache kept in there would be deleted moments before it was |
| 112 | -rm -rf /app/.git | 95 | +# consulted -- which is what happened to the last one that tried. |
| 96 | +export FRQ_TOOLCHAIN=/devshell/frq-flutter-web.toolchain | ||
| 97 | +mkdir -p "$SHELL_DIR" "$FRQ_TOOLCHAIN" | ||
| 113 | 98 | ||
| 114 | echo "sync: /app -> $SHELL_DIR" | 99 | echo "sync: /app -> $SHELL_DIR" |
| 115 | -# `nix shell --command` and not `nix profile install`: a profile install puts | ||
| 116 | -# rsync in ~/.nix-profile/bin, which is not on the PATH of the shell already | ||
| 117 | -# running. | ||
| 118 | -# | ||
| 119 | # rsync and not cp, with --checksum and not mtimes: Modal copies the source in | 100 | # rsync and not cp, with --checksum and not mtimes: Modal copies the source in |
| 120 | # with fresh timestamps every run, so a plain copy looks entirely new to | 101 | # with fresh timestamps every run, so a plain copy looks entirely new to |
| 121 | # Flutter and rebuilds the lot. --checksum compares content and leaves the | 102 | # Flutter and rebuilds the lot. --checksum compares content and leaves the |
| @@ -123,115 +104,33 @@ echo "sync: /app -> $SHELL_DIR" | |||
| 123 | # incremental build. | 104 | # incremental build. |
| 124 | # | 105 | # |
| 125 | # The excludes are the state we are here to keep -- overwriting them from /app | 106 | # The excludes are the state we are here to keep -- overwriting them from /app |
| 126 | -# would defeat the volume. `flutter/web/` is NOT on the list: it is committed | 107 | +# would defeat the volume. `flutter/web/` is NOT on the list: it is committed, |
| 127 | -# now, because the OAuth client keeps a script there, so it has to arrive from | 108 | +# because the OAuth client keeps a script there, so it has to arrive from |
| 128 | # /app like any other source. | 109 | # /app like any other source. |
| 129 | -nix shell nixpkgs#rsync --accept-flake-config \ | 110 | +rsync -a --checksum --delete \ |
| 130 | - --extra-substituters file:///nix-cache --command \ | ||
| 131 | - rsync -a --checksum --delete \ | ||
| 132 | --exclude 'flutter/.home/' \ | 111 | --exclude 'flutter/.home/' \ |
| 133 | --exclude 'flutter/.clojuredart/' \ | 112 | --exclude 'flutter/.clojuredart/' \ |
| 134 | --exclude 'flutter/build/' \ | 113 | --exclude 'flutter/build/' \ |
| 114 | + --exclude 'flutter/lib/cljd-out/' \ | ||
| 135 | --exclude 'flutter/.dart_tool/' \ | 115 | --exclude 'flutter/.dart_tool/' \ |
| 116 | + --exclude '.toolchain/' \ | ||
| 136 | --exclude '.git' \ | 117 | --exclude '.git' \ |
| 137 | /app/ "$SHELL_DIR/" | 118 | /app/ "$SHELL_DIR/" |
| 138 | 119 | ||
| 139 | cd "$SHELL_DIR" | 120 | cd "$SHELL_DIR" |
| 140 | -echo "state carried over:" | 121 | +# What survived from the last run, by presence and not by size: `du` here |
| 141 | -du -sh flutter/.home flutter/.clojuredart flutter/build 2>/dev/null \ | 122 | +# walked the pub cache, the toolchain and every object of the last build over |
| 142 | - || echo " (none yet -- first run)" | 123 | +# a network volume, for numbers nobody acts on. |
| 124 | +for d in "$FRQ_TOOLCHAIN" flutter/.clojuredart flutter/lib/cljd-out flutter/build; do | ||
| 125 | + [ -d "$d" ] && echo " carried over: $d" | ||
| 126 | +done | ||
| 143 | 127 | ||
| 144 | -# Evaluated from /app and built in the volume, as `flutter-dev` does and for | 128 | +# The same script `just flutter-web` runs, with the same single build mode: |
| 145 | -# the same two reasons: /app is the pristine copy, so nix stores a source tree | 129 | +# what is served and what a laptop compiles are the same bundle. |
| 146 | -# of the repo rather than one carrying gigabytes of flutter/build, while the | 130 | +tools/build-web.sh build |
| 147 | -# recipe still runs where the state it reuses lives. `just -f` is what puts it | ||
| 148 | -# there, since the recipe cds to its own justfile's directory. | ||
| 149 | -# The baked devShell if there is one, and `nix develop` if there is not. | ||
| 150 | -# | ||
| 151 | -# This is the difference between a two-minute rebuild and a three-minute one. | ||
| 152 | -# `nix develop /app#flutter-web` re-copies the whole repo into the nix store | ||
| 153 | -# and re-evaluates the flake on every run -- the flake's source is the tree, | ||
| 154 | -# so any edit makes it a new source -- and all of that to arrive at an | ||
| 155 | -# environment the image already computed with `print-dev-env` and wrote to | ||
| 156 | -# /etc/devshell.sh. Sourcing it is the same PATH and the same variables with | ||
| 157 | -# no evaluation at all. | ||
| 158 | -# | ||
| 159 | -# The fallback is not decoration: that setup step is best-effort, because | ||
| 160 | -# realising the devShell under gVisor can fail (see [build] setup), and a | ||
| 161 | -# container whose bake did not happen still has to build. | ||
| 162 | -# The devShell environment, computed once and kept on the volume. | ||
| 163 | -# | ||
| 164 | -# `nix develop /app#flutter-web` re-copies the whole repo into the nix store | ||
| 165 | -# and re-evaluates the flake on every run -- the flake's source IS the tree, | ||
| 166 | -# so any edit makes it a new source -- to arrive at an environment that has | ||
| 167 | -# not changed. `print-dev-env` writes that environment out as shell, and | ||
| 168 | -# sourcing it is the same PATH and the same variables with no evaluation. | ||
| 169 | -# | ||
| 170 | -# Here and not in [build] setup, which is where it used to be: an image build | ||
| 171 | -# is a Function under gVisor, where nix cannot realise a derivation, and this | ||
| 172 | -# devShell's closure contains one no cache can answer for | ||
| 173 | -# (flutter-wrapped-...-sdk-links). That bake failed every time and was made | ||
| 174 | -# non-fatal, which meant it silently never happened. On the VM it works, and | ||
| 175 | -# the volume is what makes it worth doing once. | ||
| 176 | -# | ||
| 177 | -# Regenerated when flake.lock's CONTENT changes -- by hash and never by mtime. | ||
| 178 | -# Modal copies /app in with fresh timestamps on every run, so `-nt` says the | ||
| 179 | -# lock is newer every single time and the cache never hits. That is the same | ||
| 180 | -# trap the rsync above documents and works around with --checksum; it catches | ||
| 181 | -# anything here that asks a file when it changed. | ||
| 182 | -# Beside the working tree and NOT inside it: the rsync above runs with | ||
| 183 | -# --delete, so anything under $SHELL_DIR that is not in /app is removed on | ||
| 184 | -# every run. Kept in there, this cache was deleted moments before it was | ||
| 185 | -# consulted, which is why it recomputed every time while claiming to be a | ||
| 186 | -# cache. | ||
| 187 | -ENV_DIR="$SHELL_DIR.env" | ||
| 188 | -mkdir -p "$ENV_DIR" | ||
| 189 | -ENV_SH="$ENV_DIR/devshell.sh" | ||
| 190 | -STAMP="$ENV_DIR/devshell.lock" | ||
| 191 | -WANT="$(sha256sum /app/flake.lock | cut -d' ' -f1)" | ||
| 192 | -if [ ! -s "$ENV_SH" ] || [ "$(cat "$STAMP" 2>/dev/null)" != "$WANT" ]; then | ||
| 193 | - echo "devshell: computing (first run, or flake.lock changed)" | ||
| 194 | - nix print-dev-env /app#flutter-web --accept-flake-config \ | ||
| 195 | - --extra-substituters file:///nix-cache > "$ENV_SH.tmp" | ||
| 196 | - mv "$ENV_SH.tmp" "$ENV_SH" | ||
| 197 | - echo "$WANT" > "$STAMP" | ||
| 198 | -else | ||
| 199 | - echo "devshell: reusing the computed env" | ||
| 200 | -fi | ||
| 201 | - | ||
| 202 | -# A subshell, so what the env sets does not leak into the `nix copy` below -- | ||
| 203 | -# that wants the container's own nix, not the shell's. `set +u` because a | ||
| 204 | -# printed dev env references variables that need not be set. | ||
| 205 | -( set +u; . "$ENV_SH"; set -u | ||
| 206 | - just -f "$SHELL_DIR/justfile" flutter-web ) | ||
| 207 | - | ||
| 208 | -echo "built:" | ||
| 209 | -du -sh flutter/build/web | ||
| 210 | - | ||
| 211 | -# The devShell's closure is gigabytes of Flutter and Dart, and the store it | ||
| 212 | -# landed in belongs to the image rather than to a volume -- so without this | ||
| 213 | -# every run re-fetches it from upstream. Written back, the next run | ||
| 214 | -# substitutes it from file:///nix-cache instead. | ||
| 215 | -if [ -f /nix-cache/nix-cache-info ]; then | ||
| 216 | - echo "cache: writing the devShell closure back" | ||
| 217 | - nix copy --no-check-sigs --all --to file:///nix-cache | ||
| 218 | -fi | ||
| 219 | """ | 131 | """ |
| 220 | -# Nix's own cache, on the volume rather than in the container. Without it | 132 | +# Flutter keeps its settings -- `--enable-web` among them -- under |
| 221 | -# every Sandbox starts empty and `nix develop` re-clones the flake's git | 133 | +# XDG_CONFIG_HOME, and its own caches under XDG_CACHE_HOME. Both point into |
| 222 | -# inputs, because flake.lock pins which revision to fetch and not whether it | 134 | +# the volume so a second run finds what the first one decided. Set here |
| 223 | -# is already on disk. Set here rather than in the command so an interactive | 135 | +# rather than in the command so a shell into this container gets them too. |
| 224 | -# shell into this container gets it too. | 136 | +env = { XDG_CACHE_HOME = "/devshell/frq-flutter-web.toolchain/.cache", XDG_CONFIG_HOME = "/devshell/frq-flutter-web.toolchain/.config", FRQ_TOOLCHAIN = "/devshell/frq-flutter-web.toolchain" } |
| 225 | -env = { XDG_CACHE_HOME = "/devshell/frq-flutter-web/.cache" } | ||
| 226 | - | ||
| 227 | -[nix] | ||
| 228 | -# Every nix command in the container reads the mounted cache, including one | ||
| 229 | -# typed by hand in a shell. | ||
| 230 | -substituters = ["file:///nix-cache"] | ||
| 231 | -# No devShell warming at image build time: this enters `nix develop` at run | ||
| 232 | -# time, on the VM, where the cache answers for its closure. The ptyshim that | ||
| 233 | -# warming would need under gVisor is deprecated and does not come back. | ||
| 234 | -flake = false | ||
| 235 | -shim = false | ||
| 236 | - | ||
| 237 | -# [experimental] overrides the sandbox default of vm_runtime = true. | ||
modified
flake.nix +11 -36 | @@ -1043,43 +1043,18 @@ | ||
| 1043 | 1043 | }; |
| 1044 | 1044 | |
| 1045 | 1045 | # The third frontend, and the first that is not a window: the same |
| 1046 | - # ClojureDart half again, over Flutter's web target. `flutter build | |
| 1047 | - # web` compiles the generated Dart with dart2js and writes a | |
| 1048 | - # directory of HTML, JS and assets rather than an executable. | |
| 1046 | + # No `flutter-web` shell here any more. The web target was the one | |
| 1047 | + # that needed nothing of the host -- no JDK and no Android SDK as | |
| 1048 | + # the APK wants, no GTK and no C++ and no nixGL as the desktop one | |
| 1049 | + # does -- and a devShell whose only job is to hand over a Dart and | |
| 1050 | + # a JVM is a devShell that a pinned tarball can replace. It did: | |
| 1051 | + # `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by | |
| 1052 | + # sha256, `tools/build-web.sh` builds out of them, and | |
| 1053 | + # `.modal/flutter-web/` runs that same script on a plain Debian | |
| 1054 | + # image with no store to populate. | |
| 1049 | 1055 | # |
| 1050 | - # The thinnest of the three shells, because the web target is the | |
| 1051 | - # one that needs no host toolchain at all: no JDK and no SDK as the | |
| 1052 | - # APK wants, no GTK and no C++ as the desktop one does, and no nixGL | |
| 1053 | - # — the GL is the browser's problem and the browser is not ours. | |
| 1054 | - # mkShellNoCC says so: nothing here compiles C. | |
| 1055 | - # | |
| 1056 | - # python3 is not a build input. It is `just flutter-web serve`: the | |
| 1057 | - # output is a directory of static files and something has to hand it | |
| 1058 | - # over HTTP, and the alternative — `flutter run -d web-server` — | |
| 1059 | - # rebuilds rather than serving what was built, which is the wrong | |
| 1060 | - # half of the loop when the build already happened in a container. | |
| 1061 | - flutter-web = pkgs.mkShellNoCC { | |
| 1062 | - name = "frq-flutter-web"; | |
| 1063 | - | |
| 1064 | - packages = [ | |
| 1065 | - pkgs.clojure | |
| 1066 | - pkgs.flutter | |
| 1067 | - pkgs.just | |
| 1068 | - pkgs.git | |
| 1069 | - pkgs.python3 | |
| 1070 | - ]; | |
| 1071 | - | |
| 1072 | - # The `flutter` shell's caches, and deliberately the same ones for | |
| 1073 | - # the same reason `flutter-desktop` shares them: all three targets | |
| 1074 | - # are one `clojure -M:cljd compile` over one deps.edn, and a third | |
| 1075 | - # set of caches would be a third answer to what it resolved | |
| 1076 | - # against. | |
| 1077 | - FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}"; | |
| 1078 | - | |
| 1079 | - # The recipe's re-entry test, the way FRQ_FLUTTER_DESKTOP is the | |
| 1080 | - # desktop one's. | |
| 1081 | - FRQ_FLUTTER_WEB = "1"; | |
| 1082 | - }; | |
| 1056 | + # The other two shells stay. What they supply is a host toolchain, | |
| 1057 | + # which is exactly what nix is better at than a tarball. | |
| 1083 | 1058 | }); |
| 1084 | 1059 | |
| 1085 | 1060 | apps = forEachSystem (pkgs: { |
| @@ -1043,43 +1043,18 @@ | |||
| 1043 | }; | 1043 | }; |
| 1044 | 1044 | ||
| 1045 | # The third frontend, and the first that is not a window: the same | 1045 | # The third frontend, and the first that is not a window: the same |
| 1046 | - # ClojureDart half again, over Flutter's web target. `flutter build | 1046 | + # No `flutter-web` shell here any more. The web target was the one |
| 1047 | - # web` compiles the generated Dart with dart2js and writes a | 1047 | + # that needed nothing of the host -- no JDK and no Android SDK as |
| 1048 | - # directory of HTML, JS and assets rather than an executable. | 1048 | + # the APK wants, no GTK and no C++ and no nixGL as the desktop one |
| 1049 | + # does -- and a devShell whose only job is to hand over a Dart and | ||
| 1050 | + # a JVM is a devShell that a pinned tarball can replace. It did: | ||
| 1051 | + # `tools/toolchain.sh` fetches Flutter, a JDK and the Clojure CLI by | ||
| 1052 | + # sha256, `tools/build-web.sh` builds out of them, and | ||
| 1053 | + # `.modal/flutter-web/` runs that same script on a plain Debian | ||
| 1054 | + # image with no store to populate. | ||
| 1049 | # | 1055 | # |
| 1050 | - # The thinnest of the three shells, because the web target is the | 1056 | + # The other two shells stay. What they supply is a host toolchain, |
| 1051 | - # one that needs no host toolchain at all: no JDK and no SDK as the | 1057 | + # which is exactly what nix is better at than a tarball. |
| 1052 | - # APK wants, no GTK and no C++ as the desktop one does, and no nixGL | ||
| 1053 | - # — the GL is the browser's problem and the browser is not ours. | ||
| 1054 | - # mkShellNoCC says so: nothing here compiles C. | ||
| 1055 | - # | ||
| 1056 | - # python3 is not a build input. It is `just flutter-web serve`: the | ||
| 1057 | - # output is a directory of static files and something has to hand it | ||
| 1058 | - # over HTTP, and the alternative — `flutter run -d web-server` — | ||
| 1059 | - # rebuilds rather than serving what was built, which is the wrong | ||
| 1060 | - # half of the loop when the build already happened in a container. | ||
| 1061 | - flutter-web = pkgs.mkShellNoCC { | ||
| 1062 | - name = "frq-flutter-web"; | ||
| 1063 | - | ||
| 1064 | - packages = [ | ||
| 1065 | - pkgs.clojure | ||
| 1066 | - pkgs.flutter | ||
| 1067 | - pkgs.just | ||
| 1068 | - pkgs.git | ||
| 1069 | - pkgs.python3 | ||
| 1070 | - ]; | ||
| 1071 | - | ||
| 1072 | - # The `flutter` shell's caches, and deliberately the same ones for | ||
| 1073 | - # the same reason `flutter-desktop` shares them: all three targets | ||
| 1074 | - # are one `clojure -M:cljd compile` over one deps.edn, and a third | ||
| 1075 | - # set of caches would be a third answer to what it resolved | ||
| 1076 | - # against. | ||
| 1077 | - FRQ_CLJD_DEPS = "${self.packages.${pkgs.stdenv.hostPlatform.system}.cljd-deps}"; | ||
| 1078 | - | ||
| 1079 | - # The recipe's re-entry test, the way FRQ_FLUTTER_DESKTOP is the | ||
| 1080 | - # desktop one's. | ||
| 1081 | - FRQ_FLUTTER_WEB = "1"; | ||
| 1082 | - }; | ||
| 1083 | }); | 1058 | }); |
| 1084 | 1059 | ||
| 1085 | apps = forEachSystem (pkgs: { | 1060 | apps = forEachSystem (pkgs: { |
modified
justfile +21 -67 | @@ -427,10 +427,17 @@ flutter-desktop action="build": | ||
| 427 | 427 | # instead of its Linux one — dart2js instead of CMake and Ninja, and a |
| 428 | 428 | # directory of static files instead of a bundle with an executable in it. |
| 429 | 429 | # |
| 430 | -# Impure for the one reason the other two are and not the other: pub.dev | |
| 431 | -# resolution and Flutter's engine artifacts are network. There is no | |
| 432 | -# writable-SDK dance and no nixGL, because nothing here writes into the store | |
| 433 | -# and nothing here paints — the browser does both. | |
| 430 | +# And the one target with no nix in it. The other two need the host: a JDK | |
| 431 | +# and the Android SDK for `apk`, GTK and a C++ toolchain and nixGL for | |
| 432 | +# `flutter-desktop`. This one needs a Dart, a JVM and a browser, and the | |
| 433 | +# browser is not ours — so `tools/toolchain.sh` fetches the first two as | |
| 434 | +# pinned tarballs into `.toolchain/` and there is nothing left for a devShell | |
| 435 | +# to supply. That is what lets the container in `.modal/flutter-web/` drop | |
| 436 | +# its image build too: same script, same three pins, no store to populate. | |
| 437 | +# | |
| 438 | +# Impure for the reason the other two are: pub.dev resolution and Flutter's | |
| 439 | +# engine artifacts are network, and now the toolchain is as well — pinned by | |
| 440 | +# sha256, which is the reproducibility that was worth having out of the store. | |
| 434 | 441 | # |
| 435 | 442 | # The entry point is `frq.main-web`, not `frq.main`: path_provider has no web |
| 436 | 443 | # implementation, so the `getApplicationSupportDirectory` that `frq.main` |
| @@ -439,74 +446,21 @@ flutter-desktop action="build": | ||
| 439 | 446 | # nothing. `frq.net.dart` is still the socket half, so connecting will want a |
| 440 | 447 | # WebSocket before this does more than paint. |
| 441 | 448 | # |
| 449 | +# One build and no `--debug` variant, because there is nothing to gain from | |
| 450 | +# one: dart2js at -O1 measured 52.5s against the release build's 49.8s on the | |
| 451 | +# same source change here, so a second, larger bundle would buy noise. See | |
| 452 | +# `tools/build-web.sh`, which writes the numbers down. | |
| 453 | +# | |
| 442 | 454 | # just flutter-web build build/web |
| 443 | 455 | # just flutter-web serve build it and serve it on $PORT (8080) |
| 456 | +# just flutter-web serve 3000 ...on another port | |
| 444 | 457 | flutter-web action="build" port="8080": |
| 445 | 458 | #!/usr/bin/env bash |
| 446 | 459 | set -euo pipefail |
| 447 | - cd "{{justfile_directory()}}" | |
| 448 | - if [ -z "${FRQ_FLUTTER_WEB:-}" ]; then | |
| 449 | - exec {{nix}} develop .#flutter-web --max-jobs {{jobs}} \ | |
| 450 | - --command just flutter-web "$@" | |
| 451 | - fi | |
| 452 | - cd flutter | |
| 453 | - | |
| 454 | - # The same three caches `apk` and `flutter-desktop` seed, in the same | |
| 455 | - # place and out of the same flake output. `.home/` is `apk`'s directory by | |
| 456 | - # name and all three write only this into it: whichever recipe runs first | |
| 457 | - # pays for the copy and the other two find it warm. | |
| 458 | - # | |
| 459 | - # m2 and gitlibs are set here for the reason they are set there — the JVM | |
| 460 | - # reads user.home out of /etc/passwd, so neither follows HOME. | |
| 461 | - export PUB_CACHE="$PWD/.home/.pub-cache" | |
| 462 | - export GITLIBS="$PWD/.home/gitlibs" | |
| 463 | - m2="$PWD/.home/m2" | |
| 464 | - | |
| 465 | - seed() { | |
| 466 | - [ -e "$2" ] && return 0 | |
| 467 | - mkdir -p "$(dirname "$2")" | |
| 468 | - cp -r "$FRQ_CLJD_DEPS/$1" "$2" | |
| 469 | - chmod -R u+w "$2" | |
| 470 | - } | |
| 471 | - seed m2 "$m2" | |
| 472 | - seed gitlibs "$GITLIBS" | |
| 473 | - seed pub-cache "$PUB_CACHE" | |
| 474 | - seed clojuredart/cache "$PWD/.clojuredart/cache" | |
| 475 | - | |
| 476 | - # Resolved here rather than in cljd-deps, which was not allowed to name | |
| 477 | - # the store — see the same loop in `apk`. | |
| 478 | - for helper in .clojuredart/cache/*/cljd_helper; do | |
| 479 | - [ -d "$helper" ] || continue | |
| 480 | - [ -e "$helper/.dart_tool/package_config.json" ] && continue | |
| 481 | - ( cd "$helper" && flutter pub get --offline ) | |
| 482 | - done | |
| 483 | - | |
| 484 | - # The web target is off in a checkout created for Android and Linux. | |
| 485 | - # `flutter/web/` itself IS committed now, unlike the Android and Linux | |
| 486 | - # runners: the OAuth client needs a script of its own beside the bundle | |
| 487 | - # (see web/frq_dpop.js), and a directory `flutter create` regenerates is | |
| 488 | - # no place to keep one. | |
| 489 | - flutter config --enable-web >/dev/null || true | |
| 490 | - | |
| 491 | - # `frq.main-web` and not `frq.main`: the compile walks out from the | |
| 492 | - # namespace it is given, which is what keeps `dart:html` in the web build | |
| 493 | - # and out of the other two. `flutter/lib/main_web.dart` is the one-line | |
| 494 | - # export beside the generated `main.dart` that -t points at. | |
| 495 | - clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile frq.main-web | |
| 496 | - flutter build web -t lib/main_web.dart | |
| 497 | - | |
| 498 | - case "{{action}}" in | |
| 499 | - build) echo "built $PWD/build/web" ;; | |
| 500 | - serve) | |
| 501 | - echo "serving $PWD/build/web on :{{port}}" | |
| 502 | - # --bind 0.0.0.0 and not the default loopback: in the container | |
| 503 | - # this is behind a Modal tunnel, and a server bound to 127.0.0.1 | |
| 504 | - # is one the tunnel cannot reach. | |
| 505 | - exec python3 -m http.server {{port}} --bind 0.0.0.0 \ | |
| 506 | - --directory build/web | |
| 507 | - ;; | |
| 508 | - *) echo "usage: just flutter-web [build|serve]" >&2; exit 1 ;; | |
| 509 | - esac | |
| 460 | + # A wrapper and nothing else. The build is a shell script because the | |
| 461 | + # container runs it too, and a container that had to install `just` to | |
| 462 | + # start would be one dependency away from the point. | |
| 463 | + exec "{{justfile_directory()}}/tools/build-web.sh" {{action}} {{port}} | |
| 510 | 464 | |
| 511 | 465 | # The containers in `.modal/`, run on Modal rather than here. This machine |
| 512 | 466 | # evaluates and Modal builds — see CLAUDE.md, which says so rather more |
| @@ -427,10 +427,17 @@ flutter-desktop action="build": | |||
| 427 | # instead of its Linux one — dart2js instead of CMake and Ninja, and a | 427 | # instead of its Linux one — dart2js instead of CMake and Ninja, and a |
| 428 | # directory of static files instead of a bundle with an executable in it. | 428 | # directory of static files instead of a bundle with an executable in it. |
| 429 | # | 429 | # |
| 430 | -# Impure for the one reason the other two are and not the other: pub.dev | 430 | +# And the one target with no nix in it. The other two need the host: a JDK |
| 431 | -# resolution and Flutter's engine artifacts are network. There is no | 431 | +# and the Android SDK for `apk`, GTK and a C++ toolchain and nixGL for |
| 432 | -# writable-SDK dance and no nixGL, because nothing here writes into the store | 432 | +# `flutter-desktop`. This one needs a Dart, a JVM and a browser, and the |
| 433 | -# and nothing here paints — the browser does both. | 433 | +# browser is not ours — so `tools/toolchain.sh` fetches the first two as |
| 434 | +# pinned tarballs into `.toolchain/` and there is nothing left for a devShell | ||
| 435 | +# to supply. That is what lets the container in `.modal/flutter-web/` drop | ||
| 436 | +# its image build too: same script, same three pins, no store to populate. | ||
| 437 | +# | ||
| 438 | +# Impure for the reason the other two are: pub.dev resolution and Flutter's | ||
| 439 | +# engine artifacts are network, and now the toolchain is as well — pinned by | ||
| 440 | +# sha256, which is the reproducibility that was worth having out of the store. | ||
| 434 | # | 441 | # |
| 435 | # The entry point is `frq.main-web`, not `frq.main`: path_provider has no web | 442 | # The entry point is `frq.main-web`, not `frq.main`: path_provider has no web |
| 436 | # implementation, so the `getApplicationSupportDirectory` that `frq.main` | 443 | # implementation, so the `getApplicationSupportDirectory` that `frq.main` |
| @@ -439,74 +446,21 @@ flutter-desktop action="build": | |||
| 439 | # nothing. `frq.net.dart` is still the socket half, so connecting will want a | 446 | # nothing. `frq.net.dart` is still the socket half, so connecting will want a |
| 440 | # WebSocket before this does more than paint. | 447 | # WebSocket before this does more than paint. |
| 441 | # | 448 | # |
| 449 | +# One build and no `--debug` variant, because there is nothing to gain from | ||
| 450 | +# one: dart2js at -O1 measured 52.5s against the release build's 49.8s on the | ||
| 451 | +# same source change here, so a second, larger bundle would buy noise. See | ||
| 452 | +# `tools/build-web.sh`, which writes the numbers down. | ||
| 453 | +# | ||
| 442 | # just flutter-web build build/web | 454 | # just flutter-web build build/web |
| 443 | # just flutter-web serve build it and serve it on $PORT (8080) | 455 | # just flutter-web serve build it and serve it on $PORT (8080) |
| 456 | +# just flutter-web serve 3000 ...on another port | ||
| 444 | flutter-web action="build" port="8080": | 457 | flutter-web action="build" port="8080": |
| 445 | #!/usr/bin/env bash | 458 | #!/usr/bin/env bash |
| 446 | set -euo pipefail | 459 | set -euo pipefail |
| 447 | - cd "{{justfile_directory()}}" | 460 | + # A wrapper and nothing else. The build is a shell script because the |
| 448 | - if [ -z "${FRQ_FLUTTER_WEB:-}" ]; then | 461 | + # container runs it too, and a container that had to install `just` to |
| 449 | - exec {{nix}} develop .#flutter-web --max-jobs {{jobs}} \ | 462 | + # start would be one dependency away from the point. |
| 450 | - --command just flutter-web "$@" | 463 | + exec "{{justfile_directory()}}/tools/build-web.sh" {{action}} {{port}} |
| 451 | - fi | ||
| 452 | - cd flutter | ||
| 453 | - | ||
| 454 | - # The same three caches `apk` and `flutter-desktop` seed, in the same | ||
| 455 | - # place and out of the same flake output. `.home/` is `apk`'s directory by | ||
| 456 | - # name and all three write only this into it: whichever recipe runs first | ||
| 457 | - # pays for the copy and the other two find it warm. | ||
| 458 | - # | ||
| 459 | - # m2 and gitlibs are set here for the reason they are set there — the JVM | ||
| 460 | - # reads user.home out of /etc/passwd, so neither follows HOME. | ||
| 461 | - export PUB_CACHE="$PWD/.home/.pub-cache" | ||
| 462 | - export GITLIBS="$PWD/.home/gitlibs" | ||
| 463 | - m2="$PWD/.home/m2" | ||
| 464 | - | ||
| 465 | - seed() { | ||
| 466 | - [ -e "$2" ] && return 0 | ||
| 467 | - mkdir -p "$(dirname "$2")" | ||
| 468 | - cp -r "$FRQ_CLJD_DEPS/$1" "$2" | ||
| 469 | - chmod -R u+w "$2" | ||
| 470 | - } | ||
| 471 | - seed m2 "$m2" | ||
| 472 | - seed gitlibs "$GITLIBS" | ||
| 473 | - seed pub-cache "$PUB_CACHE" | ||
| 474 | - seed clojuredart/cache "$PWD/.clojuredart/cache" | ||
| 475 | - | ||
| 476 | - # Resolved here rather than in cljd-deps, which was not allowed to name | ||
| 477 | - # the store — see the same loop in `apk`. | ||
| 478 | - for helper in .clojuredart/cache/*/cljd_helper; do | ||
| 479 | - [ -d "$helper" ] || continue | ||
| 480 | - [ -e "$helper/.dart_tool/package_config.json" ] && continue | ||
| 481 | - ( cd "$helper" && flutter pub get --offline ) | ||
| 482 | - done | ||
| 483 | - | ||
| 484 | - # The web target is off in a checkout created for Android and Linux. | ||
| 485 | - # `flutter/web/` itself IS committed now, unlike the Android and Linux | ||
| 486 | - # runners: the OAuth client needs a script of its own beside the bundle | ||
| 487 | - # (see web/frq_dpop.js), and a directory `flutter create` regenerates is | ||
| 488 | - # no place to keep one. | ||
| 489 | - flutter config --enable-web >/dev/null || true | ||
| 490 | - | ||
| 491 | - # `frq.main-web` and not `frq.main`: the compile walks out from the | ||
| 492 | - # namespace it is given, which is what keeps `dart:html` in the web build | ||
| 493 | - # and out of the other two. `flutter/lib/main_web.dart` is the one-line | ||
| 494 | - # export beside the generated `main.dart` that -t points at. | ||
| 495 | - clojure -Sdeps "{:mvn/local-repo \"$m2\"}" -M:cljd compile frq.main-web | ||
| 496 | - flutter build web -t lib/main_web.dart | ||
| 497 | - | ||
| 498 | - case "{{action}}" in | ||
| 499 | - build) echo "built $PWD/build/web" ;; | ||
| 500 | - serve) | ||
| 501 | - echo "serving $PWD/build/web on :{{port}}" | ||
| 502 | - # --bind 0.0.0.0 and not the default loopback: in the container | ||
| 503 | - # this is behind a Modal tunnel, and a server bound to 127.0.0.1 | ||
| 504 | - # is one the tunnel cannot reach. | ||
| 505 | - exec python3 -m http.server {{port}} --bind 0.0.0.0 \ | ||
| 506 | - --directory build/web | ||
| 507 | - ;; | ||
| 508 | - *) echo "usage: just flutter-web [build|serve]" >&2; exit 1 ;; | ||
| 509 | - esac | ||
| 510 | 464 | ||
| 511 | # The containers in `.modal/`, run on Modal rather than here. This machine | 465 | # The containers in `.modal/`, run on Modal rather than here. This machine |
| 512 | # evaluates and Modal builds — see CLAUDE.md, which says so rather more | 466 | # evaluates and Modal builds — see CLAUDE.md, which says so rather more |
added
tools/build-web.sh +89 -0 | new file mode 100755 | ||
| @@ -0,0 +1,89 @@ | ||
| 1 | +#!/usr/bin/env bash | |
| 2 | +# The Flutter web build: one `clojure -M:cljd compile` over `flutter/src` and | |
| 3 | +# `common/`, then `flutter build web` over what that generated. | |
| 4 | +# | |
| 5 | +# A shell script and not a `just` recipe in a devShell, because this is the | |
| 6 | +# one target that needs nothing from the host — no JDK of the machine's, no | |
| 7 | +# GTK, no Android SDK, no nix. `tools/toolchain.sh` fetches the three tarballs | |
| 8 | +# it does need, and everything below runs out of `.toolchain/`. The container | |
| 9 | +# in `.modal/flutter-web/` runs this same file; `just flutter-web` is a | |
| 10 | +# wrapper around it. | |
| 11 | +# | |
| 12 | +# tools/build-web.sh build build/web | |
| 13 | +# tools/build-web.sh serve 8080 build it and serve it | |
| 14 | +# | |
| 15 | +# The entry point is `frq.main-web`, not `frq.main`: path_provider has no web | |
| 16 | +# implementation, so the `getApplicationSupportDirectory` that `frq.main` | |
| 17 | +# awaits throws MissingPluginException before any widget is built. The web | |
| 18 | +# entry installs `frq.io.web` — localStorage behind the same seam — and awaits | |
| 19 | +# nothing. | |
| 20 | +set -euo pipefail | |
| 21 | + | |
| 22 | +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | |
| 23 | +action="${1:-build}" | |
| 24 | +port="${2:-8080}" | |
| 25 | + | |
| 26 | +case "$action" in build|serve) ;; *) | |
| 27 | + echo "usage: build-web.sh [build|serve] [port]" >&2; exit 1 ;; | |
| 28 | +esac | |
| 29 | + | |
| 30 | +# One build, and it is the release one. There was a `fast` mode here for a | |
| 31 | +# while — dart2js at -O1, no icon tree-shaking, no service worker — on the | |
| 32 | +# theory that the edit-compile loop should not pay for the bundle it ships. | |
| 33 | +# Measured on this program it pays for almost nothing: -O1 came out at 52.5s | |
| 34 | +# against release's 49.8s on the same source change, because what dart2js | |
| 35 | +# spends its time on here is reading and linking the whole program, not | |
| 36 | +# optimising it. A second bundle, twice the surface to reason about and a | |
| 37 | +# megabyte more to serve, for noise. If a future dart2js changes that, the | |
| 38 | +# flags to reach for are `--optimization-level=1`, `--no-tree-shake-icons` | |
| 39 | +# and `--no-source-maps`, and the number to beat is written down above. | |
| 40 | +# | |
| 41 | +# `--no-wasm-dry-run` is the one that did pay: 52.0s against 55.7s, about 6%, | |
| 42 | +# for skipping a dry-run compile of the whole program against the wasm | |
| 43 | +# backend that can never succeed here. `frq.io.web`, `frq.net.web`, | |
| 44 | +# `frq.oauth.web` and three more are `dart:html`, which wasm does not | |
| 45 | +# support and which is the entire reason those namespaces exist. The build | |
| 46 | +# was being told, at whole-program cost, something the source already says. | |
| 47 | +build_flags=(--release --no-wasm-dry-run) | |
| 48 | + | |
| 49 | +eval "$("$root/tools/toolchain.sh" env)" | |
| 50 | +cd "$root/flutter" | |
| 51 | + | |
| 52 | +# The web target is off in a checkout created for Android and Linux. | |
| 53 | +# `flutter/web/` itself IS committed, unlike the Android and Linux runners: | |
| 54 | +# the OAuth client keeps a script of its own beside the bundle (see | |
| 55 | +# web/frq_dpop.js), and a directory `flutter create` regenerates is no place | |
| 56 | +# to keep one. | |
| 57 | +# | |
| 58 | +# Stamped, because `flutter config` is a Dart VM start and a settings-file | |
| 59 | +# rewrite for an answer that cannot change under us — this is the only thing | |
| 60 | +# that sets the flag, and the toolchain directory is already where this build | |
| 61 | +# remembers what it has done. | |
| 62 | +if [ ! -e "$FRQ_TOOLCHAIN/.web-enabled" ]; then | |
| 63 | + flutter config --enable-web >/dev/null || true | |
| 64 | + touch "$FRQ_TOOLCHAIN/.web-enabled" | |
| 65 | +fi | |
| 66 | + | |
| 67 | +# `frq.main-web` and not `frq.main`: the compile walks out from the namespace | |
| 68 | +# it is given, which is what keeps `dart:html` in the web build and out of the | |
| 69 | +# other two. `flutter/lib/main_web.dart` is the one-line export beside the | |
| 70 | +# generated `main.dart` that -t points at. | |
| 71 | +# | |
| 72 | +# `-Sdeps` with an explicit local repo rather than ~/.m2, for GITLIBS' reason: | |
| 73 | +# the JVM will not look where HOME says. | |
| 74 | +clojure -Sdeps "{:mvn/local-repo \"$FRQ_M2\"}" -M:cljd compile frq.main-web | |
| 75 | + | |
| 76 | +flutter build web -t lib/main_web.dart "${build_flags[@]}" | |
| 77 | +echo "built $PWD/build/web ($(du -sh build/web | cut -f1))" | |
| 78 | + | |
| 79 | +if [ "$action" = serve ]; then | |
| 80 | + echo "serving $PWD/build/web on :$port" | |
| 81 | + # Dart's own file server, out of the toolchain, because the toolchain is | |
| 82 | + # the whole dependency list: reaching for python3 here would put a fourth | |
| 83 | + # language on the list of things a machine must already have to serve a | |
| 84 | + # directory. | |
| 85 | + # | |
| 86 | + # --bind 0.0.0.0 and not loopback: in the container this is behind a Modal | |
| 87 | + # tunnel, and a server bound to 127.0.0.1 is one the tunnel cannot reach. | |
| 88 | + exec dart "$root/tools/serve-dir.dart" build/web "$port" | |
| 89 | +fi | |
| new file mode 100755 | |||
| @@ -0,0 +1,89 @@ | |||
| 1 | +#!/usr/bin/env bash | ||
| 2 | +# The Flutter web build: one `clojure -M:cljd compile` over `flutter/src` and | ||
| 3 | +# `common/`, then `flutter build web` over what that generated. | ||
| 4 | +# | ||
| 5 | +# A shell script and not a `just` recipe in a devShell, because this is the | ||
| 6 | +# one target that needs nothing from the host — no JDK of the machine's, no | ||
| 7 | +# GTK, no Android SDK, no nix. `tools/toolchain.sh` fetches the three tarballs | ||
| 8 | +# it does need, and everything below runs out of `.toolchain/`. The container | ||
| 9 | +# in `.modal/flutter-web/` runs this same file; `just flutter-web` is a | ||
| 10 | +# wrapper around it. | ||
| 11 | +# | ||
| 12 | +# tools/build-web.sh build build/web | ||
| 13 | +# tools/build-web.sh serve 8080 build it and serve it | ||
| 14 | +# | ||
| 15 | +# The entry point is `frq.main-web`, not `frq.main`: path_provider has no web | ||
| 16 | +# implementation, so the `getApplicationSupportDirectory` that `frq.main` | ||
| 17 | +# awaits throws MissingPluginException before any widget is built. The web | ||
| 18 | +# entry installs `frq.io.web` — localStorage behind the same seam — and awaits | ||
| 19 | +# nothing. | ||
| 20 | +set -euo pipefail | ||
| 21 | + | ||
| 22 | +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | ||
| 23 | +action="${1:-build}" | ||
| 24 | +port="${2:-8080}" | ||
| 25 | + | ||
| 26 | +case "$action" in build|serve) ;; *) | ||
| 27 | + echo "usage: build-web.sh [build|serve] [port]" >&2; exit 1 ;; | ||
| 28 | +esac | ||
| 29 | + | ||
| 30 | +# One build, and it is the release one. There was a `fast` mode here for a | ||
| 31 | +# while — dart2js at -O1, no icon tree-shaking, no service worker — on the | ||
| 32 | +# theory that the edit-compile loop should not pay for the bundle it ships. | ||
| 33 | +# Measured on this program it pays for almost nothing: -O1 came out at 52.5s | ||
| 34 | +# against release's 49.8s on the same source change, because what dart2js | ||
| 35 | +# spends its time on here is reading and linking the whole program, not | ||
| 36 | +# optimising it. A second bundle, twice the surface to reason about and a | ||
| 37 | +# megabyte more to serve, for noise. If a future dart2js changes that, the | ||
| 38 | +# flags to reach for are `--optimization-level=1`, `--no-tree-shake-icons` | ||
| 39 | +# and `--no-source-maps`, and the number to beat is written down above. | ||
| 40 | +# | ||
| 41 | +# `--no-wasm-dry-run` is the one that did pay: 52.0s against 55.7s, about 6%, | ||
| 42 | +# for skipping a dry-run compile of the whole program against the wasm | ||
| 43 | +# backend that can never succeed here. `frq.io.web`, `frq.net.web`, | ||
| 44 | +# `frq.oauth.web` and three more are `dart:html`, which wasm does not | ||
| 45 | +# support and which is the entire reason those namespaces exist. The build | ||
| 46 | +# was being told, at whole-program cost, something the source already says. | ||
| 47 | +build_flags=(--release --no-wasm-dry-run) | ||
| 48 | + | ||
| 49 | +eval "$("$root/tools/toolchain.sh" env)" | ||
| 50 | +cd "$root/flutter" | ||
| 51 | + | ||
| 52 | +# The web target is off in a checkout created for Android and Linux. | ||
| 53 | +# `flutter/web/` itself IS committed, unlike the Android and Linux runners: | ||
| 54 | +# the OAuth client keeps a script of its own beside the bundle (see | ||
| 55 | +# web/frq_dpop.js), and a directory `flutter create` regenerates is no place | ||
| 56 | +# to keep one. | ||
| 57 | +# | ||
| 58 | +# Stamped, because `flutter config` is a Dart VM start and a settings-file | ||
| 59 | +# rewrite for an answer that cannot change under us — this is the only thing | ||
| 60 | +# that sets the flag, and the toolchain directory is already where this build | ||
| 61 | +# remembers what it has done. | ||
| 62 | +if [ ! -e "$FRQ_TOOLCHAIN/.web-enabled" ]; then | ||
| 63 | + flutter config --enable-web >/dev/null || true | ||
| 64 | + touch "$FRQ_TOOLCHAIN/.web-enabled" | ||
| 65 | +fi | ||
| 66 | + | ||
| 67 | +# `frq.main-web` and not `frq.main`: the compile walks out from the namespace | ||
| 68 | +# it is given, which is what keeps `dart:html` in the web build and out of the | ||
| 69 | +# other two. `flutter/lib/main_web.dart` is the one-line export beside the | ||
| 70 | +# generated `main.dart` that -t points at. | ||
| 71 | +# | ||
| 72 | +# `-Sdeps` with an explicit local repo rather than ~/.m2, for GITLIBS' reason: | ||
| 73 | +# the JVM will not look where HOME says. | ||
| 74 | +clojure -Sdeps "{:mvn/local-repo \"$FRQ_M2\"}" -M:cljd compile frq.main-web | ||
| 75 | + | ||
| 76 | +flutter build web -t lib/main_web.dart "${build_flags[@]}" | ||
| 77 | +echo "built $PWD/build/web ($(du -sh build/web | cut -f1))" | ||
| 78 | + | ||
| 79 | +if [ "$action" = serve ]; then | ||
| 80 | + echo "serving $PWD/build/web on :$port" | ||
| 81 | + # Dart's own file server, out of the toolchain, because the toolchain is | ||
| 82 | + # the whole dependency list: reaching for python3 here would put a fourth | ||
| 83 | + # language on the list of things a machine must already have to serve a | ||
| 84 | + # directory. | ||
| 85 | + # | ||
| 86 | + # --bind 0.0.0.0 and not loopback: in the container this is behind a Modal | ||
| 87 | + # tunnel, and a server bound to 127.0.0.1 is one the tunnel cannot reach. | ||
| 88 | + exec dart "$root/tools/serve-dir.dart" build/web "$port" | ||
| 89 | +fi | ||
added
tools/serve-dir.dart +84 -0 | new file mode 100644 | ||
| @@ -0,0 +1,84 @@ | ||
| 1 | +// A static file server in one file, for `tools/build-web.sh serve`. | |
| 2 | +// | |
| 3 | +// `dart:io` and no packages, so there is nothing to `pub get` and nothing to | |
| 4 | +// pin: the toolchain already has a Dart, because Flutter ships one, and that | |
| 5 | +// is the whole reason this is Dart rather than the `python3 -m http.server` | |
| 6 | +// it replaces. Serving a directory should not add a language to the list of | |
| 7 | +// things a machine must have. | |
| 8 | +// | |
| 9 | +// It is the development server and says so: no caching headers, no | |
| 10 | +// compression, no range requests. `.modal/flutter-web/serve.py` is the one | |
| 11 | +// that faces a browser over the internet, and it has all three. | |
| 12 | +// | |
| 13 | +// dart tools/serve-dir.dart <directory> [port] | |
| 14 | +import 'dart:io'; | |
| 15 | + | |
| 16 | +// Enough of them for a Flutter web bundle, which is HTML, JavaScript, JSON, a | |
| 17 | +// wasm blob, fonts and images. Anything else goes out as bytes. | |
| 18 | +const _types = <String, String>{ | |
| 19 | + '.html': 'text/html; charset=utf-8', | |
| 20 | + '.js': 'application/javascript; charset=utf-8', | |
| 21 | + '.mjs': 'application/javascript; charset=utf-8', | |
| 22 | + '.json': 'application/json; charset=utf-8', | |
| 23 | + '.css': 'text/css; charset=utf-8', | |
| 24 | + '.wasm': 'application/wasm', | |
| 25 | + '.png': 'image/png', | |
| 26 | + '.jpg': 'image/jpeg', | |
| 27 | + '.jpeg': 'image/jpeg', | |
| 28 | + '.gif': 'image/gif', | |
| 29 | + '.svg': 'image/svg+xml', | |
| 30 | + '.ico': 'image/x-icon', | |
| 31 | + '.ttf': 'font/ttf', | |
| 32 | + '.otf': 'font/otf', | |
| 33 | + '.woff': 'font/woff', | |
| 34 | + '.woff2': 'font/woff2', | |
| 35 | + '.map': 'application/json; charset=utf-8', | |
| 36 | +}; | |
| 37 | + | |
| 38 | +String _typeOf(String path) { | |
| 39 | + final dot = path.lastIndexOf('.'); | |
| 40 | + if (dot < 0) return 'application/octet-stream'; | |
| 41 | + return _types[path.substring(dot).toLowerCase()] ?? 'application/octet-stream'; | |
| 42 | +} | |
| 43 | + | |
| 44 | +Future<void> main(List<String> args) async { | |
| 45 | + if (args.isEmpty) { | |
| 46 | + stderr.writeln('usage: dart serve-dir.dart <directory> [port]'); | |
| 47 | + exit(2); | |
| 48 | + } | |
| 49 | + final root = Directory(args[0]).absolute; | |
| 50 | + final port = args.length > 1 ? int.parse(args[1]) : 8080; | |
| 51 | + | |
| 52 | + // 0.0.0.0 and not loopback: in the container this is behind a Modal tunnel, | |
| 53 | + // and a server bound to 127.0.0.1 is one the tunnel cannot reach. | |
| 54 | + final server = await HttpServer.bind(InternetAddress.anyIPv4, port); | |
| 55 | + stdout.writeln('serving ${root.path} on :$port'); | |
| 56 | + | |
| 57 | + await for (final request in server) { | |
| 58 | + var path = Uri.decodeComponent(request.uri.path); | |
| 59 | + if (path.endsWith('/')) path = '${path}index.html'; | |
| 60 | + while (path.startsWith('/')) path = path.substring(1); | |
| 61 | + // The one rule that is not "read the file": a path that escapes the root | |
| 62 | + // is refused rather than resolved. This binds to 0.0.0.0, which in a | |
| 63 | + // container means the internet is one tunnel away. | |
| 64 | + final file = File('${root.path}/$path').absolute; | |
| 65 | + final resolved = file.path; | |
| 66 | + if (!resolved.startsWith(root.path)) { | |
| 67 | + request.response.statusCode = HttpStatus.forbidden; | |
| 68 | + await request.response.close(); | |
| 69 | + continue; | |
| 70 | + } | |
| 71 | + if (!await file.exists()) { | |
| 72 | + request.response.statusCode = HttpStatus.notFound; | |
| 73 | + request.response.write('not found: $path'); | |
| 74 | + await request.response.close(); | |
| 75 | + continue; | |
| 76 | + } | |
| 77 | + request.response.headers.contentType = ContentType.parse(_typeOf(resolved)); | |
| 78 | + // A single-page app served from a build directory: nothing here is | |
| 79 | + // versioned by name, so every response is one the browser must re-ask for. | |
| 80 | + request.response.headers.set('cache-control', 'no-store'); | |
| 81 | + await request.response.addStream(file.openRead()); | |
| 82 | + await request.response.close(); | |
| 83 | + } | |
| 84 | +} | |
| new file mode 100644 | |||
| @@ -0,0 +1,84 @@ | |||
| 1 | +// A static file server in one file, for `tools/build-web.sh serve`. | ||
| 2 | +// | ||
| 3 | +// `dart:io` and no packages, so there is nothing to `pub get` and nothing to | ||
| 4 | +// pin: the toolchain already has a Dart, because Flutter ships one, and that | ||
| 5 | +// is the whole reason this is Dart rather than the `python3 -m http.server` | ||
| 6 | +// it replaces. Serving a directory should not add a language to the list of | ||
| 7 | +// things a machine must have. | ||
| 8 | +// | ||
| 9 | +// It is the development server and says so: no caching headers, no | ||
| 10 | +// compression, no range requests. `.modal/flutter-web/serve.py` is the one | ||
| 11 | +// that faces a browser over the internet, and it has all three. | ||
| 12 | +// | ||
| 13 | +// dart tools/serve-dir.dart <directory> [port] | ||
| 14 | +import 'dart:io'; | ||
| 15 | + | ||
| 16 | +// Enough of them for a Flutter web bundle, which is HTML, JavaScript, JSON, a | ||
| 17 | +// wasm blob, fonts and images. Anything else goes out as bytes. | ||
| 18 | +const _types = <String, String>{ | ||
| 19 | + '.html': 'text/html; charset=utf-8', | ||
| 20 | + '.js': 'application/javascript; charset=utf-8', | ||
| 21 | + '.mjs': 'application/javascript; charset=utf-8', | ||
| 22 | + '.json': 'application/json; charset=utf-8', | ||
| 23 | + '.css': 'text/css; charset=utf-8', | ||
| 24 | + '.wasm': 'application/wasm', | ||
| 25 | + '.png': 'image/png', | ||
| 26 | + '.jpg': 'image/jpeg', | ||
| 27 | + '.jpeg': 'image/jpeg', | ||
| 28 | + '.gif': 'image/gif', | ||
| 29 | + '.svg': 'image/svg+xml', | ||
| 30 | + '.ico': 'image/x-icon', | ||
| 31 | + '.ttf': 'font/ttf', | ||
| 32 | + '.otf': 'font/otf', | ||
| 33 | + '.woff': 'font/woff', | ||
| 34 | + '.woff2': 'font/woff2', | ||
| 35 | + '.map': 'application/json; charset=utf-8', | ||
| 36 | +}; | ||
| 37 | + | ||
| 38 | +String _typeOf(String path) { | ||
| 39 | + final dot = path.lastIndexOf('.'); | ||
| 40 | + if (dot < 0) return 'application/octet-stream'; | ||
| 41 | + return _types[path.substring(dot).toLowerCase()] ?? 'application/octet-stream'; | ||
| 42 | +} | ||
| 43 | + | ||
| 44 | +Future<void> main(List<String> args) async { | ||
| 45 | + if (args.isEmpty) { | ||
| 46 | + stderr.writeln('usage: dart serve-dir.dart <directory> [port]'); | ||
| 47 | + exit(2); | ||
| 48 | + } | ||
| 49 | + final root = Directory(args[0]).absolute; | ||
| 50 | + final port = args.length > 1 ? int.parse(args[1]) : 8080; | ||
| 51 | + | ||
| 52 | + // 0.0.0.0 and not loopback: in the container this is behind a Modal tunnel, | ||
| 53 | + // and a server bound to 127.0.0.1 is one the tunnel cannot reach. | ||
| 54 | + final server = await HttpServer.bind(InternetAddress.anyIPv4, port); | ||
| 55 | + stdout.writeln('serving ${root.path} on :$port'); | ||
| 56 | + | ||
| 57 | + await for (final request in server) { | ||
| 58 | + var path = Uri.decodeComponent(request.uri.path); | ||
| 59 | + if (path.endsWith('/')) path = '${path}index.html'; | ||
| 60 | + while (path.startsWith('/')) path = path.substring(1); | ||
| 61 | + // The one rule that is not "read the file": a path that escapes the root | ||
| 62 | + // is refused rather than resolved. This binds to 0.0.0.0, which in a | ||
| 63 | + // container means the internet is one tunnel away. | ||
| 64 | + final file = File('${root.path}/$path').absolute; | ||
| 65 | + final resolved = file.path; | ||
| 66 | + if (!resolved.startsWith(root.path)) { | ||
| 67 | + request.response.statusCode = HttpStatus.forbidden; | ||
| 68 | + await request.response.close(); | ||
| 69 | + continue; | ||
| 70 | + } | ||
| 71 | + if (!await file.exists()) { | ||
| 72 | + request.response.statusCode = HttpStatus.notFound; | ||
| 73 | + request.response.write('not found: $path'); | ||
| 74 | + await request.response.close(); | ||
| 75 | + continue; | ||
| 76 | + } | ||
| 77 | + request.response.headers.contentType = ContentType.parse(_typeOf(resolved)); | ||
| 78 | + // A single-page app served from a build directory: nothing here is | ||
| 79 | + // versioned by name, so every response is one the browser must re-ask for. | ||
| 80 | + request.response.headers.set('cache-control', 'no-store'); | ||
| 81 | + await request.response.addStream(file.openRead()); | ||
| 82 | + await request.response.close(); | ||
| 83 | + } | ||
| 84 | +} | ||
added
tools/toolchain.sh +160 -0 | new file mode 100755 | ||
| @@ -0,0 +1,160 @@ | ||
| 1 | +#!/usr/bin/env bash | |
| 2 | +# The toolchain the web build needs, fetched by hand. | |
| 3 | +# | |
| 4 | +# Three tarballs — Flutter (which carries Dart), a JDK, and the Clojure CLI — | |
| 5 | +# pinned by version and by sha256, unpacked into `.toolchain/`, and put on a | |
| 6 | +# PATH. That is the whole of it. No nix, no image, no devShell: a checkout | |
| 7 | +# plus this script is a machine that can build the web bundle, and the same | |
| 8 | +# script is what the Modal container runs. | |
| 9 | +# | |
| 10 | +# Why not DotSlash, which the rest of the repo uses for its native libraries: | |
| 11 | +# DotSlash hands out an *immutable* cached artifact, and Flutter is not one. | |
| 12 | +# `flutter build web` downloads its engine artifacts into `bin/cache/` inside | |
| 13 | +# its own SDK directory the first time it runs, so the SDK has to be writable | |
| 14 | +# — which is the same thing `just apk` learned when it had to copy the | |
| 15 | +# store's Android SDK out to `flutter/.home` before Gradle would touch it. | |
| 16 | +# A pinned URL and a checked hash give the reproducibility DotSlash is for; | |
| 17 | +# the writability is what it cannot give. | |
| 18 | +# | |
| 19 | +# tools/toolchain.sh fetch whatever is missing | |
| 20 | +# eval "$(tools/toolchain.sh env)" ...and put it on this shell's PATH | |
| 21 | +# tools/toolchain.sh exec -- flutter --version | |
| 22 | +# | |
| 23 | +# `FRQ_TOOLCHAIN` says where it all lives; the default is `.toolchain/` at | |
| 24 | +# the top of the checkout, and the container points it at a volume so the | |
| 25 | +# fetch happens once across runs rather than once across containers. | |
| 26 | +set -euo pipefail | |
| 27 | + | |
| 28 | +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | |
| 29 | +TC="${FRQ_TOOLCHAIN:-$root/.toolchain}" | |
| 30 | + | |
| 31 | +# The pins. A version and its hash, and nothing derived at run time: a | |
| 32 | +# toolchain that resolves "latest" is a toolchain that changes under you | |
| 33 | +# between two builds of the same commit. | |
| 34 | +# | |
| 35 | +# Flutter 3.47.0 is the version this tree was building with under nix, and | |
| 36 | +# its Dart 3.13.0 is what `flutter/pubspec.yaml` asks for with `sdk: ^3.13.0`. | |
| 37 | +# To move it: take `version`, `archive` and `sha256` from | |
| 38 | +# https://storage.googleapis.com/flutter_infra_release/releases/releases_linux.json | |
| 39 | +FLUTTER_VERSION="3.47.0" | |
| 40 | +FLUTTER_URL="https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_${FLUTTER_VERSION}-stable.tar.xz" | |
| 41 | +FLUTTER_SHA="26cd99d3d94b1367e6b50535a18aeef0282c10a535bbe3ec493534dcdab75296" | |
| 42 | + | |
| 43 | +# Temurin 17, because `clojure` is a JVM program and ClojureDart's compiler | |
| 44 | +# runs there. Nothing else in this build wants a JVM. | |
| 45 | +JDK_VERSION="17.0.20.1+1" | |
| 46 | +JDK_URL="https://github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.20.1%2B1/OpenJDK17U-jdk_x64_linux_hotspot_17.0.20.1_1.tar.gz" | |
| 47 | +JDK_SHA="3808d1d15e3ec6bd5b84057fb5d84c33d8a1536a258146bcea2e603fc726e08e" | |
| 48 | + | |
| 49 | +# The Clojure CLI, which is a pair of shell scripts and a jar. Upstream ships | |
| 50 | +# an installer; `install_clojure` below is the four lines of it that matter. | |
| 51 | +CLOJURE_VERSION="1.12.6.1673" | |
| 52 | +CLOJURE_URL="https://github.com/clojure/brew-install/releases/download/${CLOJURE_VERSION}/clojure-tools-${CLOJURE_VERSION}.tar.gz" | |
| 53 | +CLOJURE_SHA="fe9194858e75d5af13c2e2aff92d710674d5bc5105f2b42f90a7d94d82ec023c" | |
| 54 | + | |
| 55 | +# What the host still has to bring. Small, boring, and on every machine and | |
| 56 | +# in every base image that is not deliberately empty — but Flutter shells out | |
| 57 | +# to `git` on its own SDK and to `unzip` on its downloads, so a missing one | |
| 58 | +# fails somewhere far from here with a much worse message than this. | |
| 59 | +require_host_tools() { | |
| 60 | + local missing=() | |
| 61 | + for t in curl tar git unzip; do | |
| 62 | + command -v "$t" >/dev/null 2>&1 || missing+=("$t") | |
| 63 | + done | |
| 64 | + if [ ${#missing[@]} -gt 0 ]; then | |
| 65 | + echo "toolchain: this needs ${missing[*]} on PATH and cannot fetch them" >&2 | |
| 66 | + exit 1 | |
| 67 | + fi | |
| 68 | +} | |
| 69 | + | |
| 70 | +# One archive, unpacked once. The stamp holds the hash rather than the | |
| 71 | +# version, so re-pointing a pin at the same version with different bytes also | |
| 72 | +# refetches, and a half-finished unpack is never mistaken for a finished one: | |
| 73 | +# the work happens in `.tmp` and the `mv` at the end is what publishes it. | |
| 74 | +install_archive() { | |
| 75 | + local name=$1 url=$2 sha=$3 strip=$4 | |
| 76 | + local dest="$TC/$name" stamp="$TC/$name.sha256" | |
| 77 | + if [ -d "$dest" ] && [ "$(cat "$stamp" 2>/dev/null || true)" = "$sha" ]; then | |
| 78 | + return 0 | |
| 79 | + fi | |
| 80 | + echo "toolchain: fetching $name" >&2 | |
| 81 | + local dl="$TC/.download.$name" | |
| 82 | + rm -rf "$dest" "$dest.tmp" "$dl" | |
| 83 | + mkdir -p "$dest.tmp" | |
| 84 | + curl -fsSL --retry 3 -o "$dl" "$url" | |
| 85 | + echo "$sha $dl" | sha256sum -c - >/dev/null | |
| 86 | + tar -xf "$dl" -C "$dest.tmp" --strip-components="$strip" | |
| 87 | + rm -f "$dl" | |
| 88 | + mv "$dest.tmp" "$dest" | |
| 89 | + echo "$sha" > "$stamp" | |
| 90 | +} | |
| 91 | + | |
| 92 | +# Upstream's install.sh, minus the ruby. The scripts ship with `PREFIX` and | |
| 93 | +# `BINDIR` written into them literally and an installer that substitutes the | |
| 94 | +# directory it is installing to; this is that, done where the tarball landed. | |
| 95 | +install_clojure() { | |
| 96 | + local dest="$TC/clojure" | |
| 97 | + [ -x "$dest/bin/clojure" ] && return 0 | |
| 98 | + mkdir -p "$dest/libexec" "$dest/bin" | |
| 99 | + cp "$dest"/*.jar "$dest/libexec/" | |
| 100 | + sed "s|PREFIX|$dest|g" "$dest/clojure" > "$dest/bin/clojure" | |
| 101 | + sed "s|BINDIR|$dest/bin|g" "$dest/clj" > "$dest/bin/clj" | |
| 102 | + chmod +x "$dest/bin/clojure" "$dest/bin/clj" | |
| 103 | +} | |
| 104 | + | |
| 105 | +install_all() { | |
| 106 | + require_host_tools | |
| 107 | + mkdir -p "$TC" | |
| 108 | + install_archive flutter "$FLUTTER_URL" "$FLUTTER_SHA" 1 | |
| 109 | + install_archive jdk "$JDK_URL" "$JDK_SHA" 1 | |
| 110 | + install_archive clojure "$CLOJURE_URL" "$CLOJURE_SHA" 1 | |
| 111 | + install_clojure | |
| 112 | +} | |
| 113 | + | |
| 114 | +# The environment, as shell. Everything that would otherwise land in a home | |
| 115 | +# directory is named here and kept inside the toolchain instead: the pub | |
| 116 | +# cache, the git dependencies tools.deps clones, the local maven repo. One | |
| 117 | +# directory to keep on a volume, one directory to delete when it goes wrong. | |
| 118 | +# | |
| 119 | +# GITLIBS and the maven repo are set for the reason `just apk` sets them — | |
| 120 | +# the JVM reads user.home out of /etc/passwd, so neither of them follows HOME. | |
| 121 | +print_env() { | |
| 122 | + cat <<ENV | |
| 123 | +export FRQ_TOOLCHAIN="$TC" | |
| 124 | +# Flutter's SDK tarball is a git checkout, and the tool shells out to git | |
| 125 | +# against it for its version -- which fails with "detected dubious ownership" | |
| 126 | +# whenever the files' owner is not the user running the build. That is the | |
| 127 | +# normal case on a Modal volume, and the failure is not a warning: the dart | |
| 128 | +# process ClojureDart's live analyzer talks to dies with it, and the compile | |
| 129 | +# ends at 'EOF while reading' with nothing about git in the message. | |
| 130 | +# | |
| 131 | +# Said through the environment rather than 'git config --global', so it | |
| 132 | +# travels with this shell and writes nothing into anyone's ~/.gitconfig. | |
| 133 | +export GIT_CONFIG_COUNT=1 | |
| 134 | +export GIT_CONFIG_KEY_0=safe.directory | |
| 135 | +export GIT_CONFIG_VALUE_0="$TC/flutter" | |
| 136 | +export JAVA_HOME="$TC/jdk" | |
| 137 | +export PUB_CACHE="$TC/pub-cache" | |
| 138 | +export GITLIBS="$TC/gitlibs" | |
| 139 | +export FRQ_M2="$TC/m2" | |
| 140 | +export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:\$PATH" | |
| 141 | +ENV | |
| 142 | +} | |
| 143 | + | |
| 144 | +case "${1:-install}" in | |
| 145 | + install) install_all ;; | |
| 146 | + env) install_all; print_env ;; | |
| 147 | + exec) | |
| 148 | + install_all | |
| 149 | + shift | |
| 150 | + [ "${1:-}" = "--" ] && shift | |
| 151 | + eval "$(print_env)" | |
| 152 | + exec "$@" | |
| 153 | + ;; | |
| 154 | + versions) | |
| 155 | + echo "flutter $FLUTTER_VERSION" | |
| 156 | + echo "jdk $JDK_VERSION" | |
| 157 | + echo "clojure $CLOJURE_VERSION" | |
| 158 | + ;; | |
| 159 | + *) echo "usage: toolchain.sh [install|env|exec -- cmd...|versions]" >&2; exit 1 ;; | |
| 160 | +esac | |
| new file mode 100755 | |||
| @@ -0,0 +1,160 @@ | |||
| 1 | +#!/usr/bin/env bash | ||
| 2 | +# The toolchain the web build needs, fetched by hand. | ||
| 3 | +# | ||
| 4 | +# Three tarballs — Flutter (which carries Dart), a JDK, and the Clojure CLI — | ||
| 5 | +# pinned by version and by sha256, unpacked into `.toolchain/`, and put on a | ||
| 6 | +# PATH. That is the whole of it. No nix, no image, no devShell: a checkout | ||
| 7 | +# plus this script is a machine that can build the web bundle, and the same | ||
| 8 | +# script is what the Modal container runs. | ||
| 9 | +# | ||
| 10 | +# Why not DotSlash, which the rest of the repo uses for its native libraries: | ||
| 11 | +# DotSlash hands out an *immutable* cached artifact, and Flutter is not one. | ||
| 12 | +# `flutter build web` downloads its engine artifacts into `bin/cache/` inside | ||
| 13 | +# its own SDK directory the first time it runs, so the SDK has to be writable | ||
| 14 | +# — which is the same thing `just apk` learned when it had to copy the | ||
| 15 | +# store's Android SDK out to `flutter/.home` before Gradle would touch it. | ||
| 16 | +# A pinned URL and a checked hash give the reproducibility DotSlash is for; | ||
| 17 | +# the writability is what it cannot give. | ||
| 18 | +# | ||
| 19 | +# tools/toolchain.sh fetch whatever is missing | ||
| 20 | +# eval "$(tools/toolchain.sh env)" ...and put it on this shell's PATH | ||
| 21 | +# tools/toolchain.sh exec -- flutter --version | ||
| 22 | +# | ||
| 23 | +# `FRQ_TOOLCHAIN` says where it all lives; the default is `.toolchain/` at | ||
| 24 | +# the top of the checkout, and the container points it at a volume so the | ||
| 25 | +# fetch happens once across runs rather than once across containers. | ||
| 26 | +set -euo pipefail | ||
| 27 | + | ||
| 28 | +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | ||
| 29 | +TC="${FRQ_TOOLCHAIN:-$root/.toolchain}" | ||
| 30 | + | ||
| 31 | +# The pins. A version and its hash, and nothing derived at run time: a | ||
| 32 | +# toolchain that resolves "latest" is a toolchain that changes under you | ||
| 33 | +# between two builds of the same commit. | ||
| 34 | +# | ||
| 35 | +# Flutter 3.47.0 is the version this tree was building with under nix, and | ||
| 36 | +# its Dart 3.13.0 is what `flutter/pubspec.yaml` asks for with `sdk: ^3.13.0`. | ||
| 37 | +# To move it: take `version`, `archive` and `sha256` from | ||
| 38 | +# https://storage.googleapis.com/flutter_infra_release/releases/releases_linux.json | ||
| 39 | +FLUTTER_VERSION="3.47.0" | ||
| 40 | +FLUTTER_URL="https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_${FLUTTER_VERSION}-stable.tar.xz" | ||
| 41 | +FLUTTER_SHA="26cd99d3d94b1367e6b50535a18aeef0282c10a535bbe3ec493534dcdab75296" | ||
| 42 | + | ||
| 43 | +# Temurin 17, because `clojure` is a JVM program and ClojureDart's compiler | ||
| 44 | +# runs there. Nothing else in this build wants a JVM. | ||
| 45 | +JDK_VERSION="17.0.20.1+1" | ||
| 46 | +JDK_URL="https://github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.20.1%2B1/OpenJDK17U-jdk_x64_linux_hotspot_17.0.20.1_1.tar.gz" | ||
| 47 | +JDK_SHA="3808d1d15e3ec6bd5b84057fb5d84c33d8a1536a258146bcea2e603fc726e08e" | ||
| 48 | + | ||
| 49 | +# The Clojure CLI, which is a pair of shell scripts and a jar. Upstream ships | ||
| 50 | +# an installer; `install_clojure` below is the four lines of it that matter. | ||
| 51 | +CLOJURE_VERSION="1.12.6.1673" | ||
| 52 | +CLOJURE_URL="https://github.com/clojure/brew-install/releases/download/${CLOJURE_VERSION}/clojure-tools-${CLOJURE_VERSION}.tar.gz" | ||
| 53 | +CLOJURE_SHA="fe9194858e75d5af13c2e2aff92d710674d5bc5105f2b42f90a7d94d82ec023c" | ||
| 54 | + | ||
| 55 | +# What the host still has to bring. Small, boring, and on every machine and | ||
| 56 | +# in every base image that is not deliberately empty — but Flutter shells out | ||
| 57 | +# to `git` on its own SDK and to `unzip` on its downloads, so a missing one | ||
| 58 | +# fails somewhere far from here with a much worse message than this. | ||
| 59 | +require_host_tools() { | ||
| 60 | + local missing=() | ||
| 61 | + for t in curl tar git unzip; do | ||
| 62 | + command -v "$t" >/dev/null 2>&1 || missing+=("$t") | ||
| 63 | + done | ||
| 64 | + if [ ${#missing[@]} -gt 0 ]; then | ||
| 65 | + echo "toolchain: this needs ${missing[*]} on PATH and cannot fetch them" >&2 | ||
| 66 | + exit 1 | ||
| 67 | + fi | ||
| 68 | +} | ||
| 69 | + | ||
| 70 | +# One archive, unpacked once. The stamp holds the hash rather than the | ||
| 71 | +# version, so re-pointing a pin at the same version with different bytes also | ||
| 72 | +# refetches, and a half-finished unpack is never mistaken for a finished one: | ||
| 73 | +# the work happens in `.tmp` and the `mv` at the end is what publishes it. | ||
| 74 | +install_archive() { | ||
| 75 | + local name=$1 url=$2 sha=$3 strip=$4 | ||
| 76 | + local dest="$TC/$name" stamp="$TC/$name.sha256" | ||
| 77 | + if [ -d "$dest" ] && [ "$(cat "$stamp" 2>/dev/null || true)" = "$sha" ]; then | ||
| 78 | + return 0 | ||
| 79 | + fi | ||
| 80 | + echo "toolchain: fetching $name" >&2 | ||
| 81 | + local dl="$TC/.download.$name" | ||
| 82 | + rm -rf "$dest" "$dest.tmp" "$dl" | ||
| 83 | + mkdir -p "$dest.tmp" | ||
| 84 | + curl -fsSL --retry 3 -o "$dl" "$url" | ||
| 85 | + echo "$sha $dl" | sha256sum -c - >/dev/null | ||
| 86 | + tar -xf "$dl" -C "$dest.tmp" --strip-components="$strip" | ||
| 87 | + rm -f "$dl" | ||
| 88 | + mv "$dest.tmp" "$dest" | ||
| 89 | + echo "$sha" > "$stamp" | ||
| 90 | +} | ||
| 91 | + | ||
| 92 | +# Upstream's install.sh, minus the ruby. The scripts ship with `PREFIX` and | ||
| 93 | +# `BINDIR` written into them literally and an installer that substitutes the | ||
| 94 | +# directory it is installing to; this is that, done where the tarball landed. | ||
| 95 | +install_clojure() { | ||
| 96 | + local dest="$TC/clojure" | ||
| 97 | + [ -x "$dest/bin/clojure" ] && return 0 | ||
| 98 | + mkdir -p "$dest/libexec" "$dest/bin" | ||
| 99 | + cp "$dest"/*.jar "$dest/libexec/" | ||
| 100 | + sed "s|PREFIX|$dest|g" "$dest/clojure" > "$dest/bin/clojure" | ||
| 101 | + sed "s|BINDIR|$dest/bin|g" "$dest/clj" > "$dest/bin/clj" | ||
| 102 | + chmod +x "$dest/bin/clojure" "$dest/bin/clj" | ||
| 103 | +} | ||
| 104 | + | ||
| 105 | +install_all() { | ||
| 106 | + require_host_tools | ||
| 107 | + mkdir -p "$TC" | ||
| 108 | + install_archive flutter "$FLUTTER_URL" "$FLUTTER_SHA" 1 | ||
| 109 | + install_archive jdk "$JDK_URL" "$JDK_SHA" 1 | ||
| 110 | + install_archive clojure "$CLOJURE_URL" "$CLOJURE_SHA" 1 | ||
| 111 | + install_clojure | ||
| 112 | +} | ||
| 113 | + | ||
| 114 | +# The environment, as shell. Everything that would otherwise land in a home | ||
| 115 | +# directory is named here and kept inside the toolchain instead: the pub | ||
| 116 | +# cache, the git dependencies tools.deps clones, the local maven repo. One | ||
| 117 | +# directory to keep on a volume, one directory to delete when it goes wrong. | ||
| 118 | +# | ||
| 119 | +# GITLIBS and the maven repo are set for the reason `just apk` sets them — | ||
| 120 | +# the JVM reads user.home out of /etc/passwd, so neither of them follows HOME. | ||
| 121 | +print_env() { | ||
| 122 | + cat <<ENV | ||
| 123 | +export FRQ_TOOLCHAIN="$TC" | ||
| 124 | +# Flutter's SDK tarball is a git checkout, and the tool shells out to git | ||
| 125 | +# against it for its version -- which fails with "detected dubious ownership" | ||
| 126 | +# whenever the files' owner is not the user running the build. That is the | ||
| 127 | +# normal case on a Modal volume, and the failure is not a warning: the dart | ||
| 128 | +# process ClojureDart's live analyzer talks to dies with it, and the compile | ||
| 129 | +# ends at 'EOF while reading' with nothing about git in the message. | ||
| 130 | +# | ||
| 131 | +# Said through the environment rather than 'git config --global', so it | ||
| 132 | +# travels with this shell and writes nothing into anyone's ~/.gitconfig. | ||
| 133 | +export GIT_CONFIG_COUNT=1 | ||
| 134 | +export GIT_CONFIG_KEY_0=safe.directory | ||
| 135 | +export GIT_CONFIG_VALUE_0="$TC/flutter" | ||
| 136 | +export JAVA_HOME="$TC/jdk" | ||
| 137 | +export PUB_CACHE="$TC/pub-cache" | ||
| 138 | +export GITLIBS="$TC/gitlibs" | ||
| 139 | +export FRQ_M2="$TC/m2" | ||
| 140 | +export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:\$PATH" | ||
| 141 | +ENV | ||
| 142 | +} | ||
| 143 | + | ||
| 144 | +case "${1:-install}" in | ||
| 145 | + install) install_all ;; | ||
| 146 | + env) install_all; print_env ;; | ||
| 147 | + exec) | ||
| 148 | + install_all | ||
| 149 | + shift | ||
| 150 | + [ "${1:-}" = "--" ] && shift | ||
| 151 | + eval "$(print_env)" | ||
| 152 | + exec "$@" | ||
| 153 | + ;; | ||
| 154 | + versions) | ||
| 155 | + echo "flutter $FLUTTER_VERSION" | ||
| 156 | + echo "jdk $JDK_VERSION" | ||
| 157 | + echo "clojure $CLOJURE_VERSION" | ||
| 158 | + ;; | ||
| 159 | + *) echo "usage: toolchain.sh [install|env|exec -- cmd...|versions]" >&2; exit 1 ;; | ||
| 160 | +esac | ||