rickub
nandi/frqpublic⑂ Fork 0
⑂ 2e24e647a7766c822d13d253777204c3bf92fb7a
Branches
claude/add-remote-repository-47d12e main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/frq.git
git clone ssh://git@rickub.com/nandi/frq.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

toolchain.sh · 229 lines · 10.8 KBBash Blame HistoryRaw
Three tarballs where a devShell was 5ce66d5 nandi yesterday1#!/usr/bin/env bash
2# The toolchain the web build needs, fetched by hand.
3#
Six verbs, and the last of the nix 2e24e64 nandi 16h ago4# Archives — Flutter (which carries Dart), a JDK, the Clojure CLI and Nim —
Three tarballs where a devShell was 5ce66d5 nandi yesterday5# pinned by version and by sha256, unpacked into `.toolchain/`, and put on a
6# PATH. That is the whole of it. No nix, no image, no devShell: a checkout
Six verbs, and the last of the nix 2e24e64 nandi 16h ago7# plus this script is a machine that can build any target here, and the same
8# script is what the Modal containers run.
9#
10# The Android SDK is the one thing not fetched by default, because only
11# `just build apk` wants it and it is large: `tools/toolchain.sh android`
12# fetches Google's command-line tools and lets sdkmanager finish the job.
Three tarballs where a devShell was 5ce66d5 nandi yesterday13#
14# Why not DotSlash, which the rest of the repo uses for its native libraries:
15# DotSlash hands out an *immutable* cached artifact, and Flutter is not one.
16# `flutter build web` downloads its engine artifacts into `bin/cache/` inside
17# its own SDK directory the first time it runs, so the SDK has to be writable
Six verbs, and the last of the nix 2e24e64 nandi 16h ago18# — which is the same reason the Android SDK below is fetched into a
19# directory of ours rather than used read-only from anywhere.
Three tarballs where a devShell was 5ce66d5 nandi yesterday20# A pinned URL and a checked hash give the reproducibility DotSlash is for;
21# the writability is what it cannot give.
22#
23# tools/toolchain.sh fetch whatever is missing
Six verbs, and the last of the nix 2e24e64 nandi 16h ago24# tools/toolchain.sh android ...and the Android SDK as well
Three tarballs where a devShell was 5ce66d5 nandi yesterday25# eval "$(tools/toolchain.sh env)" ...and put it on this shell's PATH
26# tools/toolchain.sh exec -- flutter --version
27#
28# `FRQ_TOOLCHAIN` says where it all lives; the default is `.toolchain/` at
29# the top of the checkout, and the container points it at a volume so the
30# fetch happens once across runs rather than once across containers.
31set -euo pipefail
32
33root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
34TC="${FRQ_TOOLCHAIN:-$root/.toolchain}"
35
36# The pins. A version and its hash, and nothing derived at run time: a
37# toolchain that resolves "latest" is a toolchain that changes under you
38# between two builds of the same commit.
39#
40# Flutter 3.47.0 is the version this tree was building with under nix, and
41# its Dart 3.13.0 is what `flutter/pubspec.yaml` asks for with `sdk: ^3.13.0`.
42# To move it: take `version`, `archive` and `sha256` from
43# https://storage.googleapis.com/flutter_infra_release/releases/releases_linux.json
44FLUTTER_VERSION="3.47.0"
45FLUTTER_URL="https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_${FLUTTER_VERSION}-stable.tar.xz"
46FLUTTER_SHA="26cd99d3d94b1367e6b50535a18aeef0282c10a535bbe3ec493534dcdab75296"
47
48# Temurin 17, because `clojure` is a JVM program and ClojureDart's compiler
49# runs there. Nothing else in this build wants a JVM.
50JDK_VERSION="17.0.20.1+1"
51JDK_URL="https://github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.20.1%2B1/OpenJDK17U-jdk_x64_linux_hotspot_17.0.20.1_1.tar.gz"
52JDK_SHA="3808d1d15e3ec6bd5b84057fb5d84c33d8a1536a258146bcea2e603fc726e08e"
53
54# The Clojure CLI, which is a pair of shell scripts and a jar. Upstream ships
55# an installer; `install_clojure` below is the four lines of it that matter.
56CLOJURE_VERSION="1.12.6.1673"
57CLOJURE_URL="https://github.com/clojure/brew-install/releases/download/${CLOJURE_VERSION}/clojure-tools-${CLOJURE_VERSION}.tar.gz"
58CLOJURE_SHA="fe9194858e75d5af13c2e2aff92d710674d5bc5105f2b42f90a7d94d82ec023c"
59
Six verbs, and the last of the nix 2e24e64 nandi 16h ago60# Nim, for `nim/` — the portable core. Upstream's prebuilt linux_x64 tarball,
61# so there is no bootstrap compile here. The core deliberately has no
62# dependencies outside Nim's standard library, so the compiler is all of it.
63#
64# Two things Nim wants from the host rather than from here: a C compiler,
65# because `nim c` shells out to one, and OpenSSL, because `-d:ssl` in
66# nim/nim.cfg makes std/net resolve -lssl and -lcrypto through dynlib at run
67# time. A missing libssl is a SIGSEGV in `newContext` that says nothing about
68# SSL, which is why require_host_tools checks for cc up front.
69NIM_VERSION="2.2.4"
70NIM_URL="https://nim-lang.org/download/nim-${NIM_VERSION}-linux_x64.tar.xz"
71NIM_SHA="791802138aaf19c8579232c50b4998ce2ae2928b791127ce5b4ef3c7af53fb46"
72
73# Google's command-line tools, which is the smallest thing that can install an
74# Android SDK. The platform and build-tools are not pinned here: the versions
75# come from whatever Flutter asks Gradle for, and sdkmanager fetches them into
76# the same writable directory on first use. `install_android` below.
77ANDROID_TOOLS_VERSION="11076708"
78ANDROID_TOOLS_URL="https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_TOOLS_VERSION}_latest.zip"
79ANDROID_TOOLS_SHA="2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258"
80
Three tarballs where a devShell was 5ce66d5 nandi yesterday81# What the host still has to bring. Small, boring, and on every machine and
82# in every base image that is not deliberately empty — but Flutter shells out
83# to `git` on its own SDK and to `unzip` on its downloads, so a missing one
84# fails somewhere far from here with a much worse message than this.
85require_host_tools() {
86 local missing=()
Six verbs, and the last of the nix 2e24e64 nandi 16h ago87 for t in curl tar git unzip cc; do
Three tarballs where a devShell was 5ce66d5 nandi yesterday88 command -v "$t" >/dev/null 2>&1 || missing+=("$t")
89 done
90 if [ ${#missing[@]} -gt 0 ]; then
91 echo "toolchain: this needs ${missing[*]} on PATH and cannot fetch them" >&2
92 exit 1
93 fi
94}
95
96# One archive, unpacked once. The stamp holds the hash rather than the
97# version, so re-pointing a pin at the same version with different bytes also
98# refetches, and a half-finished unpack is never mistaken for a finished one:
99# the work happens in `.tmp` and the `mv` at the end is what publishes it.
100install_archive() {
101 local name=$1 url=$2 sha=$3 strip=$4
102 local dest="$TC/$name" stamp="$TC/$name.sha256"
103 if [ -d "$dest" ] && [ "$(cat "$stamp" 2>/dev/null || true)" = "$sha" ]; then
104 return 0
105 fi
106 echo "toolchain: fetching $name" >&2
107 local dl="$TC/.download.$name"
108 rm -rf "$dest" "$dest.tmp" "$dl"
109 mkdir -p "$dest.tmp"
110 curl -fsSL --retry 3 -o "$dl" "$url"
111 echo "$sha $dl" | sha256sum -c - >/dev/null
112 tar -xf "$dl" -C "$dest.tmp" --strip-components="$strip"
113 rm -f "$dl"
114 mv "$dest.tmp" "$dest"
115 echo "$sha" > "$stamp"
116}
117
118# Upstream's install.sh, minus the ruby. The scripts ship with `PREFIX` and
119# `BINDIR` written into them literally and an installer that substitutes the
120# directory it is installing to; this is that, done where the tarball landed.
121install_clojure() {
122 local dest="$TC/clojure"
123 [ -x "$dest/bin/clojure" ] && return 0
124 mkdir -p "$dest/libexec" "$dest/bin"
125 cp "$dest"/*.jar "$dest/libexec/"
126 sed "s|PREFIX|$dest|g" "$dest/clojure" > "$dest/bin/clojure"
127 sed "s|BINDIR|$dest/bin|g" "$dest/clj" > "$dest/bin/clj"
128 chmod +x "$dest/bin/clojure" "$dest/bin/clj"
129}
130
Six verbs, and the last of the nix 2e24e64 nandi 16h ago131# The Android SDK, which is a zip rather than a tarball and has to land in a
132# layout sdkmanager recognises: cmdline-tools/latest/, with the tools' own
133# top-level directory renamed. Everything after that — platform-tools, the
134# platform, the build-tools — Gradle asks sdkmanager for as it goes, which is
135# why this directory is ours and writable rather than a read-only artifact.
136#
137# No ndkVersion in android/app/build.gradle.kts, and nothing here installs an
138# NDK: there is no native code in the app to need one.
139install_android() {
140 local dest="$TC/android-sdk"
141 local stamp="$dest/.cmdline-tools.sha256"
142 if [ ! -d "$dest/cmdline-tools/latest" ] \
143 || [ "$(cat "$stamp" 2>/dev/null || true)" != "$ANDROID_TOOLS_SHA" ]; then
144 echo "toolchain: fetching the Android command-line tools" >&2
145 local dl="$TC/.download.android"
146 rm -rf "$dest/cmdline-tools" "$dl"
147 mkdir -p "$dest/cmdline-tools"
148 curl -fsSL --retry 3 -o "$dl" "$ANDROID_TOOLS_URL"
149 echo "$ANDROID_TOOLS_SHA $dl" | sha256sum -c - >/dev/null
150 unzip -q "$dl" -d "$dest/cmdline-tools"
151 rm -f "$dl"
152 mv "$dest/cmdline-tools/cmdline-tools" "$dest/cmdline-tools/latest"
153 echo "$ANDROID_TOOLS_SHA" > "$stamp"
154 fi
155 # Gradle will not install anything into an SDK whose licences are
156 # unaccepted, and it fails late and obscurely when they are not.
157 if [ ! -d "$dest/licenses" ]; then
158 JAVA_HOME="$TC/jdk" ANDROID_HOME="$dest" \
159 yes | "$dest/cmdline-tools/latest/bin/sdkmanager" --licenses >/dev/null
160 fi
161}
162
Three tarballs where a devShell was 5ce66d5 nandi yesterday163install_all() {
164 require_host_tools
165 mkdir -p "$TC"
166 install_archive flutter "$FLUTTER_URL" "$FLUTTER_SHA" 1
167 install_archive jdk "$JDK_URL" "$JDK_SHA" 1
168 install_archive clojure "$CLOJURE_URL" "$CLOJURE_SHA" 1
169 install_clojure
Six verbs, and the last of the nix 2e24e64 nandi 16h ago170 install_archive nim "$NIM_URL" "$NIM_SHA" 1
Three tarballs where a devShell was 5ce66d5 nandi yesterday171}
172
173# The environment, as shell. Everything that would otherwise land in a home
174# directory is named here and kept inside the toolchain instead: the pub
175# cache, the git dependencies tools.deps clones, the local maven repo. One
176# directory to keep on a volume, one directory to delete when it goes wrong.
177#
Six verbs, and the last of the nix 2e24e64 nandi 16h ago178# GITLIBS and the maven repo are named here because
Three tarballs where a devShell was 5ce66d5 nandi yesterday179# the JVM reads user.home out of /etc/passwd, so neither of them follows HOME.
180print_env() {
181 cat <<ENV
182export FRQ_TOOLCHAIN="$TC"
183# Flutter's SDK tarball is a git checkout, and the tool shells out to git
184# against it for its version -- which fails with "detected dubious ownership"
185# whenever the files' owner is not the user running the build. That is the
186# normal case on a Modal volume, and the failure is not a warning: the dart
187# process ClojureDart's live analyzer talks to dies with it, and the compile
188# ends at 'EOF while reading' with nothing about git in the message.
189#
190# Said through the environment rather than 'git config --global', so it
191# travels with this shell and writes nothing into anyone's ~/.gitconfig.
192export GIT_CONFIG_COUNT=1
193export GIT_CONFIG_KEY_0=safe.directory
194export GIT_CONFIG_VALUE_0="$TC/flutter"
195export JAVA_HOME="$TC/jdk"
196export PUB_CACHE="$TC/pub-cache"
197export GITLIBS="$TC/gitlibs"
198export FRQ_M2="$TC/m2"
Six verbs, and the last of the nix 2e24e64 nandi 16h ago199export ANDROID_HOME="$TC/android-sdk"
200export ANDROID_SDK_ROOT="$TC/android-sdk"
201# adb keeps the key the phone has already trusted here. Left to its default
202# it would follow HOME, and a build that moved HOME would hand the device a
203# new identity -- after which the deploy ends in "no devices/emulators found"
204# while \`adb devices\` in any other shell lists the phone perfectly well.
205export ANDROID_USER_HOME="\${ANDROID_USER_HOME:-\$HOME/.android}"
206export PATH="$TC/flutter/bin:$TC/jdk/bin:$TC/clojure/bin:$TC/nim/bin:$TC/android-sdk/platform-tools:\$PATH"
Three tarballs where a devShell was 5ce66d5 nandi yesterday207ENV
208}
209
210case "${1:-install}" in
211 install) install_all ;;
Six verbs, and the last of the nix 2e24e64 nandi 16h ago212 android) install_all; install_android ;;
Three tarballs where a devShell was 5ce66d5 nandi yesterday213 env) install_all; print_env ;;
214 exec)
215 install_all
216 shift
217 [ "${1:-}" = "--" ] && shift
218 eval "$(print_env)"
219 exec "$@"
220 ;;
221 versions)
222 echo "flutter $FLUTTER_VERSION"
223 echo "jdk $JDK_VERSION"
224 echo "clojure $CLOJURE_VERSION"
Six verbs, and the last of the nix 2e24e64 nandi 16h ago225 echo "nim $NIM_VERSION"
226 echo "android-tools $ANDROID_TOOLS_VERSION"
Three tarballs where a devShell was 5ce66d5 nandi yesterday227 ;;
Six verbs, and the last of the nix 2e24e64 nandi 16h ago228 *) echo "usage: toolchain.sh [install|android|env|exec -- cmd...|versions]" >&2; exit 1 ;;
Three tarballs where a devShell was 5ce66d5 nandi yesterday229esac