rickub
nandi/frqpublic⑂ Fork 0
⑂ 57a93bbc4ec783ac1772cfadcc4c573d3f30971c
Branches
claude/add-remote-repository-47d12e main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/frq.git
git clone ssh://git@rickub.com/nandi/frq.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

android.nix · 395 lines · 16.7 KBNix Blame HistoryRaw
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 17d ago1# The APK, as derivations.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago2#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 17d ago3# The only APK build there is: this replaced a buck2 graph and the script
4# before it, both of which named an Android SDK, an NDK, a hand-built Chez
5# cross target and an OpenSSL by absolute path and stopped when one was
6# missing. Every one of those is built or fetched here instead, so this works
7# from nothing on a machine with none of them and no ~/.cache at all:
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago8#
9# nix build .#apk
10#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 17d ago11# `just apk` is the same build with the store path handed to adb afterwards;
Inline the babashka scripts into the justfile 76dcc6c nandi 11d ago12# see the `apk` recipe in the justfile.
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 17d ago13#
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago14# On a machine with a remote builder configured, prefer
15#
16# nix build .#apk --store ssh-ng://eu.nixbuild.net --eval-store auto
17#
18# rather than letting `builders` do it. With `builders`, nix copies the output
19# of every remotely-built derivation back, and androidenv's NDK is both
20# `preferLocalBuild` and absent from cache.nixos.org — so the 3.1 GB unpacked
21# toolchain is built here and uploaded. With the remote as the *store* the
22# whole graph stays there, only .drv files go up, and the builder fetches
23# Google's zip over its own link.
24#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 17d ago25# The steps are the ones the graph before it ran, in the same order; where a
26# genrule read a path out of `read_root_config`, a derivation takes an
27# argument.
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago28{ pkgs, lib, self, chez-src, jolt-native, jolt-native-android, glimmer, joltAndroid, androidSdk, ndk }:
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago29
30let
31 # What the APK targets, in the three spellings the tools want it in.
32 apiLevel = "28";
33 targetSdk = "36";
34 abi = "arm64-v8a";
35 package = "uk.nandi.frq";
36 version = "0.1.0";
37
38 sdk = "${androidSdk}/libexec/android-sdk";
39 buildTools = "${sdk}/build-tools/36.0.0";
40 androidJar = "${sdk}/platforms/android-${targetSdk}/android.jar";
41
42 # The NDK's clang finds its sysroot, resource directory and the rest of LLVM
43 # relative to itself, so it is named by path rather than copied anywhere.
44 # androidenv installs the tree under libexec/android-sdk and leaves
45 # `ndk-bundle` pointing at the versioned directory beside it.
46 ndkRoot = "${ndk}/libexec/android-sdk/ndk-bundle";
47 ndkBin = "${ndkRoot}/toolchains/llvm/prebuilt/linux-x86_64/bin";
48 cc = "${ndkBin}/aarch64-linux-android${apiLevel}-clang";
49
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago50 # What comes out of jolt-native's CI, as a flake input rather than a fetchurl:
51 # nix unpacks a tarball input itself, and flake.lock holds the digest that
52 # used to be written here by hand. `just bump` moves it with
53 # `nix flake update jolt-native-android` and nothing in this file changes.
Ask for the camera and the microphone from the app 6c745df nandi 18d ago54 #
Paint frq with jvui, and take the last Rust out of the window 77963dc nandi 9d ago55 # One archive. libvidya (the retained-tree UI) is what is taken from it, and
56 # include/ beside it for the glue's headers.
57 #
58 # It also carries libjoltmoq and the libc++_shared that object needs, and
59 # neither is packed any more. frq's media plane is `frq.av.plane` now — MoQ
60 # over QUIC through libmoq_ffi, Opus, H.264 and the devices — and none of
61 # that is built for Android yet, so a phone has no media plane at all and
62 # `frq.av/available?` says so. Shipping the Rust one it no longer loads was
63 # seventeen megabytes of APK for an object nothing opens.
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago64 nativeLibs = jolt-native-android;
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago65
Paint frq with jvui, and take the last Rust out of the window 77963dc nandi 9d ago66 # No libc++_shared. It was here because openh264 is C++ and libjoltmoq
67 # carried it as a DT_NEEDED; libvidya links neither — its NEEDED list is
68 # libdl, libandroid, liblog, libm and libc — so with the media plane gone
69 # the C++ runtime has nothing to serve.
Ask for the camera and the microphone from the app 6c745df nandi 18d ago70
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago71 # --- Chez's arm64 cross target ------------------------------------------
72 # The one piece with no nixpkgs equivalent: `pkgs.chez` builds a Scheme for
73 # this machine, and what the boot image needs is Chez's `tarm64le` workarea —
74 # the target boot files, the cross compiler's xpatch, and the arm64
75 # libkernel.a that libjoltapp links.
76 #
77 # Three builds in one derivation, because each needs the one before it:
78 #
79 # ta6le a host Scheme, which is what cross-compiles anything
80 # boot XM=... the target's boot files and xpatch, made by that host
81 # tarm64le the target kernel, compiled by the NDK
82 #
83 # The flags are the ones the hand-built tree under ~/.cache/vidya-chez-android
84 # was configured with, read back out of its Mf-config. zlib is Android's own
85 # (`-lz`, which Bionic has); lz4 is the in-tree submodule, built for arm64
86 # here because a cross configure links it rather than building it.
87 chezAndroid = pkgs.stdenv.mkDerivation {
88 pname = "chez-scheme-android";
89 version = "10.4.1";
90 src = chez-src;
91
92 strictDeps = true;
93 nativeBuildInputs = with pkgs; [ gnumake which ];
94
95 dontConfigure = true;
96
97 buildPhase = ''
98 runHook preBuild
99
100 # Both workareas turn off the expression editor's two dependencies, which
101 # is what ~/.cache/vidya-chez-android was configured with (its Mf-config
102 # has empty cursesLib/ncursesLib, and disablex11=yes on the target). The
103 # host Scheme here is only ever a cross compiler, and Bionic has no
104 # curses.h at all — so on the target it is not a preference but a
105 # requirement.
106 ./configure -m=ta6le --disable-x11 --disable-curses CC_FOR_BUILD="$CC"
107 make -j"$NIX_BUILD_CORES"
108 make boot XM=tarm64le -j"$NIX_BUILD_CORES"
109
110 # lz4 for the phone, not for this machine: the host build above left an
111 # x86_64 liblz4.a in the same place, and the cross link needs it gone.
112 make -C lz4/lib clean
113 make -C lz4/lib liblz4.a -j"$NIX_BUILD_CORES" \
114 CC=${cc} AR=${ndkBin}/llvm-ar
115
116 # --disable-auto-flags stops configure appending -lrt and -lpthread, which
117 # is what its unix branch does for a glibc host and what Bionic has no
118 # separate libraries for — both live in libc there. Everything it would
119 # otherwise add is passed explicitly below, matching the Mf-config of
120 # the tree this was reconstructed from.
121 ./configure -m=tarm64le --cross --disable-x11 --disable-curses \
122 --disable-auto-flags \
123 LIBS="-ldl -lm" \
124 CC=${cc} \
125 AR=${ndkBin}/llvm-ar \
126 CC_FOR_BUILD="$CC" \
127 ZLIB=-lz \
128 LZ4="$PWD/lz4/lib/liblz4.a" \
129 CPPFLAGS="-I$PWD/lz4/lib" \
130 CFLAGS="-O2 -D_REENTRANT -pthread -fPIC"
131 make -j"$NIX_BUILD_CORES"
132
133 runHook postBuild
134 '';
135
136 # The whole workarea, at the paths CHEZ_ANDROID means: the host Scheme
137 # loads xpatch out of xc-tarm64le/s, and the link below reads two archives
138 # from elsewhere in the tree. Pruning it would only be guessing at which
139 # of those the cross compiler still opens.
140 installPhase = ''
141 runHook preInstall
142 mkdir -p "$out"
143 cp -r . "$out/"
144 runHook postInstall
145 '';
146
147 # A Scheme built for this machine and a kernel built for another one; the
148 # usual fixups have an opinion about both, and neither wants it.
149 dontStrip = true;
150 dontPatchELF = true;
151 };
152
153 hostScheme = "${chezAndroid}/ta6le/bin/ta6le/scheme";
154 targetBoot = "${chezAndroid}/boot/tarm64le";
155 xpatch = "${chezAndroid}/xc-tarm64le/s/xpatch";
156
157 # --- the Jolt half --------------------------------------------------------
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 17d ago158 # frq's Scheme, cross-compiled to an arm64 boot image. The deps.edn below is
159 # written out by hand rather than resolved: there is no dependency resolution
160 # inside a cross compile, so every source root deps.edn would have resolved is
161 # named as a :path instead.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago162 #
163 # The jolt that runs it is the fork, not upstream and not the one the desktop
164 # package builds: upstream reads the socket address out of `struct addrinfo`
165 # at glibc's offset, which on Bionic is `ai_canonname`, and an APK built with
166 # it cannot open a TLS connection at all.
167 joltBoot = pkgs.stdenv.mkDerivation {
168 pname = "frq-jolt-boot";
169 inherit version;
170
171 dontUnpack = true;
172 strictDeps = true;
173 nativeBuildInputs = [ joltAndroid ];
174
175 buildPhase = ''
176 runHook preBuild
177
178 export HOME="$TMPDIR"
179 mkdir -p project cross
180
181 cat > project/deps.edn <<EOF
182 {:paths ["${self}/src" "${glimmer}/src" "${jolt-native}/jolt/glimmer-vidya/src"]}
183 EOF
184
185 # The flat build, which is the one shape make-boot-file can take.
186 ( cd project && JOLT_NO_FLAT_SPLIT=1 jolt build -m frq.app -o app )
187
188 cat > cross/compile.ss <<EOF
189 (import (chezscheme))
190 (load "${xpatch}")
191 (optimize-level 2)
192 (generate-inspector-information #f)
Ship the APK smaller: strip the libraries, pack the image 7434fee nandi 18d ago193 ;; Packed harder, not packed for the first time: fasl output is
194 ;; compressed already, but with lz4 at its fastest setting, and on
195 ;; this image that leaves 2.3 MB on the table. What reads it back is
196 ;; the kernel linked into libjoltapp, which has zlib because
197 ;; chezAndroid is configured ZLIB=-lz — so nothing extra ships to
198 ;; decompress it.
199 ;;
200 ;; Less than the ratio of the whole file suggests (15.9 MB to 13.6):
201 ;; compression is per fasl entry rather than over the image.
202 (fasl-compressed #t)
203 (compress-format 'gzip)
204 (compress-level 'maximum)
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago205 (compile-file "$PWD/project/app.build/flat.ss" "$PWD/cross/flat.so")
206 (make-boot-file "$PWD/jolt.boot" '()
207 "${targetBoot}/petite.boot"
208 "${targetBoot}/scheme.boot"
209 "$PWD/cross/flat.so")
210 EOF
211
212 SCHEMEHEAPDIRS="${chezAndroid}/ta6le/boot/ta6le" \
213 ${hostScheme} --script cross/compile.ss
214
215 runHook postBuild
216 '';
217
218 # scheme.h travels with the image because jolt_main.c includes it.
219 installPhase = ''
220 runHook preInstall
221 mkdir -p "$out"
222 cp jolt.boot "$out/jolt.boot"
223 cp ${targetBoot}/scheme.h "$out/scheme.h"
224 runHook postInstall
225 '';
226 };
227
228 # The image travels as a blob in an object file's data section. The
229 # _binary_jolt_boot_{start,end} symbols jolt_main.c reads are named after the
230 # input *path*, so this copies the file somewhere it is called exactly
231 # `jolt.boot` before converting it.
232 joltBootObj = pkgs.runCommand "jolt-boot-obj" { } ''
233 cp ${joltBoot}/jolt.boot jolt.boot
234 ${ndkBin}/llvm-objcopy \
235 --input-target=binary --output-target=elf64-littleaarch64 \
236 --binary-architecture=aarch64 jolt.boot "$out"
237 '';
238
239 # The glue: jolt-native's jolt_main.c over the boot image, linked against
240 # libvidya by name. --no-undefined is what makes a symbol the Scheme side
241 # registers but the ABI no longer exports a build failure here rather than a
242 # crash on the phone.
243 libjoltapp = pkgs.runCommand "libjoltapp.so" { } ''
244 mkdir -p lib
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago245 cp ${nativeLibs}/lib/${abi}/libvidya.so lib/libvidya.so
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago246
Paint frq with jvui, and take the last Rust out of the window 77963dc nandi 9d ago247 # -DJOLT_WITHOUT_MOQ: the glue registers libjoltmoq's symbols for the
248 # Scheme side, and --no-undefined below turns each of those into a link
249 # error once the object is not linked. The define takes them out, which
250 # is the honest way to say "this app has no Rust media plane" rather
251 # than linking one to satisfy a reference nothing calls.
252 ${cc} -shared -fPIC -O2 -DJOLT_WITHOUT_MOQ -o "$out" \
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago253 ${jolt-native}/android/jolt_main.c \
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago254 ${joltBootObj} \
255 -I${joltBoot} \
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago256 -I${nativeLibs}/include \
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago257 -Llib \
258 ${chezAndroid}/tarm64le/boot/tarm64le/libkernel.a \
259 ${chezAndroid}/lz4/lib/liblz4.a \
Paint frq with jvui, and take the last Rust out of the window 77963dc nandi 9d ago260 -lvidya -landroid -llog -lz -ldl -lm -Wl,--no-undefined
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago261 '';
262
263 # --- the Java half --------------------------------------------------------
264 # One class: the photo chooser's result has to land somewhere, and native
265 # code is not somewhere.
266 classesDex = pkgs.runCommand "classes.dex"
267 {
268 nativeBuildInputs = [ pkgs.jdk17 ];
269 } ''
270 mkdir -p classes out
271 # -encoding, because a build sandbox has no locale and javac then reads
272 # the source as US-ASCII — on which the comments' em dashes are errors.
273 javac --release 17 -encoding UTF-8 --class-path ${androidJar} -d classes \
274 $(find ${self}/android/java -name '*.java')
275 ${buildTools}/d8 --min-api ${apiLevel} --output out $(find classes -name '*.class')
276 cp out/classes.dex "$out"
277 '';
278
279 # --- the package ----------------------------------------------------------
280 # OpenSSL travels with the app because the platform's own is not ours to
281 # load: an app's linker namespace refuses /system/lib64/libssl.so, and
282 # without one there is no TLS on the phone at all.
283 # Built by the NDK rather than by pkgsCross.aarch64-android: that cross
284 # stdenv cannot build its own compiler-rt on this nixpkgs — os_version_check.c
285 # includes <pthread.h> and the sysroot it is handed has no such header — and
286 # an APK has no use for a second toolchain anyway. OpenSSL's own android-arm64
287 # target wants the NDK's llvm on PATH and takes the API level from the flag.
288 # The version is the one ~/.cache/frq-openssl-android was built from.
289 opensslAndroid = pkgs.stdenv.mkDerivation {
290 pname = "openssl-android";
291 version = "3.5.4";
292
293 src = pkgs.fetchurl {
294 url = "https://github.com/openssl/openssl/releases/download/openssl-3.5.4/openssl-3.5.4.tar.gz";
295 sha256 = "16ay6ppxsky3qhg6573370iz93kihfwx9n5ipmlnjcam97w12wwn";
296 };
297
298 strictDeps = true;
299 nativeBuildInputs = with pkgs; [ perl ];
300
301 configurePhase = ''
302 runHook preConfigure
303 export ANDROID_NDK_ROOT="${ndkRoot}"
304 export PATH="${ndkBin}:$PATH"
305 # Through perl rather than as a program: its shebang is /usr/bin/env,
306 # which a build sandbox does not have.
307 perl ./Configure android-arm64 -D__ANDROID_API__=${apiLevel} \
308 shared no-tests no-docs \
309 --prefix="$out" --openssldir="$out/etc/ssl"
310 runHook postConfigure
311 '';
312
313 # install_sw, not install: the rest of an OpenSSL install is for a machine
314 # that runs it, and this one only ships two .so files into an APK.
315 installTargets = [ "install_sw" ];
316
317 dontStrip = true;
318 dontPatchELF = true;
319 };
320
321 # The libraries are stored rather than deflated: the loader maps them
322 # straight out of the APK. The dex is read rather than mapped, so it may as
323 # well compress.
324 apkUnsigned = pkgs.runCommand "frq-unsigned.apk"
325 {
326 nativeBuildInputs = [ pkgs.zip ];
327 } ''
328 mkdir -p stage/lib/${abi}
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago329 cp ${nativeLibs}/lib/${abi}/libvidya.so stage/lib/${abi}/libvidya.so
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago330 cp ${libjoltapp} stage/lib/${abi}/libjoltapp.so
331 cp ${opensslAndroid.out}/lib/libssl.so stage/lib/${abi}/libssl.so
332 cp ${opensslAndroid.out}/lib/libcrypto.so stage/lib/${abi}/libcrypto.so
333 cp ${classesDex} stage/classes.dex
334 chmod -R u+w stage
335
Ship the APK smaller: strip the libraries, pack the image 7434fee nandi 18d ago336 # Everything the loader needs is in .dynsym, and that is what --strip-all
337 # keeps: what goes is .symtab and the debug sections, which are read by a
Paint frq with jvui, and take the last Rust out of the window 77963dc nandi 9d ago338 # debugger and by nothing on the phone. Worth less than it used to be —
339 # libjoltmoq and libc++_shared were most of it and are no longer packed —
340 # but the release libraries still arrive unstripped, because jolt-native's
341 # own build does not strip them.
Ship the APK smaller: strip the libraries, pack the image 7434fee nandi 18d ago342 #
343 # Here rather than in the derivations that produce them: the inputs stay
344 # whole (a stripped libjoltapp is a worse thing to hand a debugger, and
345 # `nix build .#libjoltapp` is how it is looked at), and this is the one
346 # place that knows the difference between an object and a shipped one.
347 # The NDK's, not nixpkgs' — the host strip has no opinion worth trusting
348 # about an arm64 object.
349 ${ndkBin}/llvm-strip --strip-all stage/lib/${abi}/*.so
350
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago351 ${buildTools}/aapt2 link -o "$out" -I ${androidJar} \
352 --manifest ${self}/android/AndroidManifest.xml \
353 --min-sdk-version ${apiLevel} --target-sdk-version ${targetSdk} \
354 --version-code 1 --version-name ${version}
355
356 ( cd stage && \
Paint frq with jvui, and take the last Rust out of the window 77963dc nandi 9d ago357 zip -q -0 "$out" lib/${abi}/libvidya.so lib/${abi}/libjoltapp.so \
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago358 lib/${abi}/libssl.so lib/${abi}/libcrypto.so && \
359 zip -q "$out" classes.dex )
360 '';
361
362 # Aligned and signed with a debug key. The key is generated here rather than
363 # read from ~/.android, which is the one place this build is deliberately
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 17d ago364 # not what the builds before it did: a keystore outside the store would make
365 # the output
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago366 # depend on the machine, and a release key has no business in the store at
367 # all. So this output is installable and not reproducible — keytool stamps
368 # the certificate with the time — and anything meant for a store should be
369 # signed from .#apk-unsigned instead.
370 apk = pkgs.runCommand "frq-${version}.apk"
371 {
372 nativeBuildInputs = [ pkgs.jdk17 ];
373 meta = {
374 description = "frq for Android, debug-signed";
375 platforms = [ "x86_64-linux" ];
376 };
377 } ''
378 export HOME="$TMPDIR"
379 keytool -genkeypair -keystore debug.keystore \
380 -storepass android -keypass android -alias androiddebugkey \
381 -keyalg RSA -keysize 2048 -validity 10000 \
382 -dname 'CN=Android Debug,O=Android,C=US'
383
384 ${buildTools}/zipalign -f -p 4 ${apkUnsigned} aligned.apk
385 ${buildTools}/apksigner sign --ks debug.keystore \
386 --ks-key-alias androiddebugkey \
387 --ks-pass pass:android --key-pass pass:android \
388 --out "$out" aligned.apk
389 ${buildTools}/apksigner verify "$out"
390 '';
391in
392{
393 inherit chezAndroid joltBoot libjoltapp classesDex apk;
394 apk-unsigned = apkUnsigned;
395}