rickub
nandi/frqpublic⑂ Fork 0
⑂ 472e179b3bf7b6b6b58384036615ee6693c0f9d7
Branches
claude/add-remote-repository-47d12e main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/frq.git
git clone ssh://git@rickub.com/nandi/frq.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

android.nix · 397 lines · 16.8 KBNix Blame HistoryRaw
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago1# The APK, as derivations.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago2#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago3# The only APK build there is: this replaced a buck2 graph and the script
4# before it, both of which named an Android SDK, an NDK, a hand-built Chez
5# cross target and an OpenSSL by absolute path and stopped when one was
6# missing. Every one of those is built or fetched here instead, so this works
7# from nothing on a machine with none of them and no ~/.cache at all:
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago8#
9# nix build .#apk
10#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago11# `just apk` is the same build with the store path handed to adb afterwards;
12# see scripts/apk.bb.
13#
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago14# On a machine with a remote builder configured, prefer
15#
16# nix build .#apk --store ssh-ng://eu.nixbuild.net --eval-store auto
17#
18# rather than letting `builders` do it. With `builders`, nix copies the output
19# of every remotely-built derivation back, and androidenv's NDK is both
20# `preferLocalBuild` and absent from cache.nixos.org — so the 3.1 GB unpacked
21# toolchain is built here and uploaded. With the remote as the *store* the
22# whole graph stays there, only .drv files go up, and the builder fetches
23# Google's zip over its own link.
24#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago25# The steps are the ones the graph before it ran, in the same order; where a
26# genrule read a path out of `read_root_config`, a derivation takes an
27# argument.
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago28{ pkgs, lib, self, chez-src, jolt-native, jolt-native-android, glimmer, joltAndroid, androidSdk, ndk }:
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago29
30let
31 # What the APK targets, in the three spellings the tools want it in.
32 apiLevel = "28";
33 targetSdk = "36";
34 abi = "arm64-v8a";
35 package = "uk.nandi.frq";
36 version = "0.1.0";
37
38 sdk = "${androidSdk}/libexec/android-sdk";
39 buildTools = "${sdk}/build-tools/36.0.0";
40 androidJar = "${sdk}/platforms/android-${targetSdk}/android.jar";
41
42 # The NDK's clang finds its sysroot, resource directory and the rest of LLVM
43 # relative to itself, so it is named by path rather than copied anywhere.
44 # androidenv installs the tree under libexec/android-sdk and leaves
45 # `ndk-bundle` pointing at the versioned directory beside it.
46 ndkRoot = "${ndk}/libexec/android-sdk/ndk-bundle";
47 ndkBin = "${ndkRoot}/toolchains/llvm/prebuilt/linux-x86_64/bin";
48 cc = "${ndkBin}/aarch64-linux-android${apiLevel}-clang";
49
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago50 # What comes out of jolt-native's CI, as a flake input rather than a fetchurl:
51 # nix unpacks a tarball input itself, and flake.lock holds the digest that
52 # used to be written here by hand. `just bump` moves it with
53 # `nix flake update jolt-native-android` and nothing in this file changes.
Ask for the camera and the microphone from the app 6c745df nandi 18d ago54 #
55 # One archive, two libraries: libvidya (the retained-tree UI) and libjoltmoq
56 # (the AV media plane). They are built together and only make sense together
57 # — libjoltapp links both — so there is one pin for the pair rather than two
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago58 # that could drift apart. It carries include/ and libc++_shared.so besides,
59 # which is why the glue's headers and the C++ runtime come from here too.
60 nativeLibs = jolt-native-android;
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago61
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago62 # The C++ runtime, out of the archive rather than the NDK composed here.
Ask for the camera and the microphone from the app 6c745df nandi 18d ago63 #
64 # openh264 is C++, and its build script asks to be linked against
65 # `libc++_shared.so` by name — so libjoltmoq carries that as a DT_NEEDED. An
66 # app's linker namespace will not hand out the platform's own copy (there is
67 # no stable one to hand out), so the APK carries it, exactly as it carries
68 # OpenSSL below and for the same reason.
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago69 #
70 # jolt-native ships it beside the libraries that need it, so this is the copy
71 # they were actually linked against — where the NDK path was whichever
72 # revision `composeAndroidPackages` happened to resolve here.
73 libcxx = "${nativeLibs}/lib/${abi}/libc++_shared.so";
Ask for the camera and the microphone from the app 6c745df nandi 18d ago74
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago75 # --- Chez's arm64 cross target ------------------------------------------
76 # The one piece with no nixpkgs equivalent: `pkgs.chez` builds a Scheme for
77 # this machine, and what the boot image needs is Chez's `tarm64le` workarea —
78 # the target boot files, the cross compiler's xpatch, and the arm64
79 # libkernel.a that libjoltapp links.
80 #
81 # Three builds in one derivation, because each needs the one before it:
82 #
83 # ta6le a host Scheme, which is what cross-compiles anything
84 # boot XM=... the target's boot files and xpatch, made by that host
85 # tarm64le the target kernel, compiled by the NDK
86 #
87 # The flags are the ones the hand-built tree under ~/.cache/vidya-chez-android
88 # was configured with, read back out of its Mf-config. zlib is Android's own
89 # (`-lz`, which Bionic has); lz4 is the in-tree submodule, built for arm64
90 # here because a cross configure links it rather than building it.
91 chezAndroid = pkgs.stdenv.mkDerivation {
92 pname = "chez-scheme-android";
93 version = "10.4.1";
94 src = chez-src;
95
96 strictDeps = true;
97 nativeBuildInputs = with pkgs; [ gnumake which ];
98
99 dontConfigure = true;
100
101 buildPhase = ''
102 runHook preBuild
103
104 # Both workareas turn off the expression editor's two dependencies, which
105 # is what ~/.cache/vidya-chez-android was configured with (its Mf-config
106 # has empty cursesLib/ncursesLib, and disablex11=yes on the target). The
107 # host Scheme here is only ever a cross compiler, and Bionic has no
108 # curses.h at all — so on the target it is not a preference but a
109 # requirement.
110 ./configure -m=ta6le --disable-x11 --disable-curses CC_FOR_BUILD="$CC"
111 make -j"$NIX_BUILD_CORES"
112 make boot XM=tarm64le -j"$NIX_BUILD_CORES"
113
114 # lz4 for the phone, not for this machine: the host build above left an
115 # x86_64 liblz4.a in the same place, and the cross link needs it gone.
116 make -C lz4/lib clean
117 make -C lz4/lib liblz4.a -j"$NIX_BUILD_CORES" \
118 CC=${cc} AR=${ndkBin}/llvm-ar
119
120 # --disable-auto-flags stops configure appending -lrt and -lpthread, which
121 # is what its unix branch does for a glibc host and what Bionic has no
122 # separate libraries for — both live in libc there. Everything it would
123 # otherwise add is passed explicitly below, matching the Mf-config of
124 # the tree this was reconstructed from.
125 ./configure -m=tarm64le --cross --disable-x11 --disable-curses \
126 --disable-auto-flags \
127 LIBS="-ldl -lm" \
128 CC=${cc} \
129 AR=${ndkBin}/llvm-ar \
130 CC_FOR_BUILD="$CC" \
131 ZLIB=-lz \
132 LZ4="$PWD/lz4/lib/liblz4.a" \
133 CPPFLAGS="-I$PWD/lz4/lib" \
134 CFLAGS="-O2 -D_REENTRANT -pthread -fPIC"
135 make -j"$NIX_BUILD_CORES"
136
137 runHook postBuild
138 '';
139
140 # The whole workarea, at the paths CHEZ_ANDROID means: the host Scheme
141 # loads xpatch out of xc-tarm64le/s, and the link below reads two archives
142 # from elsewhere in the tree. Pruning it would only be guessing at which
143 # of those the cross compiler still opens.
144 installPhase = ''
145 runHook preInstall
146 mkdir -p "$out"
147 cp -r . "$out/"
148 runHook postInstall
149 '';
150
151 # A Scheme built for this machine and a kernel built for another one; the
152 # usual fixups have an opinion about both, and neither wants it.
153 dontStrip = true;
154 dontPatchELF = true;
155 };
156
157 hostScheme = "${chezAndroid}/ta6le/bin/ta6le/scheme";
158 targetBoot = "${chezAndroid}/boot/tarm64le";
159 xpatch = "${chezAndroid}/xc-tarm64le/s/xpatch";
160
161 # --- the Jolt half --------------------------------------------------------
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago162 # frq's Scheme, cross-compiled to an arm64 boot image. The deps.edn below is
163 # written out by hand rather than resolved: there is no dependency resolution
164 # inside a cross compile, so every source root deps.edn would have resolved is
165 # named as a :path instead.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago166 #
167 # The jolt that runs it is the fork, not upstream and not the one the desktop
168 # package builds: upstream reads the socket address out of `struct addrinfo`
169 # at glibc's offset, which on Bionic is `ai_canonname`, and an APK built with
170 # it cannot open a TLS connection at all.
171 joltBoot = pkgs.stdenv.mkDerivation {
172 pname = "frq-jolt-boot";
173 inherit version;
174
175 dontUnpack = true;
176 strictDeps = true;
177 nativeBuildInputs = [ joltAndroid ];
178
179 buildPhase = ''
180 runHook preBuild
181
182 export HOME="$TMPDIR"
183 mkdir -p project cross
184
185 cat > project/deps.edn <<EOF
186 {:paths ["${self}/src" "${glimmer}/src" "${jolt-native}/jolt/glimmer-vidya/src"]}
187 EOF
188
189 # The flat build, which is the one shape make-boot-file can take.
190 ( cd project && JOLT_NO_FLAT_SPLIT=1 jolt build -m frq.app -o app )
191
192 cat > cross/compile.ss <<EOF
193 (import (chezscheme))
194 (load "${xpatch}")
195 (optimize-level 2)
196 (generate-inspector-information #f)
Ship the APK smaller: strip the libraries, pack the image 7434fee nandi 18d ago197 ;; Packed harder, not packed for the first time: fasl output is
198 ;; compressed already, but with lz4 at its fastest setting, and on
199 ;; this image that leaves 2.3 MB on the table. What reads it back is
200 ;; the kernel linked into libjoltapp, which has zlib because
201 ;; chezAndroid is configured ZLIB=-lz — so nothing extra ships to
202 ;; decompress it.
203 ;;
204 ;; Less than the ratio of the whole file suggests (15.9 MB to 13.6):
205 ;; compression is per fasl entry rather than over the image.
206 (fasl-compressed #t)
207 (compress-format 'gzip)
208 (compress-level 'maximum)
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago209 (compile-file "$PWD/project/app.build/flat.ss" "$PWD/cross/flat.so")
210 (make-boot-file "$PWD/jolt.boot" '()
211 "${targetBoot}/petite.boot"
212 "${targetBoot}/scheme.boot"
213 "$PWD/cross/flat.so")
214 EOF
215
216 SCHEMEHEAPDIRS="${chezAndroid}/ta6le/boot/ta6le" \
217 ${hostScheme} --script cross/compile.ss
218
219 runHook postBuild
220 '';
221
222 # scheme.h travels with the image because jolt_main.c includes it.
223 installPhase = ''
224 runHook preInstall
225 mkdir -p "$out"
226 cp jolt.boot "$out/jolt.boot"
227 cp ${targetBoot}/scheme.h "$out/scheme.h"
228 runHook postInstall
229 '';
230 };
231
232 # The image travels as a blob in an object file's data section. The
233 # _binary_jolt_boot_{start,end} symbols jolt_main.c reads are named after the
234 # input *path*, so this copies the file somewhere it is called exactly
235 # `jolt.boot` before converting it.
236 joltBootObj = pkgs.runCommand "jolt-boot-obj" { } ''
237 cp ${joltBoot}/jolt.boot jolt.boot
238 ${ndkBin}/llvm-objcopy \
239 --input-target=binary --output-target=elf64-littleaarch64 \
240 --binary-architecture=aarch64 jolt.boot "$out"
241 '';
242
243 # The glue: jolt-native's jolt_main.c over the boot image, linked against
244 # libvidya by name. --no-undefined is what makes a symbol the Scheme side
245 # registers but the ABI no longer exports a build failure here rather than a
246 # crash on the phone.
247 libjoltapp = pkgs.runCommand "libjoltapp.so" { } ''
248 mkdir -p lib
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago249 cp ${nativeLibs}/lib/${abi}/libvidya.so lib/libvidya.so
250 cp ${nativeLibs}/lib/${abi}/libjoltmoq.so lib/libjoltmoq.so
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago251
252 ${cc} -shared -fPIC -O2 -o "$out" \
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago253 ${jolt-native}/android/jolt_main.c \
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago254 ${joltBootObj} \
255 -I${joltBoot} \
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago256 -I${nativeLibs}/include \
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago257 -Llib \
258 ${chezAndroid}/tarm64le/boot/tarm64le/libkernel.a \
259 ${chezAndroid}/lz4/lib/liblz4.a \
Ask for the camera and the microphone from the app 6c745df nandi 18d ago260 -lvidya -ljoltmoq -landroid -llog -lz -ldl -lm -Wl,--no-undefined
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago261 '';
262
263 # --- the Java half --------------------------------------------------------
264 # One class: the photo chooser's result has to land somewhere, and native
265 # code is not somewhere.
266 classesDex = pkgs.runCommand "classes.dex"
267 {
268 nativeBuildInputs = [ pkgs.jdk17 ];
269 } ''
270 mkdir -p classes out
271 # -encoding, because a build sandbox has no locale and javac then reads
272 # the source as US-ASCII — on which the comments' em dashes are errors.
273 javac --release 17 -encoding UTF-8 --class-path ${androidJar} -d classes \
274 $(find ${self}/android/java -name '*.java')
275 ${buildTools}/d8 --min-api ${apiLevel} --output out $(find classes -name '*.class')
276 cp out/classes.dex "$out"
277 '';
278
279 # --- the package ----------------------------------------------------------
280 # OpenSSL travels with the app because the platform's own is not ours to
281 # load: an app's linker namespace refuses /system/lib64/libssl.so, and
282 # without one there is no TLS on the phone at all.
283 # Built by the NDK rather than by pkgsCross.aarch64-android: that cross
284 # stdenv cannot build its own compiler-rt on this nixpkgs — os_version_check.c
285 # includes <pthread.h> and the sysroot it is handed has no such header — and
286 # an APK has no use for a second toolchain anyway. OpenSSL's own android-arm64
287 # target wants the NDK's llvm on PATH and takes the API level from the flag.
288 # The version is the one ~/.cache/frq-openssl-android was built from.
289 opensslAndroid = pkgs.stdenv.mkDerivation {
290 pname = "openssl-android";
291 version = "3.5.4";
292
293 src = pkgs.fetchurl {
294 url = "https://github.com/openssl/openssl/releases/download/openssl-3.5.4/openssl-3.5.4.tar.gz";
295 sha256 = "16ay6ppxsky3qhg6573370iz93kihfwx9n5ipmlnjcam97w12wwn";
296 };
297
298 strictDeps = true;
299 nativeBuildInputs = with pkgs; [ perl ];
300
301 configurePhase = ''
302 runHook preConfigure
303 export ANDROID_NDK_ROOT="${ndkRoot}"
304 export PATH="${ndkBin}:$PATH"
305 # Through perl rather than as a program: its shebang is /usr/bin/env,
306 # which a build sandbox does not have.
307 perl ./Configure android-arm64 -D__ANDROID_API__=${apiLevel} \
308 shared no-tests no-docs \
309 --prefix="$out" --openssldir="$out/etc/ssl"
310 runHook postConfigure
311 '';
312
313 # install_sw, not install: the rest of an OpenSSL install is for a machine
314 # that runs it, and this one only ships two .so files into an APK.
315 installTargets = [ "install_sw" ];
316
317 dontStrip = true;
318 dontPatchELF = true;
319 };
320
321 # The libraries are stored rather than deflated: the loader maps them
322 # straight out of the APK. The dex is read rather than mapped, so it may as
323 # well compress.
324 apkUnsigned = pkgs.runCommand "frq-unsigned.apk"
325 {
326 nativeBuildInputs = [ pkgs.zip ];
327 } ''
328 mkdir -p stage/lib/${abi}
Take the Android objects from jolt-native's CI, always the latest 472e179 nandi 11d ago329 cp ${nativeLibs}/lib/${abi}/libvidya.so stage/lib/${abi}/libvidya.so
330 cp ${nativeLibs}/lib/${abi}/libjoltmoq.so stage/lib/${abi}/libjoltmoq.so
Ask for the camera and the microphone from the app 6c745df nandi 18d ago331 cp ${libcxx} stage/lib/${abi}/libc++_shared.so
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago332 cp ${libjoltapp} stage/lib/${abi}/libjoltapp.so
333 cp ${opensslAndroid.out}/lib/libssl.so stage/lib/${abi}/libssl.so
334 cp ${opensslAndroid.out}/lib/libcrypto.so stage/lib/${abi}/libcrypto.so
335 cp ${classesDex} stage/classes.dex
336 chmod -R u+w stage
337
Ship the APK smaller: strip the libraries, pack the image 7434fee nandi 18d ago338 # Everything the loader needs is in .dynsym, and that is what --strip-all
339 # keeps: what goes is .symtab and the debug sections, which are read by a
340 # debugger and by nothing on the phone. Worth about a third of the package
341 # — libjoltmoq and libc++_shared are most of it, and the release libraries
342 # arrive unstripped because jolt-native's own build does not strip them.
343 #
344 # Here rather than in the derivations that produce them: the inputs stay
345 # whole (a stripped libjoltapp is a worse thing to hand a debugger, and
346 # `nix build .#libjoltapp` is how it is looked at), and this is the one
347 # place that knows the difference between an object and a shipped one.
348 # The NDK's, not nixpkgs' — the host strip has no opinion worth trusting
349 # about an arm64 object.
350 ${ndkBin}/llvm-strip --strip-all stage/lib/${abi}/*.so
351
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago352 ${buildTools}/aapt2 link -o "$out" -I ${androidJar} \
353 --manifest ${self}/android/AndroidManifest.xml \
354 --min-sdk-version ${apiLevel} --target-sdk-version ${targetSdk} \
355 --version-code 1 --version-name ${version}
356
357 ( cd stage && \
Ask for the camera and the microphone from the app 6c745df nandi 18d ago358 zip -q -0 "$out" lib/${abi}/libvidya.so lib/${abi}/libjoltmoq.so \
359 lib/${abi}/libc++_shared.so lib/${abi}/libjoltapp.so \
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago360 lib/${abi}/libssl.so lib/${abi}/libcrypto.so && \
361 zip -q "$out" classes.dex )
362 '';
363
364 # Aligned and signed with a debug key. The key is generated here rather than
365 # read from ~/.android, which is the one place this build is deliberately
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago366 # not what the builds before it did: a keystore outside the store would make
367 # the output
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago368 # depend on the machine, and a release key has no business in the store at
369 # all. So this output is installable and not reproducible — keytool stamps
370 # the certificate with the time — and anything meant for a store should be
371 # signed from .#apk-unsigned instead.
372 apk = pkgs.runCommand "frq-${version}.apk"
373 {
374 nativeBuildInputs = [ pkgs.jdk17 ];
375 meta = {
376 description = "frq for Android, debug-signed";
377 platforms = [ "x86_64-linux" ];
378 };
379 } ''
380 export HOME="$TMPDIR"
381 keytool -genkeypair -keystore debug.keystore \
382 -storepass android -keypass android -alias androiddebugkey \
383 -keyalg RSA -keysize 2048 -validity 10000 \
384 -dname 'CN=Android Debug,O=Android,C=US'
385
386 ${buildTools}/zipalign -f -p 4 ${apkUnsigned} aligned.apk
387 ${buildTools}/apksigner sign --ks debug.keystore \
388 --ks-key-alias androiddebugkey \
389 --ks-pass pass:android --key-pass pass:android \
390 --out "$out" aligned.apk
391 ${buildTools}/apksigner verify "$out"
392 '';
393in
394{
395 inherit chezAndroid joltBoot libjoltapp classesDex apk;
396 apk-unsigned = apkUnsigned;
397}