rickub
nandi/frqpublic⑂ Fork 0
⑂ 230bbef06d4061e8e7bba28fab2aefff9d1213e2
Branches
claude/add-remote-repository-47d12e main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/frq.git
git clone ssh://git@rickub.com/nandi/frq.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

android.nix · 413 lines · 17.3 KBNix Blame HistoryRaw
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago1# The APK, as derivations.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago2#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago3# The only APK build there is: this replaced a buck2 graph and the script
4# before it, both of which named an Android SDK, an NDK, a hand-built Chez
5# cross target and an OpenSSL by absolute path and stopped when one was
6# missing. Every one of those is built or fetched here instead, so this works
7# from nothing on a machine with none of them and no ~/.cache at all:
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago8#
9# nix build .#apk
10#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago11# `just apk` is the same build with the store path handed to adb afterwards;
12# see scripts/apk.bb.
13#
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago14# On a machine with a remote builder configured, prefer
15#
16# nix build .#apk --store ssh-ng://eu.nixbuild.net --eval-store auto
17#
18# rather than letting `builders` do it. With `builders`, nix copies the output
19# of every remotely-built derivation back, and androidenv's NDK is both
20# `preferLocalBuild` and absent from cache.nixos.org — so the 3.1 GB unpacked
21# toolchain is built here and uploaded. With the remote as the *store* the
22# whole graph stays there, only .drv files go up, and the builder fetches
23# Google's zip over its own link.
24#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago25# The steps are the ones the graph before it ran, in the same order; where a
26# genrule read a path out of `read_root_config`, a derivation takes an
27# argument.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago28{ pkgs, lib, self, chez-src, jolt-native, glimmer, joltAndroid, androidSdk, ndk }:
29
30let
31 # What the APK targets, in the three spellings the tools want it in.
32 apiLevel = "28";
33 targetSdk = "36";
34 abi = "arm64-v8a";
35 package = "uk.nandi.frq";
36 version = "0.1.0";
37
38 sdk = "${androidSdk}/libexec/android-sdk";
39 buildTools = "${sdk}/build-tools/36.0.0";
40 androidJar = "${sdk}/platforms/android-${targetSdk}/android.jar";
41
42 # The NDK's clang finds its sysroot, resource directory and the rest of LLVM
43 # relative to itself, so it is named by path rather than copied anywhere.
44 # androidenv installs the tree under libexec/android-sdk and leaves
45 # `ndk-bundle` pointing at the versioned directory beside it.
46 ndkRoot = "${ndk}/libexec/android-sdk/ndk-bundle";
47 ndkBin = "${ndkRoot}/toolchains/llvm/prebuilt/linux-x86_64/bin";
48 cc = "${ndkBin}/aarch64-linux-android${apiLevel}-clang";
49
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago50 # What comes out of jolt-native's releases. These two pins are the Android
51 # half of what `just bump` moves — the desktop half stays in
52 # scripts/*.dotslash, and the tag written here is the one written there, so
53 # a phone and a laptop run the same release. Do not edit them by hand:
54 # bump-jolt-native.bb fetches each archive, weighs it, and writes both the
55 # url and the digest below.
Ask for the camera and the microphone from the app 6c745df nandi 18d ago56 #
57 # One archive, two libraries: libvidya (the retained-tree UI) and libjoltmoq
58 # (the AV media plane). They are built together and only make sense together
59 # — libjoltapp links both — so there is one pin for the pair rather than two
60 # that could drift apart.
61 nativeRelease = pkgs.fetchurl {
62 url = "https://gitlab.com/nandithebull/jolt-native/-/releases/v0.1.3/downloads/jolt-native-android-arm64-v0.1.3.tar.gz";
63 sha256 = "4519745bae6db9a791a71b38a26478879e246166802b0a48dbf7d45e4d45a108";
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago64 };
65
66 glue = pkgs.fetchurl {
Ask for the camera and the microphone from the app 6c745df nandi 18d ago67 url = "https://gitlab.com/nandithebull/jolt-native/-/releases/v0.1.3/downloads/jolt-native-android-glue-v0.1.3.tar.gz";
68 sha256 = "83313eda124f2a0cfff6827cf4473600c1f71db2f208d654b97068a85af38da5";
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago69 };
70
Ask for the camera and the microphone from the app 6c745df nandi 18d ago71 # Unpacked once, so the consumers below name files rather than repeat the tar.
72 nativeLibs = pkgs.runCommand "jolt-native-android" { } ''
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago73 mkdir -p "$out"
Ask for the camera and the microphone from the app 6c745df nandi 18d ago74 tar -xzf ${nativeRelease} -C "$out"
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago75 '';
76
77 glueSrc = pkgs.runCommand "jolt-android-glue" { } ''
78 mkdir -p "$out"
79 tar -xzf ${glue} -C "$out" --strip-components=1
80 '';
81
Ask for the camera and the microphone from the app 6c745df nandi 18d ago82 # The C++ runtime, out of the same NDK the glue is compiled with.
83 #
84 # openh264 is C++, and its build script asks to be linked against
85 # `libc++_shared.so` by name — so libjoltmoq carries that as a DT_NEEDED. An
86 # app's linker namespace will not hand out the platform's own copy (there is
87 # no stable one to hand out), so the APK carries it, exactly as it carries
88 # OpenSSL below and for the same reason.
89 libcxx = "${ndkRoot}/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so";
90
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago91 # --- Chez's arm64 cross target ------------------------------------------
92 # The one piece with no nixpkgs equivalent: `pkgs.chez` builds a Scheme for
93 # this machine, and what the boot image needs is Chez's `tarm64le` workarea —
94 # the target boot files, the cross compiler's xpatch, and the arm64
95 # libkernel.a that libjoltapp links.
96 #
97 # Three builds in one derivation, because each needs the one before it:
98 #
99 # ta6le a host Scheme, which is what cross-compiles anything
100 # boot XM=... the target's boot files and xpatch, made by that host
101 # tarm64le the target kernel, compiled by the NDK
102 #
103 # The flags are the ones the hand-built tree under ~/.cache/vidya-chez-android
104 # was configured with, read back out of its Mf-config. zlib is Android's own
105 # (`-lz`, which Bionic has); lz4 is the in-tree submodule, built for arm64
106 # here because a cross configure links it rather than building it.
107 chezAndroid = pkgs.stdenv.mkDerivation {
108 pname = "chez-scheme-android";
109 version = "10.4.1";
110 src = chez-src;
111
112 strictDeps = true;
113 nativeBuildInputs = with pkgs; [ gnumake which ];
114
115 dontConfigure = true;
116
117 buildPhase = ''
118 runHook preBuild
119
120 # Both workareas turn off the expression editor's two dependencies, which
121 # is what ~/.cache/vidya-chez-android was configured with (its Mf-config
122 # has empty cursesLib/ncursesLib, and disablex11=yes on the target). The
123 # host Scheme here is only ever a cross compiler, and Bionic has no
124 # curses.h at all — so on the target it is not a preference but a
125 # requirement.
126 ./configure -m=ta6le --disable-x11 --disable-curses CC_FOR_BUILD="$CC"
127 make -j"$NIX_BUILD_CORES"
128 make boot XM=tarm64le -j"$NIX_BUILD_CORES"
129
130 # lz4 for the phone, not for this machine: the host build above left an
131 # x86_64 liblz4.a in the same place, and the cross link needs it gone.
132 make -C lz4/lib clean
133 make -C lz4/lib liblz4.a -j"$NIX_BUILD_CORES" \
134 CC=${cc} AR=${ndkBin}/llvm-ar
135
136 # --disable-auto-flags stops configure appending -lrt and -lpthread, which
137 # is what its unix branch does for a glibc host and what Bionic has no
138 # separate libraries for — both live in libc there. Everything it would
139 # otherwise add is passed explicitly below, matching the Mf-config of
140 # the tree this was reconstructed from.
141 ./configure -m=tarm64le --cross --disable-x11 --disable-curses \
142 --disable-auto-flags \
143 LIBS="-ldl -lm" \
144 CC=${cc} \
145 AR=${ndkBin}/llvm-ar \
146 CC_FOR_BUILD="$CC" \
147 ZLIB=-lz \
148 LZ4="$PWD/lz4/lib/liblz4.a" \
149 CPPFLAGS="-I$PWD/lz4/lib" \
150 CFLAGS="-O2 -D_REENTRANT -pthread -fPIC"
151 make -j"$NIX_BUILD_CORES"
152
153 runHook postBuild
154 '';
155
156 # The whole workarea, at the paths CHEZ_ANDROID means: the host Scheme
157 # loads xpatch out of xc-tarm64le/s, and the link below reads two archives
158 # from elsewhere in the tree. Pruning it would only be guessing at which
159 # of those the cross compiler still opens.
160 installPhase = ''
161 runHook preInstall
162 mkdir -p "$out"
163 cp -r . "$out/"
164 runHook postInstall
165 '';
166
167 # A Scheme built for this machine and a kernel built for another one; the
168 # usual fixups have an opinion about both, and neither wants it.
169 dontStrip = true;
170 dontPatchELF = true;
171 };
172
173 hostScheme = "${chezAndroid}/ta6le/bin/ta6le/scheme";
174 targetBoot = "${chezAndroid}/boot/tarm64le";
175 xpatch = "${chezAndroid}/xc-tarm64le/s/xpatch";
176
177 # --- the Jolt half --------------------------------------------------------
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago178 # frq's Scheme, cross-compiled to an arm64 boot image. The deps.edn below is
179 # written out by hand rather than resolved: there is no dependency resolution
180 # inside a cross compile, so every source root deps.edn would have resolved is
181 # named as a :path instead.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago182 #
183 # The jolt that runs it is the fork, not upstream and not the one the desktop
184 # package builds: upstream reads the socket address out of `struct addrinfo`
185 # at glibc's offset, which on Bionic is `ai_canonname`, and an APK built with
186 # it cannot open a TLS connection at all.
187 joltBoot = pkgs.stdenv.mkDerivation {
188 pname = "frq-jolt-boot";
189 inherit version;
190
191 dontUnpack = true;
192 strictDeps = true;
193 nativeBuildInputs = [ joltAndroid ];
194
195 buildPhase = ''
196 runHook preBuild
197
198 export HOME="$TMPDIR"
199 mkdir -p project cross
200
201 cat > project/deps.edn <<EOF
202 {:paths ["${self}/src" "${glimmer}/src" "${jolt-native}/jolt/glimmer-vidya/src"]}
203 EOF
204
205 # The flat build, which is the one shape make-boot-file can take.
206 ( cd project && JOLT_NO_FLAT_SPLIT=1 jolt build -m frq.app -o app )
207
208 cat > cross/compile.ss <<EOF
209 (import (chezscheme))
210 (load "${xpatch}")
211 (optimize-level 2)
212 (generate-inspector-information #f)
Ship the APK smaller: strip the libraries, pack the image 7434fee nandi 18d ago213 ;; Packed harder, not packed for the first time: fasl output is
214 ;; compressed already, but with lz4 at its fastest setting, and on
215 ;; this image that leaves 2.3 MB on the table. What reads it back is
216 ;; the kernel linked into libjoltapp, which has zlib because
217 ;; chezAndroid is configured ZLIB=-lz — so nothing extra ships to
218 ;; decompress it.
219 ;;
220 ;; Less than the ratio of the whole file suggests (15.9 MB to 13.6):
221 ;; compression is per fasl entry rather than over the image.
222 (fasl-compressed #t)
223 (compress-format 'gzip)
224 (compress-level 'maximum)
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago225 (compile-file "$PWD/project/app.build/flat.ss" "$PWD/cross/flat.so")
226 (make-boot-file "$PWD/jolt.boot" '()
227 "${targetBoot}/petite.boot"
228 "${targetBoot}/scheme.boot"
229 "$PWD/cross/flat.so")
230 EOF
231
232 SCHEMEHEAPDIRS="${chezAndroid}/ta6le/boot/ta6le" \
233 ${hostScheme} --script cross/compile.ss
234
235 runHook postBuild
236 '';
237
238 # scheme.h travels with the image because jolt_main.c includes it.
239 installPhase = ''
240 runHook preInstall
241 mkdir -p "$out"
242 cp jolt.boot "$out/jolt.boot"
243 cp ${targetBoot}/scheme.h "$out/scheme.h"
244 runHook postInstall
245 '';
246 };
247
248 # The image travels as a blob in an object file's data section. The
249 # _binary_jolt_boot_{start,end} symbols jolt_main.c reads are named after the
250 # input *path*, so this copies the file somewhere it is called exactly
251 # `jolt.boot` before converting it.
252 joltBootObj = pkgs.runCommand "jolt-boot-obj" { } ''
253 cp ${joltBoot}/jolt.boot jolt.boot
254 ${ndkBin}/llvm-objcopy \
255 --input-target=binary --output-target=elf64-littleaarch64 \
256 --binary-architecture=aarch64 jolt.boot "$out"
257 '';
258
259 # The glue: jolt-native's jolt_main.c over the boot image, linked against
260 # libvidya by name. --no-undefined is what makes a symbol the Scheme side
261 # registers but the ABI no longer exports a build failure here rather than a
262 # crash on the phone.
263 libjoltapp = pkgs.runCommand "libjoltapp.so" { } ''
264 mkdir -p lib
Ask for the camera and the microphone from the app 6c745df nandi 18d ago265 cp ${nativeLibs}/libvidya.so lib/libvidya.so
266 cp ${nativeLibs}/libjoltmoq.so lib/libjoltmoq.so
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago267
268 ${cc} -shared -fPIC -O2 -o "$out" \
269 ${glueSrc}/android/jolt_main.c \
270 ${joltBootObj} \
271 -I${joltBoot} \
272 -I${glueSrc}/include \
273 -Llib \
274 ${chezAndroid}/tarm64le/boot/tarm64le/libkernel.a \
275 ${chezAndroid}/lz4/lib/liblz4.a \
Ask for the camera and the microphone from the app 6c745df nandi 18d ago276 -lvidya -ljoltmoq -landroid -llog -lz -ldl -lm -Wl,--no-undefined
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago277 '';
278
279 # --- the Java half --------------------------------------------------------
280 # One class: the photo chooser's result has to land somewhere, and native
281 # code is not somewhere.
282 classesDex = pkgs.runCommand "classes.dex"
283 {
284 nativeBuildInputs = [ pkgs.jdk17 ];
285 } ''
286 mkdir -p classes out
287 # -encoding, because a build sandbox has no locale and javac then reads
288 # the source as US-ASCII — on which the comments' em dashes are errors.
289 javac --release 17 -encoding UTF-8 --class-path ${androidJar} -d classes \
290 $(find ${self}/android/java -name '*.java')
291 ${buildTools}/d8 --min-api ${apiLevel} --output out $(find classes -name '*.class')
292 cp out/classes.dex "$out"
293 '';
294
295 # --- the package ----------------------------------------------------------
296 # OpenSSL travels with the app because the platform's own is not ours to
297 # load: an app's linker namespace refuses /system/lib64/libssl.so, and
298 # without one there is no TLS on the phone at all.
299 # Built by the NDK rather than by pkgsCross.aarch64-android: that cross
300 # stdenv cannot build its own compiler-rt on this nixpkgs — os_version_check.c
301 # includes <pthread.h> and the sysroot it is handed has no such header — and
302 # an APK has no use for a second toolchain anyway. OpenSSL's own android-arm64
303 # target wants the NDK's llvm on PATH and takes the API level from the flag.
304 # The version is the one ~/.cache/frq-openssl-android was built from.
305 opensslAndroid = pkgs.stdenv.mkDerivation {
306 pname = "openssl-android";
307 version = "3.5.4";
308
309 src = pkgs.fetchurl {
310 url = "https://github.com/openssl/openssl/releases/download/openssl-3.5.4/openssl-3.5.4.tar.gz";
311 sha256 = "16ay6ppxsky3qhg6573370iz93kihfwx9n5ipmlnjcam97w12wwn";
312 };
313
314 strictDeps = true;
315 nativeBuildInputs = with pkgs; [ perl ];
316
317 configurePhase = ''
318 runHook preConfigure
319 export ANDROID_NDK_ROOT="${ndkRoot}"
320 export PATH="${ndkBin}:$PATH"
321 # Through perl rather than as a program: its shebang is /usr/bin/env,
322 # which a build sandbox does not have.
323 perl ./Configure android-arm64 -D__ANDROID_API__=${apiLevel} \
324 shared no-tests no-docs \
325 --prefix="$out" --openssldir="$out/etc/ssl"
326 runHook postConfigure
327 '';
328
329 # install_sw, not install: the rest of an OpenSSL install is for a machine
330 # that runs it, and this one only ships two .so files into an APK.
331 installTargets = [ "install_sw" ];
332
333 dontStrip = true;
334 dontPatchELF = true;
335 };
336
337 # The libraries are stored rather than deflated: the loader maps them
338 # straight out of the APK. The dex is read rather than mapped, so it may as
339 # well compress.
340 apkUnsigned = pkgs.runCommand "frq-unsigned.apk"
341 {
342 nativeBuildInputs = [ pkgs.zip ];
343 } ''
344 mkdir -p stage/lib/${abi}
Ask for the camera and the microphone from the app 6c745df nandi 18d ago345 cp ${nativeLibs}/libvidya.so stage/lib/${abi}/libvidya.so
346 cp ${nativeLibs}/libjoltmoq.so stage/lib/${abi}/libjoltmoq.so
347 cp ${libcxx} stage/lib/${abi}/libc++_shared.so
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago348 cp ${libjoltapp} stage/lib/${abi}/libjoltapp.so
349 cp ${opensslAndroid.out}/lib/libssl.so stage/lib/${abi}/libssl.so
350 cp ${opensslAndroid.out}/lib/libcrypto.so stage/lib/${abi}/libcrypto.so
351 cp ${classesDex} stage/classes.dex
352 chmod -R u+w stage
353
Ship the APK smaller: strip the libraries, pack the image 7434fee nandi 18d ago354 # Everything the loader needs is in .dynsym, and that is what --strip-all
355 # keeps: what goes is .symtab and the debug sections, which are read by a
356 # debugger and by nothing on the phone. Worth about a third of the package
357 # — libjoltmoq and libc++_shared are most of it, and the release libraries
358 # arrive unstripped because jolt-native's own build does not strip them.
359 #
360 # Here rather than in the derivations that produce them: the inputs stay
361 # whole (a stripped libjoltapp is a worse thing to hand a debugger, and
362 # `nix build .#libjoltapp` is how it is looked at), and this is the one
363 # place that knows the difference between an object and a shipped one.
364 # The NDK's, not nixpkgs' — the host strip has no opinion worth trusting
365 # about an arm64 object.
366 ${ndkBin}/llvm-strip --strip-all stage/lib/${abi}/*.so
367
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago368 ${buildTools}/aapt2 link -o "$out" -I ${androidJar} \
369 --manifest ${self}/android/AndroidManifest.xml \
370 --min-sdk-version ${apiLevel} --target-sdk-version ${targetSdk} \
371 --version-code 1 --version-name ${version}
372
373 ( cd stage && \
Ask for the camera and the microphone from the app 6c745df nandi 18d ago374 zip -q -0 "$out" lib/${abi}/libvidya.so lib/${abi}/libjoltmoq.so \
375 lib/${abi}/libc++_shared.so lib/${abi}/libjoltapp.so \
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago376 lib/${abi}/libssl.so lib/${abi}/libcrypto.so && \
377 zip -q "$out" classes.dex )
378 '';
379
380 # Aligned and signed with a debug key. The key is generated here rather than
381 # read from ~/.android, which is the one place this build is deliberately
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago382 # not what the builds before it did: a keystore outside the store would make
383 # the output
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago384 # depend on the machine, and a release key has no business in the store at
385 # all. So this output is installable and not reproducible — keytool stamps
386 # the certificate with the time — and anything meant for a store should be
387 # signed from .#apk-unsigned instead.
388 apk = pkgs.runCommand "frq-${version}.apk"
389 {
390 nativeBuildInputs = [ pkgs.jdk17 ];
391 meta = {
392 description = "frq for Android, debug-signed";
393 platforms = [ "x86_64-linux" ];
394 };
395 } ''
396 export HOME="$TMPDIR"
397 keytool -genkeypair -keystore debug.keystore \
398 -storepass android -keypass android -alias androiddebugkey \
399 -keyalg RSA -keysize 2048 -validity 10000 \
400 -dname 'CN=Android Debug,O=Android,C=US'
401
402 ${buildTools}/zipalign -f -p 4 ${apkUnsigned} aligned.apk
403 ${buildTools}/apksigner sign --ks debug.keystore \
404 --ks-key-alias androiddebugkey \
405 --ks-pass pass:android --key-pass pass:android \
406 --out "$out" aligned.apk
407 ${buildTools}/apksigner verify "$out"
408 '';
409in
410{
411 inherit chezAndroid joltBoot libjoltapp classesDex apk;
412 apk-unsigned = apkUnsigned;
413}