rickub
nandi/frqpublic⑂ Fork 0
⑂ 234c54f1c44e80e5ffe4ddfb2e600d19b417fd80
Branches
claude/add-remote-repository-47d12e main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/frq.git
git clone ssh://git@rickub.com/nandi/frq.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

android.nix · 412 lines · 17.2 KBNix Blame HistoryRaw
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago1# The APK, as derivations.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago2#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago3# The only APK build there is: this replaced a buck2 graph and the script
4# before it, both of which named an Android SDK, an NDK, a hand-built Chez
5# cross target and an OpenSSL by absolute path and stopped when one was
6# missing. Every one of those is built or fetched here instead, so this works
7# from nothing on a machine with none of them and no ~/.cache at all:
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago8#
9# nix build .#apk
10#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago11# `just apk` is the same build with the store path handed to adb afterwards;
12# see scripts/apk.bb.
13#
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago14# On a machine with a remote builder configured, prefer
15#
16# nix build .#apk --store ssh-ng://eu.nixbuild.net --eval-store auto
17#
18# rather than letting `builders` do it. With `builders`, nix copies the output
19# of every remotely-built derivation back, and androidenv's NDK is both
20# `preferLocalBuild` and absent from cache.nixos.org — so the 3.1 GB unpacked
21# toolchain is built here and uploaded. With the remote as the *store* the
22# whole graph stays there, only .drv files go up, and the builder fetches
23# Google's zip over its own link.
24#
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago25# The steps are the ones the graph before it ran, in the same order; where a
26# genrule read a path out of `read_root_config`, a derivation takes an
27# argument.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago28{ pkgs, lib, self, chez-src, jolt-native, glimmer, joltAndroid, androidSdk, ndk }:
29
30let
31 # What the APK targets, in the three spellings the tools want it in.
32 apiLevel = "28";
33 targetSdk = "36";
34 abi = "arm64-v8a";
35 package = "uk.nandi.frq";
36 version = "0.1.0";
37
38 sdk = "${androidSdk}/libexec/android-sdk";
39 buildTools = "${sdk}/build-tools/36.0.0";
40 androidJar = "${sdk}/platforms/android-${targetSdk}/android.jar";
41
42 # The NDK's clang finds its sysroot, resource directory and the rest of LLVM
43 # relative to itself, so it is named by path rather than copied anywhere.
44 # androidenv installs the tree under libexec/android-sdk and leaves
45 # `ndk-bundle` pointing at the versioned directory beside it.
46 ndkRoot = "${ndk}/libexec/android-sdk/ndk-bundle";
47 ndkBin = "${ndkRoot}/toolchains/llvm/prebuilt/linux-x86_64/bin";
48 cc = "${ndkBin}/aarch64-linux-android${apiLevel}-clang";
49
Build the desktop libraries rather than fetching a release of them 0b81161 nandi 15d ago50 # What comes out of jolt-native's releases. These two pins are the whole of
51 # what `just bump` moves on the native side — a desktop run builds the flake
52 # input instead, so this is the only place a release is fetched. Do not edit
53 # them by hand: bump-jolt-native.bb fetches each archive, weighs it, and
54 # writes both the url and the digest below.
Ask for the camera and the microphone from the app 6c745df nandi 18d ago55 #
56 # One archive, two libraries: libvidya (the retained-tree UI) and libjoltmoq
57 # (the AV media plane). They are built together and only make sense together
58 # — libjoltapp links both — so there is one pin for the pair rather than two
59 # that could drift apart.
60 nativeRelease = pkgs.fetchurl {
61 url = "https://gitlab.com/nandithebull/jolt-native/-/releases/v0.1.3/downloads/jolt-native-android-arm64-v0.1.3.tar.gz";
62 sha256 = "4519745bae6db9a791a71b38a26478879e246166802b0a48dbf7d45e4d45a108";
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago63 };
64
65 glue = pkgs.fetchurl {
Ask for the camera and the microphone from the app 6c745df nandi 18d ago66 url = "https://gitlab.com/nandithebull/jolt-native/-/releases/v0.1.3/downloads/jolt-native-android-glue-v0.1.3.tar.gz";
67 sha256 = "83313eda124f2a0cfff6827cf4473600c1f71db2f208d654b97068a85af38da5";
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago68 };
69
Ask for the camera and the microphone from the app 6c745df nandi 18d ago70 # Unpacked once, so the consumers below name files rather than repeat the tar.
71 nativeLibs = pkgs.runCommand "jolt-native-android" { } ''
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago72 mkdir -p "$out"
Ask for the camera and the microphone from the app 6c745df nandi 18d ago73 tar -xzf ${nativeRelease} -C "$out"
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago74 '';
75
76 glueSrc = pkgs.runCommand "jolt-android-glue" { } ''
77 mkdir -p "$out"
78 tar -xzf ${glue} -C "$out" --strip-components=1
79 '';
80
Ask for the camera and the microphone from the app 6c745df nandi 18d ago81 # The C++ runtime, out of the same NDK the glue is compiled with.
82 #
83 # openh264 is C++, and its build script asks to be linked against
84 # `libc++_shared.so` by name — so libjoltmoq carries that as a DT_NEEDED. An
85 # app's linker namespace will not hand out the platform's own copy (there is
86 # no stable one to hand out), so the APK carries it, exactly as it carries
87 # OpenSSL below and for the same reason.
88 libcxx = "${ndkRoot}/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so";
89
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago90 # --- Chez's arm64 cross target ------------------------------------------
91 # The one piece with no nixpkgs equivalent: `pkgs.chez` builds a Scheme for
92 # this machine, and what the boot image needs is Chez's `tarm64le` workarea —
93 # the target boot files, the cross compiler's xpatch, and the arm64
94 # libkernel.a that libjoltapp links.
95 #
96 # Three builds in one derivation, because each needs the one before it:
97 #
98 # ta6le a host Scheme, which is what cross-compiles anything
99 # boot XM=... the target's boot files and xpatch, made by that host
100 # tarm64le the target kernel, compiled by the NDK
101 #
102 # The flags are the ones the hand-built tree under ~/.cache/vidya-chez-android
103 # was configured with, read back out of its Mf-config. zlib is Android's own
104 # (`-lz`, which Bionic has); lz4 is the in-tree submodule, built for arm64
105 # here because a cross configure links it rather than building it.
106 chezAndroid = pkgs.stdenv.mkDerivation {
107 pname = "chez-scheme-android";
108 version = "10.4.1";
109 src = chez-src;
110
111 strictDeps = true;
112 nativeBuildInputs = with pkgs; [ gnumake which ];
113
114 dontConfigure = true;
115
116 buildPhase = ''
117 runHook preBuild
118
119 # Both workareas turn off the expression editor's two dependencies, which
120 # is what ~/.cache/vidya-chez-android was configured with (its Mf-config
121 # has empty cursesLib/ncursesLib, and disablex11=yes on the target). The
122 # host Scheme here is only ever a cross compiler, and Bionic has no
123 # curses.h at all — so on the target it is not a preference but a
124 # requirement.
125 ./configure -m=ta6le --disable-x11 --disable-curses CC_FOR_BUILD="$CC"
126 make -j"$NIX_BUILD_CORES"
127 make boot XM=tarm64le -j"$NIX_BUILD_CORES"
128
129 # lz4 for the phone, not for this machine: the host build above left an
130 # x86_64 liblz4.a in the same place, and the cross link needs it gone.
131 make -C lz4/lib clean
132 make -C lz4/lib liblz4.a -j"$NIX_BUILD_CORES" \
133 CC=${cc} AR=${ndkBin}/llvm-ar
134
135 # --disable-auto-flags stops configure appending -lrt and -lpthread, which
136 # is what its unix branch does for a glibc host and what Bionic has no
137 # separate libraries for — both live in libc there. Everything it would
138 # otherwise add is passed explicitly below, matching the Mf-config of
139 # the tree this was reconstructed from.
140 ./configure -m=tarm64le --cross --disable-x11 --disable-curses \
141 --disable-auto-flags \
142 LIBS="-ldl -lm" \
143 CC=${cc} \
144 AR=${ndkBin}/llvm-ar \
145 CC_FOR_BUILD="$CC" \
146 ZLIB=-lz \
147 LZ4="$PWD/lz4/lib/liblz4.a" \
148 CPPFLAGS="-I$PWD/lz4/lib" \
149 CFLAGS="-O2 -D_REENTRANT -pthread -fPIC"
150 make -j"$NIX_BUILD_CORES"
151
152 runHook postBuild
153 '';
154
155 # The whole workarea, at the paths CHEZ_ANDROID means: the host Scheme
156 # loads xpatch out of xc-tarm64le/s, and the link below reads two archives
157 # from elsewhere in the tree. Pruning it would only be guessing at which
158 # of those the cross compiler still opens.
159 installPhase = ''
160 runHook preInstall
161 mkdir -p "$out"
162 cp -r . "$out/"
163 runHook postInstall
164 '';
165
166 # A Scheme built for this machine and a kernel built for another one; the
167 # usual fixups have an opinion about both, and neither wants it.
168 dontStrip = true;
169 dontPatchELF = true;
170 };
171
172 hostScheme = "${chezAndroid}/ta6le/bin/ta6le/scheme";
173 targetBoot = "${chezAndroid}/boot/tarm64le";
174 xpatch = "${chezAndroid}/xc-tarm64le/s/xpatch";
175
176 # --- the Jolt half --------------------------------------------------------
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago177 # frq's Scheme, cross-compiled to an arm64 boot image. The deps.edn below is
178 # written out by hand rather than resolved: there is no dependency resolution
179 # inside a cross compile, so every source root deps.edn would have resolved is
180 # named as a :path instead.
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago181 #
182 # The jolt that runs it is the fork, not upstream and not the one the desktop
183 # package builds: upstream reads the socket address out of `struct addrinfo`
184 # at glibc's offset, which on Bionic is `ai_canonname`, and an APK built with
185 # it cannot open a TLS connection at all.
186 joltBoot = pkgs.stdenv.mkDerivation {
187 pname = "frq-jolt-boot";
188 inherit version;
189
190 dontUnpack = true;
191 strictDeps = true;
192 nativeBuildInputs = [ joltAndroid ];
193
194 buildPhase = ''
195 runHook preBuild
196
197 export HOME="$TMPDIR"
198 mkdir -p project cross
199
200 cat > project/deps.edn <<EOF
201 {:paths ["${self}/src" "${glimmer}/src" "${jolt-native}/jolt/glimmer-vidya/src"]}
202 EOF
203
204 # The flat build, which is the one shape make-boot-file can take.
205 ( cd project && JOLT_NO_FLAT_SPLIT=1 jolt build -m frq.app -o app )
206
207 cat > cross/compile.ss <<EOF
208 (import (chezscheme))
209 (load "${xpatch}")
210 (optimize-level 2)
211 (generate-inspector-information #f)
Ship the APK smaller: strip the libraries, pack the image 7434fee nandi 18d ago212 ;; Packed harder, not packed for the first time: fasl output is
213 ;; compressed already, but with lz4 at its fastest setting, and on
214 ;; this image that leaves 2.3 MB on the table. What reads it back is
215 ;; the kernel linked into libjoltapp, which has zlib because
216 ;; chezAndroid is configured ZLIB=-lz — so nothing extra ships to
217 ;; decompress it.
218 ;;
219 ;; Less than the ratio of the whole file suggests (15.9 MB to 13.6):
220 ;; compression is per fasl entry rather than over the image.
221 (fasl-compressed #t)
222 (compress-format 'gzip)
223 (compress-level 'maximum)
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago224 (compile-file "$PWD/project/app.build/flat.ss" "$PWD/cross/flat.so")
225 (make-boot-file "$PWD/jolt.boot" '()
226 "${targetBoot}/petite.boot"
227 "${targetBoot}/scheme.boot"
228 "$PWD/cross/flat.so")
229 EOF
230
231 SCHEMEHEAPDIRS="${chezAndroid}/ta6le/boot/ta6le" \
232 ${hostScheme} --script cross/compile.ss
233
234 runHook postBuild
235 '';
236
237 # scheme.h travels with the image because jolt_main.c includes it.
238 installPhase = ''
239 runHook preInstall
240 mkdir -p "$out"
241 cp jolt.boot "$out/jolt.boot"
242 cp ${targetBoot}/scheme.h "$out/scheme.h"
243 runHook postInstall
244 '';
245 };
246
247 # The image travels as a blob in an object file's data section. The
248 # _binary_jolt_boot_{start,end} symbols jolt_main.c reads are named after the
249 # input *path*, so this copies the file somewhere it is called exactly
250 # `jolt.boot` before converting it.
251 joltBootObj = pkgs.runCommand "jolt-boot-obj" { } ''
252 cp ${joltBoot}/jolt.boot jolt.boot
253 ${ndkBin}/llvm-objcopy \
254 --input-target=binary --output-target=elf64-littleaarch64 \
255 --binary-architecture=aarch64 jolt.boot "$out"
256 '';
257
258 # The glue: jolt-native's jolt_main.c over the boot image, linked against
259 # libvidya by name. --no-undefined is what makes a symbol the Scheme side
260 # registers but the ABI no longer exports a build failure here rather than a
261 # crash on the phone.
262 libjoltapp = pkgs.runCommand "libjoltapp.so" { } ''
263 mkdir -p lib
Ask for the camera and the microphone from the app 6c745df nandi 18d ago264 cp ${nativeLibs}/libvidya.so lib/libvidya.so
265 cp ${nativeLibs}/libjoltmoq.so lib/libjoltmoq.so
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago266
267 ${cc} -shared -fPIC -O2 -o "$out" \
268 ${glueSrc}/android/jolt_main.c \
269 ${joltBootObj} \
270 -I${joltBoot} \
271 -I${glueSrc}/include \
272 -Llib \
273 ${chezAndroid}/tarm64le/boot/tarm64le/libkernel.a \
274 ${chezAndroid}/lz4/lib/liblz4.a \
Ask for the camera and the microphone from the app 6c745df nandi 18d ago275 -lvidya -ljoltmoq -landroid -llog -lz -ldl -lm -Wl,--no-undefined
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago276 '';
277
278 # --- the Java half --------------------------------------------------------
279 # One class: the photo chooser's result has to land somewhere, and native
280 # code is not somewhere.
281 classesDex = pkgs.runCommand "classes.dex"
282 {
283 nativeBuildInputs = [ pkgs.jdk17 ];
284 } ''
285 mkdir -p classes out
286 # -encoding, because a build sandbox has no locale and javac then reads
287 # the source as US-ASCII — on which the comments' em dashes are errors.
288 javac --release 17 -encoding UTF-8 --class-path ${androidJar} -d classes \
289 $(find ${self}/android/java -name '*.java')
290 ${buildTools}/d8 --min-api ${apiLevel} --output out $(find classes -name '*.class')
291 cp out/classes.dex "$out"
292 '';
293
294 # --- the package ----------------------------------------------------------
295 # OpenSSL travels with the app because the platform's own is not ours to
296 # load: an app's linker namespace refuses /system/lib64/libssl.so, and
297 # without one there is no TLS on the phone at all.
298 # Built by the NDK rather than by pkgsCross.aarch64-android: that cross
299 # stdenv cannot build its own compiler-rt on this nixpkgs — os_version_check.c
300 # includes <pthread.h> and the sysroot it is handed has no such header — and
301 # an APK has no use for a second toolchain anyway. OpenSSL's own android-arm64
302 # target wants the NDK's llvm on PATH and takes the API level from the flag.
303 # The version is the one ~/.cache/frq-openssl-android was built from.
304 opensslAndroid = pkgs.stdenv.mkDerivation {
305 pname = "openssl-android";
306 version = "3.5.4";
307
308 src = pkgs.fetchurl {
309 url = "https://github.com/openssl/openssl/releases/download/openssl-3.5.4/openssl-3.5.4.tar.gz";
310 sha256 = "16ay6ppxsky3qhg6573370iz93kihfwx9n5ipmlnjcam97w12wwn";
311 };
312
313 strictDeps = true;
314 nativeBuildInputs = with pkgs; [ perl ];
315
316 configurePhase = ''
317 runHook preConfigure
318 export ANDROID_NDK_ROOT="${ndkRoot}"
319 export PATH="${ndkBin}:$PATH"
320 # Through perl rather than as a program: its shebang is /usr/bin/env,
321 # which a build sandbox does not have.
322 perl ./Configure android-arm64 -D__ANDROID_API__=${apiLevel} \
323 shared no-tests no-docs \
324 --prefix="$out" --openssldir="$out/etc/ssl"
325 runHook postConfigure
326 '';
327
328 # install_sw, not install: the rest of an OpenSSL install is for a machine
329 # that runs it, and this one only ships two .so files into an APK.
330 installTargets = [ "install_sw" ];
331
332 dontStrip = true;
333 dontPatchELF = true;
334 };
335
336 # The libraries are stored rather than deflated: the loader maps them
337 # straight out of the APK. The dex is read rather than mapped, so it may as
338 # well compress.
339 apkUnsigned = pkgs.runCommand "frq-unsigned.apk"
340 {
341 nativeBuildInputs = [ pkgs.zip ];
342 } ''
343 mkdir -p stage/lib/${abi}
Ask for the camera and the microphone from the app 6c745df nandi 18d ago344 cp ${nativeLibs}/libvidya.so stage/lib/${abi}/libvidya.so
345 cp ${nativeLibs}/libjoltmoq.so stage/lib/${abi}/libjoltmoq.so
346 cp ${libcxx} stage/lib/${abi}/libc++_shared.so
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago347 cp ${libjoltapp} stage/lib/${abi}/libjoltapp.so
348 cp ${opensslAndroid.out}/lib/libssl.so stage/lib/${abi}/libssl.so
349 cp ${opensslAndroid.out}/lib/libcrypto.so stage/lib/${abi}/libcrypto.so
350 cp ${classesDex} stage/classes.dex
351 chmod -R u+w stage
352
Ship the APK smaller: strip the libraries, pack the image 7434fee nandi 18d ago353 # Everything the loader needs is in .dynsym, and that is what --strip-all
354 # keeps: what goes is .symtab and the debug sections, which are read by a
355 # debugger and by nothing on the phone. Worth about a third of the package
356 # — libjoltmoq and libc++_shared are most of it, and the release libraries
357 # arrive unstripped because jolt-native's own build does not strip them.
358 #
359 # Here rather than in the derivations that produce them: the inputs stay
360 # whole (a stripped libjoltapp is a worse thing to hand a debugger, and
361 # `nix build .#libjoltapp` is how it is looked at), and this is the one
362 # place that knows the difference between an object and a shipped one.
363 # The NDK's, not nixpkgs' — the host strip has no opinion worth trusting
364 # about an arm64 object.
365 ${ndkBin}/llvm-strip --strip-all stage/lib/${abi}/*.so
366
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago367 ${buildTools}/aapt2 link -o "$out" -I ${androidJar} \
368 --manifest ${self}/android/AndroidManifest.xml \
369 --min-sdk-version ${apiLevel} --target-sdk-version ${targetSdk} \
370 --version-code 1 --version-name ${version}
371
372 ( cd stage && \
Ask for the camera and the microphone from the app 6c745df nandi 18d ago373 zip -q -0 "$out" lib/${abi}/libvidya.so lib/${abi}/libjoltmoq.so \
374 lib/${abi}/libc++_shared.so lib/${abi}/libjoltapp.so \
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago375 lib/${abi}/libssl.so lib/${abi}/libcrypto.so && \
376 zip -q "$out" classes.dex )
377 '';
378
379 # Aligned and signed with a debug key. The key is generated here rather than
380 # read from ~/.android, which is the one place this build is deliberately
Build the APK as derivations only, and retire the buck2 graph 3bc3aaa nandi 16d ago381 # not what the builds before it did: a keystore outside the store would make
382 # the output
Build the APK from the flake, and from nothing 6aa2a6b nandi 18d ago383 # depend on the machine, and a release key has no business in the store at
384 # all. So this output is installable and not reproducible — keytool stamps
385 # the certificate with the time — and anything meant for a store should be
386 # signed from .#apk-unsigned instead.
387 apk = pkgs.runCommand "frq-${version}.apk"
388 {
389 nativeBuildInputs = [ pkgs.jdk17 ];
390 meta = {
391 description = "frq for Android, debug-signed";
392 platforms = [ "x86_64-linux" ];
393 };
394 } ''
395 export HOME="$TMPDIR"
396 keytool -genkeypair -keystore debug.keystore \
397 -storepass android -keypass android -alias androiddebugkey \
398 -keyalg RSA -keysize 2048 -validity 10000 \
399 -dname 'CN=Android Debug,O=Android,C=US'
400
401 ${buildTools}/zipalign -f -p 4 ${apkUnsigned} aligned.apk
402 ${buildTools}/apksigner sign --ks debug.keystore \
403 --ks-key-alias androiddebugkey \
404 --ks-pass pass:android --key-pass pass:android \
405 --out "$out" aligned.apk
406 ${buildTools}/apksigner verify "$out"
407 '';
408in
409{
410 inherit chezAndroid joltBoot libjoltapp classesDex apk;
411 apk-unsigned = apkUnsigned;
412}