A second host that checks, and deliberately does not publish
frq builds on rickub as well as GitLab; this repo only ever built on one, so a push that reached rickub and not GitLab was a push nothing checked. The job is the GitLab one with the same builder behind it. It cannot compile here either — a rickub runner is a container and this graph is ~800 vendored crates behind libcosmic and aws-lc-sys — so it evaluates, dispatches to eu.nixbuild.net and fetches what comes back, which means both hosts warm the same store rather than each paying for an hour. What it does not do is upload. The tarballs a consumer pins come from GitLab's package registry, and the credential for that is CI_JOB_TOKEN — a GitLab job's own, issued to the job, not a long-lived secret worth minting for a second host. One publisher, one place `latest` moves. Here the three tarballs are run artifacts and no more. One difference from the GitLab job, and it is the part worth knowing: there the whole job is root in a nixos/nix image, while install-nix-action leaves a multi-user install, so the process that opens the connection to the builder is nix-daemon as root. The key goes in /root/.ssh, and the daemon is restarted after nix.conf is appended to. Get either wrong and nothing errors — the build just stays local, which on this graph is the runner trying to compile libcosmic. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
65c27be parent: 12754c2 added
.rickub/workflows/build.yml +147 -0 | new file mode 100644 | ||
| @@ -0,0 +1,147 @@ | ||
| 1 | +# The build, on rickub. The same `nix flake check` and the same three objects | |
| 2 | +# GitLab builds next door, for the case where a push reaches this host and not | |
| 3 | +# that one. | |
| 4 | +# | |
| 5 | +# It does not compile here, and could not: a rickub runner is a Linux x86-64 | |
| 6 | +# container, and this graph is ~800 vendored crates behind libcosmic, iced, | |
| 7 | +# wgpu and aws-lc-sys. What the runner does is evaluate, dispatch and fetch — | |
| 8 | +# eu.nixbuild.net holds every vendor path from previous runs and does the | |
| 9 | +# compiling, exactly as it does for GitLab. So the two hosts share a cache | |
| 10 | +# rather than duplicating an hour of work, and a warm pipeline here is a | |
| 11 | +# pipeline that mostly substitutes. | |
| 12 | +# | |
| 13 | +# What it deliberately does NOT do is publish. The tarballs go to GitLab's | |
| 14 | +# generic package registry, and the token for that is CI_JOB_TOKEN — a GitLab | |
| 15 | +# job's own credential, issued to the job, not something to mint and carry | |
| 16 | +# here as a long-lived secret. One publisher and one place a consumer pins | |
| 17 | +# from is also just simpler to reason about than two. This host builds, checks | |
| 18 | +# and hands the tarballs back as run artifacts; `main` on GitLab is what moves | |
| 19 | +# `latest`. | |
| 20 | +# | |
| 21 | +# Lives in .rickub/workflows/ rather than .github/workflows/ because rickub | |
| 22 | +# reads one or the other and never both — see | |
| 23 | +# https://rickub.com/docs/actions and https://rickub.com/docs/migrating-from-github | |
| 24 | +name: build | |
| 25 | + | |
| 26 | +on: | |
| 27 | + push: | |
| 28 | + pull_request: | |
| 29 | + workflow_dispatch: | |
| 30 | + | |
| 31 | +jobs: | |
| 32 | + check: | |
| 33 | + runs-on: ubuntu-latest | |
| 34 | + # A cold run vendors the world and compiles the workspace. The builder does | |
| 35 | + # the compiling, but this side waits on it, and waits on the fetch of what | |
| 36 | + # comes back. GitLab gives the same job 3h. | |
| 37 | + timeout-minutes: 180 | |
| 38 | + steps: | |
| 39 | + # The flake is fetched from the checkout, and nix wants the git tree | |
| 40 | + # rather than a shallow single commit to resolve `self` from. | |
| 41 | + - uses: actions/checkout@v4 | |
| 42 | + with: | |
| 43 | + fetch-depth: 0 | |
| 44 | + | |
| 45 | + - uses: cachix/install-nix-action@v27 | |
| 46 | + with: | |
| 47 | + extra_nix_config: | | |
| 48 | + experimental-features = nix-command flakes | |
| 49 | + | |
| 50 | + # Everything the GitLab job's before_script sets up, and for the same | |
| 51 | + # reasons — the comments there are the long version. The short one: | |
| 52 | + # | |
| 53 | + # max-jobs must NOT be zero. At zero the runner refuses to build | |
| 54 | + # anything at all, which sends the fixed-output fetches (rust | |
| 55 | + # toolchain tarballs, every crate source) to nixbuild, where the build | |
| 56 | + # sandbox has no network and they can only fail. nproc keeps those and | |
| 57 | + # crane's hundreds of trivial cargo-src derivations local and parallel; | |
| 58 | + # the expensive builds still go remote. | |
| 59 | + # | |
| 60 | + # The explicit priority on nixbuild matters. nix picks a substituter by | |
| 61 | + # priority rather than by the order listed, and an ssh-ng store | |
| 62 | + # defaults to 0 against cache.nixos.org's 40 — so without it nixbuild | |
| 63 | + # wins even for stock nixpkgs paths and they crawl down one SSH | |
| 64 | + # connection instead of coming off the CDN. | |
| 65 | + # | |
| 66 | + # Keep the builder's job count modest: nixbuild's sshd caps concurrent | |
| 67 | + # sessions and refuses the excess, which nix reports as the thoroughly | |
| 68 | + # misleading "Nix daemon disconnected unexpectedly (maybe it crashed?)". | |
| 69 | + - name: Point nix at eu.nixbuild.net | |
| 70 | + env: | |
| 71 | + NIXBUILD_SSH_KEY: ${{ secrets.NIXBUILD_SSH_KEY }} | |
| 72 | + run: | | |
| 73 | + set -eu | |
| 74 | + # ROOT's ~/.ssh, and this is the one thing that differs from the | |
| 75 | + # GitLab job. There the whole job runs as root in a nixos/nix image; | |
| 76 | + # here install-nix-action leaves a multi-user install, so the process | |
| 77 | + # that opens the SSH connection to the builder is nix-daemon running | |
| 78 | + # as root. A key under the runner user's home is a key it never | |
| 79 | + # reads, and the failure is a build that silently stays local — | |
| 80 | + # which on this graph means a runner trying to compile libcosmic. | |
| 81 | + sudo mkdir -p /root/.ssh | |
| 82 | + # base64 because a CI variable can only carry a single line. | |
| 83 | + printf '%s' "$NIXBUILD_SSH_KEY" | base64 -d | sudo tee /root/.ssh/nixbuild >/dev/null | |
| 84 | + sudo chmod 600 /root/.ssh/nixbuild | |
| 85 | + printf 'Host eu.nixbuild.net\n PubkeyAcceptedKeyTypes ssh-ed25519\n IdentityFile /root/.ssh/nixbuild\n IdentitiesOnly yes\n' | sudo tee /root/.ssh/config >/dev/null | |
| 86 | + printf 'eu.nixbuild.net ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPIQCZc54poJ8vqawd8TraNryQeJnvH1eLpIDgbiqymM\n' | sudo tee /root/.ssh/known_hosts >/dev/null | |
| 87 | + # nixbuild's paths are signed, but the key is per-account and only | |
| 88 | + # readable from its web UI, so rather than carry a second secret we | |
| 89 | + # lean on the transport — the store is reached over SSH with our own | |
| 90 | + # key. cache.nixos.org is verified by its own key either way; | |
| 91 | + # require-sigs only relaxes nixbuild. | |
| 92 | + sudo tee -a /etc/nix/nix.conf >/dev/null <<EOF | |
| 93 | + builders = ssh-ng://eu.nixbuild.net x86_64-linux - 16 1 big-parallel,benchmark | |
| 94 | + builders-use-substitutes = true | |
| 95 | + substituters = https://cache.nixos.org ssh-ng://eu.nixbuild.net?priority=50 | |
| 96 | + require-sigs = false | |
| 97 | + max-jobs = $(nproc) | |
| 98 | + EOF | |
| 99 | + # The daemon reads nix.conf at start, so settings appended after | |
| 100 | + # install-nix-action has already started it do nothing until it is | |
| 101 | + # restarted. Without this the job runs with none of the above and | |
| 102 | + # the first symptom is an hour of local compilation. | |
| 103 | + sudo systemctl restart nix-daemon | |
| 104 | + nix config show builders | |
| 105 | + nix config show max-jobs | |
| 106 | + | |
| 107 | + # fmt, clippy and the test suite, plus all four desktop objects — the | |
| 108 | + # checks build them. This is the step that has historically failed, and | |
| 109 | + # it fails fast: fmt costs seconds and runs before anything is compiled. | |
| 110 | + - run: nix flake check -L | |
| 111 | + | |
| 112 | + # Store paths are read-only symlink farms; the runner uploads plain | |
| 113 | + # files, so dereference them into trees it can zip. | |
| 114 | + - name: Stage the objects | |
| 115 | + run: | | |
| 116 | + set -eu | |
| 117 | + nix build -L --no-link --print-out-paths .#libs .#android > /tmp/outs | |
| 118 | + mkdir -p artifacts && xargs -a /tmp/outs -I{} cp -rL {}/. artifacts/ | |
| 119 | + # Staged beside artifacts/ and not into it: libsPortable holds the | |
| 120 | + # same four sonames with a different RUNPATH, so merging the two | |
| 121 | + # trees would leave whichever was copied last under both names. | |
| 122 | + nix build -L --no-link --print-out-paths .#libsPortable > /tmp/portable | |
| 123 | + mkdir -p portable && xargs -a /tmp/portable -I{} cp -rL {}/. portable/ | |
| 124 | + | |
| 125 | + # The same three tarballs GitLab publishes, rooted at lib/ and include/ | |
| 126 | + # so a consumer's `flake = false` input resolves to ${input}/lib/… with | |
| 127 | + # nothing in between. Here they are only artifacts of the run — see the | |
| 128 | + # note at the top about why this host does not upload them anywhere. | |
| 129 | + - name: Tar them the way a consumer takes them | |
| 130 | + run: | | |
| 131 | + set -eu | |
| 132 | + tar czf x86_64-linux.tar.gz -C artifacts include \ | |
| 133 | + lib/libvidya.so lib/libjolttui.so lib/libjoltmoq.so lib/libjoltcosmic.so | |
| 134 | + # `-C portable lib include` and not a file list: the point of this one | |
| 135 | + # is the libraries that came along, and naming the four we know about | |
| 136 | + # would drop them. | |
| 137 | + tar czf x86_64-linux-portable.tar.gz -C portable lib include | |
| 138 | + tar czf android-arm64-v8a.tar.gz -C artifacts include lib/arm64-v8a | |
| 139 | + | |
| 140 | + - uses: actions/upload-artifact@v4 | |
| 141 | + with: | |
| 142 | + name: jolt-native-${{ github.sha }} | |
| 143 | + path: | | |
| 144 | + x86_64-linux.tar.gz | |
| 145 | + x86_64-linux-portable.tar.gz | |
| 146 | + android-arm64-v8a.tar.gz | |
| 147 | + if-no-files-found: error | |
| new file mode 100644 | |||
| @@ -0,0 +1,147 @@ | |||
| 1 | +# The build, on rickub. The same `nix flake check` and the same three objects | ||
| 2 | +# GitLab builds next door, for the case where a push reaches this host and not | ||
| 3 | +# that one. | ||
| 4 | +# | ||
| 5 | +# It does not compile here, and could not: a rickub runner is a Linux x86-64 | ||
| 6 | +# container, and this graph is ~800 vendored crates behind libcosmic, iced, | ||
| 7 | +# wgpu and aws-lc-sys. What the runner does is evaluate, dispatch and fetch — | ||
| 8 | +# eu.nixbuild.net holds every vendor path from previous runs and does the | ||
| 9 | +# compiling, exactly as it does for GitLab. So the two hosts share a cache | ||
| 10 | +# rather than duplicating an hour of work, and a warm pipeline here is a | ||
| 11 | +# pipeline that mostly substitutes. | ||
| 12 | +# | ||
| 13 | +# What it deliberately does NOT do is publish. The tarballs go to GitLab's | ||
| 14 | +# generic package registry, and the token for that is CI_JOB_TOKEN — a GitLab | ||
| 15 | +# job's own credential, issued to the job, not something to mint and carry | ||
| 16 | +# here as a long-lived secret. One publisher and one place a consumer pins | ||
| 17 | +# from is also just simpler to reason about than two. This host builds, checks | ||
| 18 | +# and hands the tarballs back as run artifacts; `main` on GitLab is what moves | ||
| 19 | +# `latest`. | ||
| 20 | +# | ||
| 21 | +# Lives in .rickub/workflows/ rather than .github/workflows/ because rickub | ||
| 22 | +# reads one or the other and never both — see | ||
| 23 | +# https://rickub.com/docs/actions and https://rickub.com/docs/migrating-from-github | ||
| 24 | +name: build | ||
| 25 | + | ||
| 26 | +on: | ||
| 27 | + push: | ||
| 28 | + pull_request: | ||
| 29 | + workflow_dispatch: | ||
| 30 | + | ||
| 31 | +jobs: | ||
| 32 | + check: | ||
| 33 | + runs-on: ubuntu-latest | ||
| 34 | + # A cold run vendors the world and compiles the workspace. The builder does | ||
| 35 | + # the compiling, but this side waits on it, and waits on the fetch of what | ||
| 36 | + # comes back. GitLab gives the same job 3h. | ||
| 37 | + timeout-minutes: 180 | ||
| 38 | + steps: | ||
| 39 | + # The flake is fetched from the checkout, and nix wants the git tree | ||
| 40 | + # rather than a shallow single commit to resolve `self` from. | ||
| 41 | + - uses: actions/checkout@v4 | ||
| 42 | + with: | ||
| 43 | + fetch-depth: 0 | ||
| 44 | + | ||
| 45 | + - uses: cachix/install-nix-action@v27 | ||
| 46 | + with: | ||
| 47 | + extra_nix_config: | | ||
| 48 | + experimental-features = nix-command flakes | ||
| 49 | + | ||
| 50 | + # Everything the GitLab job's before_script sets up, and for the same | ||
| 51 | + # reasons — the comments there are the long version. The short one: | ||
| 52 | + # | ||
| 53 | + # max-jobs must NOT be zero. At zero the runner refuses to build | ||
| 54 | + # anything at all, which sends the fixed-output fetches (rust | ||
| 55 | + # toolchain tarballs, every crate source) to nixbuild, where the build | ||
| 56 | + # sandbox has no network and they can only fail. nproc keeps those and | ||
| 57 | + # crane's hundreds of trivial cargo-src derivations local and parallel; | ||
| 58 | + # the expensive builds still go remote. | ||
| 59 | + # | ||
| 60 | + # The explicit priority on nixbuild matters. nix picks a substituter by | ||
| 61 | + # priority rather than by the order listed, and an ssh-ng store | ||
| 62 | + # defaults to 0 against cache.nixos.org's 40 — so without it nixbuild | ||
| 63 | + # wins even for stock nixpkgs paths and they crawl down one SSH | ||
| 64 | + # connection instead of coming off the CDN. | ||
| 65 | + # | ||
| 66 | + # Keep the builder's job count modest: nixbuild's sshd caps concurrent | ||
| 67 | + # sessions and refuses the excess, which nix reports as the thoroughly | ||
| 68 | + # misleading "Nix daemon disconnected unexpectedly (maybe it crashed?)". | ||
| 69 | + - name: Point nix at eu.nixbuild.net | ||
| 70 | + env: | ||
| 71 | + NIXBUILD_SSH_KEY: ${{ secrets.NIXBUILD_SSH_KEY }} | ||
| 72 | + run: | | ||
| 73 | + set -eu | ||
| 74 | + # ROOT's ~/.ssh, and this is the one thing that differs from the | ||
| 75 | + # GitLab job. There the whole job runs as root in a nixos/nix image; | ||
| 76 | + # here install-nix-action leaves a multi-user install, so the process | ||
| 77 | + # that opens the SSH connection to the builder is nix-daemon running | ||
| 78 | + # as root. A key under the runner user's home is a key it never | ||
| 79 | + # reads, and the failure is a build that silently stays local — | ||
| 80 | + # which on this graph means a runner trying to compile libcosmic. | ||
| 81 | + sudo mkdir -p /root/.ssh | ||
| 82 | + # base64 because a CI variable can only carry a single line. | ||
| 83 | + printf '%s' "$NIXBUILD_SSH_KEY" | base64 -d | sudo tee /root/.ssh/nixbuild >/dev/null | ||
| 84 | + sudo chmod 600 /root/.ssh/nixbuild | ||
| 85 | + printf 'Host eu.nixbuild.net\n PubkeyAcceptedKeyTypes ssh-ed25519\n IdentityFile /root/.ssh/nixbuild\n IdentitiesOnly yes\n' | sudo tee /root/.ssh/config >/dev/null | ||
| 86 | + printf 'eu.nixbuild.net ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPIQCZc54poJ8vqawd8TraNryQeJnvH1eLpIDgbiqymM\n' | sudo tee /root/.ssh/known_hosts >/dev/null | ||
| 87 | + # nixbuild's paths are signed, but the key is per-account and only | ||
| 88 | + # readable from its web UI, so rather than carry a second secret we | ||
| 89 | + # lean on the transport — the store is reached over SSH with our own | ||
| 90 | + # key. cache.nixos.org is verified by its own key either way; | ||
| 91 | + # require-sigs only relaxes nixbuild. | ||
| 92 | + sudo tee -a /etc/nix/nix.conf >/dev/null <<EOF | ||
| 93 | + builders = ssh-ng://eu.nixbuild.net x86_64-linux - 16 1 big-parallel,benchmark | ||
| 94 | + builders-use-substitutes = true | ||
| 95 | + substituters = https://cache.nixos.org ssh-ng://eu.nixbuild.net?priority=50 | ||
| 96 | + require-sigs = false | ||
| 97 | + max-jobs = $(nproc) | ||
| 98 | + EOF | ||
| 99 | + # The daemon reads nix.conf at start, so settings appended after | ||
| 100 | + # install-nix-action has already started it do nothing until it is | ||
| 101 | + # restarted. Without this the job runs with none of the above and | ||
| 102 | + # the first symptom is an hour of local compilation. | ||
| 103 | + sudo systemctl restart nix-daemon | ||
| 104 | + nix config show builders | ||
| 105 | + nix config show max-jobs | ||
| 106 | + | ||
| 107 | + # fmt, clippy and the test suite, plus all four desktop objects — the | ||
| 108 | + # checks build them. This is the step that has historically failed, and | ||
| 109 | + # it fails fast: fmt costs seconds and runs before anything is compiled. | ||
| 110 | + - run: nix flake check -L | ||
| 111 | + | ||
| 112 | + # Store paths are read-only symlink farms; the runner uploads plain | ||
| 113 | + # files, so dereference them into trees it can zip. | ||
| 114 | + - name: Stage the objects | ||
| 115 | + run: | | ||
| 116 | + set -eu | ||
| 117 | + nix build -L --no-link --print-out-paths .#libs .#android > /tmp/outs | ||
| 118 | + mkdir -p artifacts && xargs -a /tmp/outs -I{} cp -rL {}/. artifacts/ | ||
| 119 | + # Staged beside artifacts/ and not into it: libsPortable holds the | ||
| 120 | + # same four sonames with a different RUNPATH, so merging the two | ||
| 121 | + # trees would leave whichever was copied last under both names. | ||
| 122 | + nix build -L --no-link --print-out-paths .#libsPortable > /tmp/portable | ||
| 123 | + mkdir -p portable && xargs -a /tmp/portable -I{} cp -rL {}/. portable/ | ||
| 124 | + | ||
| 125 | + # The same three tarballs GitLab publishes, rooted at lib/ and include/ | ||
| 126 | + # so a consumer's `flake = false` input resolves to ${input}/lib/… with | ||
| 127 | + # nothing in between. Here they are only artifacts of the run — see the | ||
| 128 | + # note at the top about why this host does not upload them anywhere. | ||
| 129 | + - name: Tar them the way a consumer takes them | ||
| 130 | + run: | | ||
| 131 | + set -eu | ||
| 132 | + tar czf x86_64-linux.tar.gz -C artifacts include \ | ||
| 133 | + lib/libvidya.so lib/libjolttui.so lib/libjoltmoq.so lib/libjoltcosmic.so | ||
| 134 | + # `-C portable lib include` and not a file list: the point of this one | ||
| 135 | + # is the libraries that came along, and naming the four we know about | ||
| 136 | + # would drop them. | ||
| 137 | + tar czf x86_64-linux-portable.tar.gz -C portable lib include | ||
| 138 | + tar czf android-arm64-v8a.tar.gz -C artifacts include lib/arm64-v8a | ||
| 139 | + | ||
| 140 | + - uses: actions/upload-artifact@v4 | ||
| 141 | + with: | ||
| 142 | + name: jolt-native-${{ github.sha }} | ||
| 143 | + path: | | ||
| 144 | + x86_64-linux.tar.gz | ||
| 145 | + x86_64-linux-portable.tar.gz | ||
| 146 | + android-arm64-v8a.tar.gz | ||
| 147 | + if-no-files-found: error | ||