Fetch rustc the way buck can ship it, from what DotSlash already pins
DotSlash stays where a version is written down. What it cannot do is tell buck anything: it resolves a tool at run time, on the machine running it, against a cache in the user's home. So the compiler was not an input to any action, not in any action's cache digest, and not something a remote worker could be handed — one fact behind both of this repo's problems with it. scripts/dotslash-to-buck reads the manifests and writes the same URLs and digests out as a table buck fetches from, so there is no second copy of a digest to drift from the first. `just sync-dist` after bumping one. rustc is now a dependency of every action that compiles anything, through a wrapper that stitches the sysroot the way scripts/rustc does — the flags cannot carry it, since rustc_flags takes plain strings, and the compiler cannot carry it unconditionally, since buck's own rustc_cfg action passes an empty --sysroot and rustc refuses the option twice. Two workarounds come out with it. The PATH shadowing existed only because system_rust_toolchain hardcodes the bare name; the custom rule takes the compiler as a dep instead. The --cfg carrying rustc's version existed only because the compiler was absent from the digest, and it is not any more. The same cfg stays for zig, which is still an absolute path: a path does not change when the binary behind it does, and the cache cannot tell. That comes out when the C toolchain is a dependency too, which is what is left. Verified by building with scripts/ off PATH entirely — four real compiles, where before the host's rustc would have answered silently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
8cf1b33 parent: 600f207 modified
.buckconfig +1 -1 | @@ -30,4 +30,4 @@ | ||
| 30 | 30 | execution_platforms = root//platforms:cache |
| 31 | 31 | |
| 32 | 32 | [project] |
| 33 | - ignore = .git,target | |
| 33 | + ignore = .git,target,build | |
| @@ -30,4 +30,4 @@ | |||
| 30 | execution_platforms = root//platforms:cache | 30 | execution_platforms = root//platforms:cache |
| 31 | 31 | ||
| 32 | [project] | 32 | [project] |
| 33 | - ignore = .git,target | 33 | + ignore = .git,target,build |
modified
justfile +16 -11 | @@ -18,20 +18,19 @@ default: | ||
| 18 | 18 | build *args: |
| 19 | 19 | cargo build --release {{args}} |
| 20 | 20 | |
| 21 | -# buck2, with scripts/ reachable two ways, because two things need it | |
| 22 | -# differently. `rustc` is resolved off PATH: the prelude's | |
| 23 | -# system_rust_toolchain hardcodes the bare name and offers no attribute to | |
| 24 | -# override it, so the pinned compiler is selected by shadowing. The C tools are | |
| 25 | -# named by absolute path in .buckconfig.local, which the toolchain reads — | |
| 26 | -# see the note in toolchains/BUCK for why a bare name will not do for those. | |
| 21 | +# buck2, told where the C tools are. They are named by absolute path because | |
| 22 | +# the prelude re-execs them from a build script's own directory — see the note | |
| 23 | +# in toolchains/BUCK. rustc is not here: buck fetches it itself now, so it | |
| 24 | +# needs neither a path nor a place on PATH. | |
| 27 | 25 | # |
| 28 | -# Without the PATH half the build silently uses the host's rustc, which is a | |
| 29 | -# different compiler than the one the cache was populated with. | |
| 26 | +# The zig version goes in for the same reason the paths do not carry it: the | |
| 27 | +# path stays the same when the binary behind it changes, and the cache cannot | |
| 28 | +# tell. | |
| 30 | 29 | buck *args: |
| 31 | - printf '[jolt]\n scripts = %s\n rustc_version = %s\n' \ | |
| 30 | + printf '[jolt]\n scripts = %s\n zig_version = %s\n' \ | |
| 32 | 31 | "{{justfile_directory()}}/scripts" \ |
| 33 | - "$(scripts/rustc --version | tr -d '\n')" > .buckconfig.local | |
| 34 | - PATH="{{justfile_directory()}}/scripts:$PATH" scripts/buck2 {{args}} | |
| 32 | + "$(scripts/zig version | tr -d '\n')" > .buckconfig.local | |
| 33 | + scripts/buck2 {{args}} | |
| 35 | 34 | |
| 36 | 35 | # The same two objects under buck2, staged into one directory a consumer can |
| 37 | 36 | # put on its loader path — the same shape `build` leaves in target/release. |
| @@ -65,3 +64,9 @@ lint: | ||
| 65 | 64 | # Where a consumer points LD_LIBRARY_PATH. |
| 66 | 65 | libdir: |
| 67 | 66 | @echo "{{justfile_directory()}}/target/release" |
| 67 | + | |
| 68 | +# Regenerate the table buck fetches its toolchains from, after editing any of | |
| 69 | +# scripts/*.dotslash. DotSlash stays the one place a version is written down; | |
| 70 | +# this only restates it in a form buck can act on. | |
| 71 | +sync-dist: | |
| 72 | + scripts/dotslash-to-buck | |
| @@ -18,20 +18,19 @@ default: | |||
| 18 | build *args: | 18 | build *args: |
| 19 | cargo build --release {{args}} | 19 | cargo build --release {{args}} |
| 20 | 20 | ||
| 21 | -# buck2, with scripts/ reachable two ways, because two things need it | 21 | +# buck2, told where the C tools are. They are named by absolute path because |
| 22 | -# differently. `rustc` is resolved off PATH: the prelude's | 22 | +# the prelude re-execs them from a build script's own directory — see the note |
| 23 | -# system_rust_toolchain hardcodes the bare name and offers no attribute to | 23 | +# in toolchains/BUCK. rustc is not here: buck fetches it itself now, so it |
| 24 | -# override it, so the pinned compiler is selected by shadowing. The C tools are | 24 | +# needs neither a path nor a place on PATH. |
| 25 | -# named by absolute path in .buckconfig.local, which the toolchain reads — | ||
| 26 | -# see the note in toolchains/BUCK for why a bare name will not do for those. | ||
| 27 | # | 25 | # |
| 28 | -# Without the PATH half the build silently uses the host's rustc, which is a | 26 | +# The zig version goes in for the same reason the paths do not carry it: the |
| 29 | -# different compiler than the one the cache was populated with. | 27 | +# path stays the same when the binary behind it changes, and the cache cannot |
| 28 | +# tell. | ||
| 30 | buck *args: | 29 | buck *args: |
| 31 | - printf '[jolt]\n scripts = %s\n rustc_version = %s\n' \ | 30 | + printf '[jolt]\n scripts = %s\n zig_version = %s\n' \ |
| 32 | "{{justfile_directory()}}/scripts" \ | 31 | "{{justfile_directory()}}/scripts" \ |
| 33 | - "$(scripts/rustc --version | tr -d '\n')" > .buckconfig.local | 32 | + "$(scripts/zig version | tr -d '\n')" > .buckconfig.local |
| 34 | - PATH="{{justfile_directory()}}/scripts:$PATH" scripts/buck2 {{args}} | 33 | + scripts/buck2 {{args}} |
| 35 | 34 | ||
| 36 | # The same two objects under buck2, staged into one directory a consumer can | 35 | # The same two objects under buck2, staged into one directory a consumer can |
| 37 | # put on its loader path — the same shape `build` leaves in target/release. | 36 | # put on its loader path — the same shape `build` leaves in target/release. |
| @@ -65,3 +64,9 @@ lint: | |||
| 65 | # Where a consumer points LD_LIBRARY_PATH. | 64 | # Where a consumer points LD_LIBRARY_PATH. |
| 66 | libdir: | 65 | libdir: |
| 67 | @echo "{{justfile_directory()}}/target/release" | 66 | @echo "{{justfile_directory()}}/target/release" |
| 67 | + | ||
| 68 | +# Regenerate the table buck fetches its toolchains from, after editing any of | ||
| 69 | +# scripts/*.dotslash. DotSlash stays the one place a version is written down; | ||
| 70 | +# this only restates it in a form buck can act on. | ||
| 71 | +sync-dist: | ||
| 72 | + scripts/dotslash-to-buck | ||
added
scripts/dotslash-to-buck +76 -0 | new file mode 100755 | ||
| @@ -0,0 +1,76 @@ | ||
| 1 | +#!/usr/bin/env python3 | |
| 2 | +"""Generate buck2 http_archive data from the DotSlash files beside this script. | |
| 3 | + | |
| 4 | +DotSlash is where a tool's version, URL and digest are written down. buck needs | |
| 5 | +the same three facts to fetch that tool itself — it cannot use the shim, which | |
| 6 | +resolves on the machine at run time and so is neither an action input nor | |
| 7 | +anything a remote worker could be given. Rather than write them twice and let | |
| 8 | +them drift, this reads the manifests and emits the table buck reads. | |
| 9 | + | |
| 10 | + just sync-dist # after editing any scripts/*.dotslash | |
| 11 | + | |
| 12 | +The DotSlash format is JSON with a `//` shebang line on top: one entry per | |
| 13 | +platform, each with a url, a size, a digest and the path of the binary inside | |
| 14 | +the archive. Only the archive itself is wanted here; the path inside it is the | |
| 15 | +shim's business, and the toolchain rules name what they need. | |
| 16 | +""" | |
| 17 | + | |
| 18 | +import json | |
| 19 | +import pathlib | |
| 20 | +import sys | |
| 21 | + | |
| 22 | +HERE = pathlib.Path(__file__).resolve().parent | |
| 23 | +OUT = HERE.parent / "toolchains" / "dist" / "generated.bzl" | |
| 24 | + | |
| 25 | +# The tools buck fetches for itself. buck2 is not among them: it is the thing | |
| 26 | +# doing the fetching, and dotslash is how a human gets it. | |
| 27 | +TOOLS = {"zig": "zig", "rustc": "rust"} | |
| 28 | + | |
| 29 | + | |
| 30 | +def load(path): | |
| 31 | + text = path.read_text() | |
| 32 | + return json.loads(text[text.index("{") :]) | |
| 33 | + | |
| 34 | + | |
| 35 | +def main(): | |
| 36 | + entries = {} | |
| 37 | + for stem, name in sorted(TOOLS.items()): | |
| 38 | + for candidate in (HERE / f"{stem}.dotslash", HERE / stem): | |
| 39 | + if candidate.exists(): | |
| 40 | + manifest = load(candidate) | |
| 41 | + break | |
| 42 | + else: | |
| 43 | + sys.exit(f"no DotSlash manifest for {stem}") | |
| 44 | + | |
| 45 | + platforms = {} | |
| 46 | + for platform, entry in sorted(manifest["platforms"].items()): | |
| 47 | + if entry["hash"] != "sha256": | |
| 48 | + # http_archive takes sha256 and nothing else. | |
| 49 | + continue | |
| 50 | + url = entry["providers"][0]["url"] | |
| 51 | + # The path of the binary inside the archive starts with the | |
| 52 | + # directory the tarball unpacks into, which is what to strip. | |
| 53 | + platforms[platform] = { | |
| 54 | + "url": url, | |
| 55 | + "sha256": entry["digest"], | |
| 56 | + "strip_prefix": entry["path"].split("/")[0], | |
| 57 | + "type": entry["format"], | |
| 58 | + } | |
| 59 | + entries[name] = platforms | |
| 60 | + | |
| 61 | + lines = [ | |
| 62 | + "# @generated by scripts/dotslash-to-buck — do not edit.", | |
| 63 | + "#", | |
| 64 | + "# The same archives scripts/*.dotslash pins, as facts buck can act on.", | |
| 65 | + "# Bump the DotSlash file and run `just sync-dist`.", | |
| 66 | + "", | |
| 67 | + "DIST = " + json.dumps(entries, indent=4).replace(": true", ": True"), | |
| 68 | + "", | |
| 69 | + ] | |
| 70 | + OUT.parent.mkdir(parents=True, exist_ok=True) | |
| 71 | + OUT.write_text("\n".join(lines)) | |
| 72 | + print(f"wrote {OUT.relative_to(HERE.parent)}") | |
| 73 | + | |
| 74 | + | |
| 75 | +if __name__ == "__main__": | |
| 76 | + main() | |
| new file mode 100755 | |||
| @@ -0,0 +1,76 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +"""Generate buck2 http_archive data from the DotSlash files beside this script. | ||
| 3 | + | ||
| 4 | +DotSlash is where a tool's version, URL and digest are written down. buck needs | ||
| 5 | +the same three facts to fetch that tool itself — it cannot use the shim, which | ||
| 6 | +resolves on the machine at run time and so is neither an action input nor | ||
| 7 | +anything a remote worker could be given. Rather than write them twice and let | ||
| 8 | +them drift, this reads the manifests and emits the table buck reads. | ||
| 9 | + | ||
| 10 | + just sync-dist # after editing any scripts/*.dotslash | ||
| 11 | + | ||
| 12 | +The DotSlash format is JSON with a `//` shebang line on top: one entry per | ||
| 13 | +platform, each with a url, a size, a digest and the path of the binary inside | ||
| 14 | +the archive. Only the archive itself is wanted here; the path inside it is the | ||
| 15 | +shim's business, and the toolchain rules name what they need. | ||
| 16 | +""" | ||
| 17 | + | ||
| 18 | +import json | ||
| 19 | +import pathlib | ||
| 20 | +import sys | ||
| 21 | + | ||
| 22 | +HERE = pathlib.Path(__file__).resolve().parent | ||
| 23 | +OUT = HERE.parent / "toolchains" / "dist" / "generated.bzl" | ||
| 24 | + | ||
| 25 | +# The tools buck fetches for itself. buck2 is not among them: it is the thing | ||
| 26 | +# doing the fetching, and dotslash is how a human gets it. | ||
| 27 | +TOOLS = {"zig": "zig", "rustc": "rust"} | ||
| 28 | + | ||
| 29 | + | ||
| 30 | +def load(path): | ||
| 31 | + text = path.read_text() | ||
| 32 | + return json.loads(text[text.index("{") :]) | ||
| 33 | + | ||
| 34 | + | ||
| 35 | +def main(): | ||
| 36 | + entries = {} | ||
| 37 | + for stem, name in sorted(TOOLS.items()): | ||
| 38 | + for candidate in (HERE / f"{stem}.dotslash", HERE / stem): | ||
| 39 | + if candidate.exists(): | ||
| 40 | + manifest = load(candidate) | ||
| 41 | + break | ||
| 42 | + else: | ||
| 43 | + sys.exit(f"no DotSlash manifest for {stem}") | ||
| 44 | + | ||
| 45 | + platforms = {} | ||
| 46 | + for platform, entry in sorted(manifest["platforms"].items()): | ||
| 47 | + if entry["hash"] != "sha256": | ||
| 48 | + # http_archive takes sha256 and nothing else. | ||
| 49 | + continue | ||
| 50 | + url = entry["providers"][0]["url"] | ||
| 51 | + # The path of the binary inside the archive starts with the | ||
| 52 | + # directory the tarball unpacks into, which is what to strip. | ||
| 53 | + platforms[platform] = { | ||
| 54 | + "url": url, | ||
| 55 | + "sha256": entry["digest"], | ||
| 56 | + "strip_prefix": entry["path"].split("/")[0], | ||
| 57 | + "type": entry["format"], | ||
| 58 | + } | ||
| 59 | + entries[name] = platforms | ||
| 60 | + | ||
| 61 | + lines = [ | ||
| 62 | + "# @generated by scripts/dotslash-to-buck — do not edit.", | ||
| 63 | + "#", | ||
| 64 | + "# The same archives scripts/*.dotslash pins, as facts buck can act on.", | ||
| 65 | + "# Bump the DotSlash file and run `just sync-dist`.", | ||
| 66 | + "", | ||
| 67 | + "DIST = " + json.dumps(entries, indent=4).replace(": true", ": True"), | ||
| 68 | + "", | ||
| 69 | + ] | ||
| 70 | + OUT.parent.mkdir(parents=True, exist_ok=True) | ||
| 71 | + OUT.write_text("\n".join(lines)) | ||
| 72 | + print(f"wrote {OUT.relative_to(HERE.parent)}") | ||
| 73 | + | ||
| 74 | + | ||
| 75 | +if __name__ == "__main__": | ||
| 76 | + main() | ||
modified
toolchains/BUCK +32 -14 | @@ -1,7 +1,7 @@ | ||
| 1 | 1 | load("@prelude//toolchains:cxx.bzl", "system_cxx_toolchain") |
| 2 | 2 | load("@prelude//toolchains:genrule.bzl", "system_genrule_toolchain") |
| 3 | 3 | load("@prelude//toolchains:python.bzl", "system_python_bootstrap_toolchain") |
| 4 | -load("@prelude//toolchains:rust.bzl", "system_rust_toolchain") | |
| 4 | +load(":defs.bzl", "hermetic_rust_toolchain") | |
| 5 | 5 | |
| 6 | 6 | # Tools come from the DotSlash files in scripts/, so the build depends on |
| 7 | 7 | # pinned upstream releases rather than whatever the host happens to have. |
| @@ -13,6 +13,9 @@ load("@prelude//toolchains:rust.bzl", "system_rust_toolchain") | ||
| 13 | 13 | # re-exec calls os.execl rather than os.execlp, so a bare `zcc` on PATH is not |
| 14 | 14 | # found either. aws-lc-sys is where both show up. The `buck` recipe in justfile |
| 15 | 15 | # writes that file, so the absolute paths are generated rather than committed. |
| 16 | +# | |
| 17 | +# rustc does not come this way any more — see the toolchain at the bottom of | |
| 18 | +# this file. These three are what is left to make into dependencies. | |
| 16 | 19 | _SCRIPTS = read_root_config("jolt", "scripts", "scripts") |
| 17 | 20 | |
| 18 | 21 | # The compiler's version, written by the `buck` recipe alongside the paths. |
| @@ -23,7 +26,15 @@ _SCRIPTS = read_root_config("jolt", "scripts", "scripts") | ||
| 23 | 26 | # the same digest, and a shared cache will hand one build the other's rlibs; |
| 24 | 27 | # the failure is E0514, a long way from the cause. Naming the version in a cfg |
| 25 | 28 | # nothing reads is enough to keep the digests apart. |
| 26 | -_RUSTC_VERSION = read_root_config("jolt", "rustc_version", "unknown") | |
| 29 | +# The C toolchain's version, written by the `buck` recipe alongside the paths. | |
| 30 | +# | |
| 31 | +# rustc no longer needs this: it is a dependency now, so its identity is in | |
| 32 | +# every action's digest and two compilers cannot collide on one cache entry. | |
| 33 | +# zig still does. It is named by an absolute path that does not change when the | |
| 34 | +# binary behind it does, so a version bump would silently reuse entries built | |
| 35 | +# by the old one — the E0514 failure, one toolchain over. Naming the version in | |
| 36 | +# a cfg nothing reads keeps the digests apart until zig is a dependency too. | |
| 37 | +_ZIG_VERSION = read_root_config("jolt", "zig_version", "unknown") | |
| 27 | 38 | # |
| 28 | 39 | # This is the same arrangement .cargo/config.toml used to describe: zig cc |
| 29 | 40 | # carries its own glibc sysroot, which is why this repo builds on a machine |
| @@ -52,8 +63,13 @@ system_python_bootstrap_toolchain( | ||
| 52 | 63 | visibility = ["PUBLIC"], |
| 53 | 64 | ) |
| 54 | 65 | |
| 55 | -system_rust_toolchain( | |
| 66 | +# The compiler is the dist tarball in toolchains//dist, not a name resolved | |
| 67 | +# against the machine. See defs.bzl for why that distinction is the whole | |
| 68 | +# point, and dist.bzl for where the tarball comes from. | |
| 69 | +hermetic_rust_toolchain( | |
| 56 | 70 | name = "rust", |
| 71 | + dist = "toolchains//dist:rust", | |
| 72 | + wrapper = "toolchains//:rustc.sh", | |
| 57 | 73 | default_edition = "2021", |
| 58 | 74 | rustc_flags = [ |
| 59 | 75 | "-Clinker=" + _SCRIPTS + "/zcc", |
| @@ -63,17 +79,19 @@ system_rust_toolchain( | ||
| 63 | 79 | # without this the two objects are debug builds wearing release names. |
| 64 | 80 | # OPT_LEVEL=2 in the buildscript fixups is the same setting for the C. |
| 65 | 81 | "-Copt-level=2", |
| 66 | - "--cfg=jolt_rustc=\"" + _RUSTC_VERSION + "\"", | |
| 82 | + "--cfg=jolt_cc=\"" + _ZIG_VERSION + "\"", | |
| 67 | 83 | ], |
| 68 | - rustc_target_triple = select({ | |
| 69 | - "prelude//os:linux": select({ | |
| 70 | - "prelude//cpu:arm64": "aarch64-unknown-linux-gnu", | |
| 71 | - "prelude//cpu:x86_64": "x86_64-unknown-linux-gnu", | |
| 72 | - }), | |
| 73 | - "prelude//os:macos": select({ | |
| 74 | - "prelude//cpu:arm64": "aarch64-apple-darwin", | |
| 75 | - "prelude//cpu:x86_64": "x86_64-apple-darwin", | |
| 76 | - }), | |
| 77 | - }), | |
| 84 | + # A plain string rather than a select: the sysroot path inside the dist | |
| 85 | + # tarball is built from this, and only the triples dist.bzl fetches an | |
| 86 | + # archive for can be named here. | |
| 87 | + rustc_target_triple = "x86_64-unknown-linux-gnu", | |
| 88 | + visibility = ["PUBLIC"], | |
| 89 | +) | |
| 90 | + | |
| 91 | +# The wrapper the Rust toolchain runs rustc through, as an artifact so it is an | |
| 92 | +# input to every action that compiles anything. | |
| 93 | +export_file( | |
| 94 | + name = "rustc.sh", | |
| 95 | + mode = "reference", | |
| 78 | 96 | visibility = ["PUBLIC"], |
| 79 | 97 | ) |
| @@ -1,7 +1,7 @@ | |||
| 1 | load("@prelude//toolchains:cxx.bzl", "system_cxx_toolchain") | 1 | load("@prelude//toolchains:cxx.bzl", "system_cxx_toolchain") |
| 2 | load("@prelude//toolchains:genrule.bzl", "system_genrule_toolchain") | 2 | load("@prelude//toolchains:genrule.bzl", "system_genrule_toolchain") |
| 3 | load("@prelude//toolchains:python.bzl", "system_python_bootstrap_toolchain") | 3 | load("@prelude//toolchains:python.bzl", "system_python_bootstrap_toolchain") |
| 4 | -load("@prelude//toolchains:rust.bzl", "system_rust_toolchain") | 4 | +load(":defs.bzl", "hermetic_rust_toolchain") |
| 5 | 5 | ||
| 6 | # Tools come from the DotSlash files in scripts/, so the build depends on | 6 | # Tools come from the DotSlash files in scripts/, so the build depends on |
| 7 | # pinned upstream releases rather than whatever the host happens to have. | 7 | # pinned upstream releases rather than whatever the host happens to have. |
| @@ -13,6 +13,9 @@ load("@prelude//toolchains:rust.bzl", "system_rust_toolchain") | |||
| 13 | # re-exec calls os.execl rather than os.execlp, so a bare `zcc` on PATH is not | 13 | # re-exec calls os.execl rather than os.execlp, so a bare `zcc` on PATH is not |
| 14 | # found either. aws-lc-sys is where both show up. The `buck` recipe in justfile | 14 | # found either. aws-lc-sys is where both show up. The `buck` recipe in justfile |
| 15 | # writes that file, so the absolute paths are generated rather than committed. | 15 | # writes that file, so the absolute paths are generated rather than committed. |
| 16 | +# | ||
| 17 | +# rustc does not come this way any more — see the toolchain at the bottom of | ||
| 18 | +# this file. These three are what is left to make into dependencies. | ||
| 16 | _SCRIPTS = read_root_config("jolt", "scripts", "scripts") | 19 | _SCRIPTS = read_root_config("jolt", "scripts", "scripts") |
| 17 | 20 | ||
| 18 | # The compiler's version, written by the `buck` recipe alongside the paths. | 21 | # The compiler's version, written by the `buck` recipe alongside the paths. |
| @@ -23,7 +26,15 @@ _SCRIPTS = read_root_config("jolt", "scripts", "scripts") | |||
| 23 | # the same digest, and a shared cache will hand one build the other's rlibs; | 26 | # the same digest, and a shared cache will hand one build the other's rlibs; |
| 24 | # the failure is E0514, a long way from the cause. Naming the version in a cfg | 27 | # the failure is E0514, a long way from the cause. Naming the version in a cfg |
| 25 | # nothing reads is enough to keep the digests apart. | 28 | # nothing reads is enough to keep the digests apart. |
| 26 | -_RUSTC_VERSION = read_root_config("jolt", "rustc_version", "unknown") | 29 | +# The C toolchain's version, written by the `buck` recipe alongside the paths. |
| 30 | +# | ||
| 31 | +# rustc no longer needs this: it is a dependency now, so its identity is in | ||
| 32 | +# every action's digest and two compilers cannot collide on one cache entry. | ||
| 33 | +# zig still does. It is named by an absolute path that does not change when the | ||
| 34 | +# binary behind it does, so a version bump would silently reuse entries built | ||
| 35 | +# by the old one — the E0514 failure, one toolchain over. Naming the version in | ||
| 36 | +# a cfg nothing reads keeps the digests apart until zig is a dependency too. | ||
| 37 | +_ZIG_VERSION = read_root_config("jolt", "zig_version", "unknown") | ||
| 27 | # | 38 | # |
| 28 | # This is the same arrangement .cargo/config.toml used to describe: zig cc | 39 | # This is the same arrangement .cargo/config.toml used to describe: zig cc |
| 29 | # carries its own glibc sysroot, which is why this repo builds on a machine | 40 | # carries its own glibc sysroot, which is why this repo builds on a machine |
| @@ -52,8 +63,13 @@ system_python_bootstrap_toolchain( | |||
| 52 | visibility = ["PUBLIC"], | 63 | visibility = ["PUBLIC"], |
| 53 | ) | 64 | ) |
| 54 | 65 | ||
| 55 | -system_rust_toolchain( | 66 | +# The compiler is the dist tarball in toolchains//dist, not a name resolved |
| 67 | +# against the machine. See defs.bzl for why that distinction is the whole | ||
| 68 | +# point, and dist.bzl for where the tarball comes from. | ||
| 69 | +hermetic_rust_toolchain( | ||
| 56 | name = "rust", | 70 | name = "rust", |
| 71 | + dist = "toolchains//dist:rust", | ||
| 72 | + wrapper = "toolchains//:rustc.sh", | ||
| 57 | default_edition = "2021", | 73 | default_edition = "2021", |
| 58 | rustc_flags = [ | 74 | rustc_flags = [ |
| 59 | "-Clinker=" + _SCRIPTS + "/zcc", | 75 | "-Clinker=" + _SCRIPTS + "/zcc", |
| @@ -63,17 +79,19 @@ system_rust_toolchain( | |||
| 63 | # without this the two objects are debug builds wearing release names. | 79 | # without this the two objects are debug builds wearing release names. |
| 64 | # OPT_LEVEL=2 in the buildscript fixups is the same setting for the C. | 80 | # OPT_LEVEL=2 in the buildscript fixups is the same setting for the C. |
| 65 | "-Copt-level=2", | 81 | "-Copt-level=2", |
| 66 | - "--cfg=jolt_rustc=\"" + _RUSTC_VERSION + "\"", | 82 | + "--cfg=jolt_cc=\"" + _ZIG_VERSION + "\"", |
| 67 | ], | 83 | ], |
| 68 | - rustc_target_triple = select({ | 84 | + # A plain string rather than a select: the sysroot path inside the dist |
| 69 | - "prelude//os:linux": select({ | 85 | + # tarball is built from this, and only the triples dist.bzl fetches an |
| 70 | - "prelude//cpu:arm64": "aarch64-unknown-linux-gnu", | 86 | + # archive for can be named here. |
| 71 | - "prelude//cpu:x86_64": "x86_64-unknown-linux-gnu", | 87 | + rustc_target_triple = "x86_64-unknown-linux-gnu", |
| 72 | - }), | 88 | + visibility = ["PUBLIC"], |
| 73 | - "prelude//os:macos": select({ | 89 | +) |
| 74 | - "prelude//cpu:arm64": "aarch64-apple-darwin", | 90 | + |
| 75 | - "prelude//cpu:x86_64": "x86_64-apple-darwin", | 91 | +# The wrapper the Rust toolchain runs rustc through, as an artifact so it is an |
| 76 | - }), | 92 | +# input to every action that compiles anything. |
| 77 | - }), | 93 | +export_file( |
| 94 | + name = "rustc.sh", | ||
| 95 | + mode = "reference", | ||
| 78 | visibility = ["PUBLIC"], | 96 | visibility = ["PUBLIC"], |
| 79 | ) | 97 | ) |
added
toolchains/defs.bzl +73 -0 | new file mode 100644 | ||
| @@ -0,0 +1,73 @@ | ||
| 1 | +# A Rust toolchain whose compiler is a dependency rather than a name. | |
| 2 | +# | |
| 3 | +# The prelude's system_rust_toolchain hands RustToolchainInfo | |
| 4 | +# `RunInfo(args = ["rustc"])` — a bare string. Nothing about the compiler is an | |
| 5 | +# input to any action, so nothing about it reaches the action's cache digest, | |
| 6 | +# and there is nothing for a remote worker to materialise. Both of this repo's | |
| 7 | +# problems come from that one line: a shared cache cannot tell two compilers | |
| 8 | +# apart, and remote execution has no compiler to run. | |
| 9 | +# | |
| 10 | +# Here the dist tarball is a dep. cmd_args carries the archive's directory into | |
| 11 | +# every rustc invocation, so buck knows the action depends on it. | |
| 12 | +load("@prelude//rust:rust_toolchain.bzl", "PanicRuntime", "RustToolchainInfo") | |
| 13 | + | |
| 14 | +def _hermetic_rust_toolchain_impl(ctx): | |
| 15 | + dist = ctx.attrs.dist[DefaultInfo].default_outputs[0] | |
| 16 | + | |
| 17 | + # The dist tarball keeps its components in sibling directories, so the | |
| 18 | + # compiler's default sysroot (rustc/) holds no libstd — it lives under | |
| 19 | + # rust-std-<triple>/. The same stitching scripts/rustc does by hand, said | |
| 20 | + # to buck instead. The compiler's own codegen libraries are found by rpath, | |
| 21 | + # relative to the binary, so only the sysroot has to be named. | |
| 22 | + # rustc goes through a wrapper because the sysroot cannot travel any other | |
| 23 | + # way: rustc_flags takes plain strings, so an artifact path cannot be put | |
| 24 | + # there, and putting --sysroot on the compiler unconditionally breaks | |
| 25 | + # buck's own rustc_cfg action, which passes an empty one. Both the wrapper | |
| 26 | + # and the tarball are inputs, which is the point. | |
| 27 | + wrapper = ctx.attrs.wrapper[DefaultInfo].default_outputs[0] | |
| 28 | + compiler = cmd_args(wrapper, dist, ctx.attrs.rustc_target_triple) | |
| 29 | + rustdoc = cmd_args(dist, format = "{}/rustc/bin/rustdoc") | |
| 30 | + | |
| 31 | + return [ | |
| 32 | + DefaultInfo(), | |
| 33 | + RustToolchainInfo( | |
| 34 | + allow_lints = ctx.attrs.allow_lints, | |
| 35 | + clippy_driver = RunInfo(args = cmd_args(dist, format = "{}/clippy/bin/clippy-driver")), | |
| 36 | + clippy_toml = None, | |
| 37 | + compiler = RunInfo(args = compiler), | |
| 38 | + default_edition = ctx.attrs.default_edition, | |
| 39 | + panic_runtime = PanicRuntime("unwind"), | |
| 40 | + deny_lints = ctx.attrs.deny_lints, | |
| 41 | + doctests = ctx.attrs.doctests, | |
| 42 | + nightly_features = ctx.attrs.nightly_features, | |
| 43 | + report_unused_deps = ctx.attrs.report_unused_deps, | |
| 44 | + rustc_binary_flags = ctx.attrs.rustc_binary_flags, | |
| 45 | + rustc_flags = ctx.attrs.rustc_flags, | |
| 46 | + rustc_target_triple = ctx.attrs.rustc_target_triple, | |
| 47 | + rustc_test_flags = ctx.attrs.rustc_test_flags, | |
| 48 | + rustdoc = RunInfo(args = rustdoc), | |
| 49 | + rustdoc_flags = ctx.attrs.rustdoc_flags, | |
| 50 | + warn_lints = ctx.attrs.warn_lints, | |
| 51 | + ), | |
| 52 | + ] | |
| 53 | + | |
| 54 | +hermetic_rust_toolchain = rule( | |
| 55 | + impl = _hermetic_rust_toolchain_impl, | |
| 56 | + attrs = { | |
| 57 | + "allow_lints": attrs.list(attrs.string(), default = []), | |
| 58 | + "default_edition": attrs.option(attrs.string(), default = None), | |
| 59 | + "deny_lints": attrs.list(attrs.string(), default = []), | |
| 60 | + "dist": attrs.dep(providers = [DefaultInfo]), | |
| 61 | + "wrapper": attrs.dep(providers = [DefaultInfo]), | |
| 62 | + "doctests": attrs.bool(default = False), | |
| 63 | + "nightly_features": attrs.bool(default = True), | |
| 64 | + "report_unused_deps": attrs.bool(default = False), | |
| 65 | + "rustc_binary_flags": attrs.list(attrs.arg(), default = []), | |
| 66 | + "rustc_flags": attrs.list(attrs.arg(), default = []), | |
| 67 | + "rustc_target_triple": attrs.string(), | |
| 68 | + "rustc_test_flags": attrs.list(attrs.arg(), default = []), | |
| 69 | + "rustdoc_flags": attrs.list(attrs.arg(), default = []), | |
| 70 | + "warn_lints": attrs.list(attrs.string(), default = []), | |
| 71 | + }, | |
| 72 | + is_toolchain_rule = True, | |
| 73 | +) | |
| new file mode 100644 | |||
| @@ -0,0 +1,73 @@ | |||
| 1 | +# A Rust toolchain whose compiler is a dependency rather than a name. | ||
| 2 | +# | ||
| 3 | +# The prelude's system_rust_toolchain hands RustToolchainInfo | ||
| 4 | +# `RunInfo(args = ["rustc"])` — a bare string. Nothing about the compiler is an | ||
| 5 | +# input to any action, so nothing about it reaches the action's cache digest, | ||
| 6 | +# and there is nothing for a remote worker to materialise. Both of this repo's | ||
| 7 | +# problems come from that one line: a shared cache cannot tell two compilers | ||
| 8 | +# apart, and remote execution has no compiler to run. | ||
| 9 | +# | ||
| 10 | +# Here the dist tarball is a dep. cmd_args carries the archive's directory into | ||
| 11 | +# every rustc invocation, so buck knows the action depends on it. | ||
| 12 | +load("@prelude//rust:rust_toolchain.bzl", "PanicRuntime", "RustToolchainInfo") | ||
| 13 | + | ||
| 14 | +def _hermetic_rust_toolchain_impl(ctx): | ||
| 15 | + dist = ctx.attrs.dist[DefaultInfo].default_outputs[0] | ||
| 16 | + | ||
| 17 | + # The dist tarball keeps its components in sibling directories, so the | ||
| 18 | + # compiler's default sysroot (rustc/) holds no libstd — it lives under | ||
| 19 | + # rust-std-<triple>/. The same stitching scripts/rustc does by hand, said | ||
| 20 | + # to buck instead. The compiler's own codegen libraries are found by rpath, | ||
| 21 | + # relative to the binary, so only the sysroot has to be named. | ||
| 22 | + # rustc goes through a wrapper because the sysroot cannot travel any other | ||
| 23 | + # way: rustc_flags takes plain strings, so an artifact path cannot be put | ||
| 24 | + # there, and putting --sysroot on the compiler unconditionally breaks | ||
| 25 | + # buck's own rustc_cfg action, which passes an empty one. Both the wrapper | ||
| 26 | + # and the tarball are inputs, which is the point. | ||
| 27 | + wrapper = ctx.attrs.wrapper[DefaultInfo].default_outputs[0] | ||
| 28 | + compiler = cmd_args(wrapper, dist, ctx.attrs.rustc_target_triple) | ||
| 29 | + rustdoc = cmd_args(dist, format = "{}/rustc/bin/rustdoc") | ||
| 30 | + | ||
| 31 | + return [ | ||
| 32 | + DefaultInfo(), | ||
| 33 | + RustToolchainInfo( | ||
| 34 | + allow_lints = ctx.attrs.allow_lints, | ||
| 35 | + clippy_driver = RunInfo(args = cmd_args(dist, format = "{}/clippy/bin/clippy-driver")), | ||
| 36 | + clippy_toml = None, | ||
| 37 | + compiler = RunInfo(args = compiler), | ||
| 38 | + default_edition = ctx.attrs.default_edition, | ||
| 39 | + panic_runtime = PanicRuntime("unwind"), | ||
| 40 | + deny_lints = ctx.attrs.deny_lints, | ||
| 41 | + doctests = ctx.attrs.doctests, | ||
| 42 | + nightly_features = ctx.attrs.nightly_features, | ||
| 43 | + report_unused_deps = ctx.attrs.report_unused_deps, | ||
| 44 | + rustc_binary_flags = ctx.attrs.rustc_binary_flags, | ||
| 45 | + rustc_flags = ctx.attrs.rustc_flags, | ||
| 46 | + rustc_target_triple = ctx.attrs.rustc_target_triple, | ||
| 47 | + rustc_test_flags = ctx.attrs.rustc_test_flags, | ||
| 48 | + rustdoc = RunInfo(args = rustdoc), | ||
| 49 | + rustdoc_flags = ctx.attrs.rustdoc_flags, | ||
| 50 | + warn_lints = ctx.attrs.warn_lints, | ||
| 51 | + ), | ||
| 52 | + ] | ||
| 53 | + | ||
| 54 | +hermetic_rust_toolchain = rule( | ||
| 55 | + impl = _hermetic_rust_toolchain_impl, | ||
| 56 | + attrs = { | ||
| 57 | + "allow_lints": attrs.list(attrs.string(), default = []), | ||
| 58 | + "default_edition": attrs.option(attrs.string(), default = None), | ||
| 59 | + "deny_lints": attrs.list(attrs.string(), default = []), | ||
| 60 | + "dist": attrs.dep(providers = [DefaultInfo]), | ||
| 61 | + "wrapper": attrs.dep(providers = [DefaultInfo]), | ||
| 62 | + "doctests": attrs.bool(default = False), | ||
| 63 | + "nightly_features": attrs.bool(default = True), | ||
| 64 | + "report_unused_deps": attrs.bool(default = False), | ||
| 65 | + "rustc_binary_flags": attrs.list(attrs.arg(), default = []), | ||
| 66 | + "rustc_flags": attrs.list(attrs.arg(), default = []), | ||
| 67 | + "rustc_target_triple": attrs.string(), | ||
| 68 | + "rustc_test_flags": attrs.list(attrs.arg(), default = []), | ||
| 69 | + "rustdoc_flags": attrs.list(attrs.arg(), default = []), | ||
| 70 | + "warn_lints": attrs.list(attrs.string(), default = []), | ||
| 71 | + }, | ||
| 72 | + is_toolchain_rule = True, | ||
| 73 | +) | ||
modified
toolchains/dist.bzl +23 -44 | @@ -1,54 +1,33 @@ | ||
| 1 | 1 | # The compilers, as build artifacts rather than things found on the machine. |
| 2 | 2 | # |
| 3 | -# scripts/*.dotslash pins the same tarballs for humans; these fetch them into | |
| 4 | -# buck's own tree so that every action's inputs include the compiler that ran | |
| 5 | -# it. Two things follow. An action's cache digest now covers the toolchain, so | |
| 6 | -# a compiler change can no longer collide with the old entries — the failure | |
| 7 | -# that produced E0514 "please recompile that crate" against a shared cache. | |
| 8 | -# And a remote worker can materialise the tools, which is what remote | |
| 9 | -# execution needs and what a host path can never give it. | |
| 3 | +# scripts/*.dotslash is where a tool's version and digest are written down, and | |
| 4 | +# it stays that way: generated.bzl beside this file is derived from those | |
| 5 | +# manifests by scripts/dotslash-to-buck, so there is one place to bump a | |
| 6 | +# version and no second copy of a digest to drift. | |
| 10 | 7 | # |
| 11 | -# Keep the versions and digests here equal to the ones in scripts/. | |
| 12 | -RUST_VERSION = "1.98.0" | |
| 13 | -RUST_DATE = "2026-08-20" | |
| 14 | -ZIG_VERSION = "0.16.0" | |
| 8 | +# buck cannot use the shims themselves. DotSlash resolves a tool at run time, | |
| 9 | +# on the machine running it, against a cache in the user's home — so the tool | |
| 10 | +# is not an input to any action, is not in any action's cache digest, and is | |
| 11 | +# not something a remote worker can be handed. Fetching the identical archive | |
| 12 | +# by the identical digest is how buck gets those three properties without | |
| 13 | +# taking the pinning away from DotSlash. | |
| 14 | +load("//dist:generated.bzl", "DIST") | |
| 15 | 15 | |
| 16 | -_RUST_DIST = { | |
| 17 | - "x86_64-unknown-linux-gnu": "aa30409afa67bd1ada244cefd82c7980e6a65bc113bb978e934b2413c75e3900", | |
| 18 | - "aarch64-unknown-linux-gnu": "5fbb4282403046d52a4672765c6761a809bf9f33e699b17e6eb7a93ab7770cc3", | |
| 19 | -} | |
| 20 | - | |
| 21 | -_ZIG_DIST = { | |
| 22 | - "x86_64-linux": "70e49664a74374b48b51e6f3fdfbf437f6395d42509050588bd49abe52ba3d00", | |
| 23 | - "aarch64-linux": "ea4b09bfb22ec6f6c6ceac57ab63efb6b46e17ab08d21f69f3a48b38e1534f17", | |
| 24 | -} | |
| 25 | - | |
| 26 | -def rust_dist(name, triple): | |
| 27 | - """The whole Rust dist tarball: rustc, rustdoc, cargo and the std components.""" | |
| 16 | +def _archive(name, tool, platform): | |
| 17 | + entry = DIST[tool][platform] | |
| 28 | 18 | native.http_archive( |
| 29 | 19 | name = name, |
| 30 | - urls = ["https://static.rust-lang.org/dist/{}/rust-{}-{}.tar.gz".format( | |
| 31 | - RUST_DATE, | |
| 32 | - RUST_VERSION, | |
| 33 | - triple, | |
| 34 | - )], | |
| 35 | - sha256 = _RUST_DIST[triple], | |
| 36 | - strip_prefix = "rust-{}-{}".format(RUST_VERSION, triple), | |
| 37 | - type = "tar.gz", | |
| 20 | + urls = [entry["url"]], | |
| 21 | + sha256 = entry["sha256"], | |
| 22 | + strip_prefix = entry["strip_prefix"], | |
| 23 | + type = entry["type"], | |
| 38 | 24 | visibility = ["PUBLIC"], |
| 39 | 25 | ) |
| 40 | 26 | |
| 41 | -def zig_dist(name, triple): | |
| 27 | +def rust_dist(name, platform): | |
| 28 | + """The Rust dist tarball: rustc, rustdoc and the std components.""" | |
| 29 | + _archive(name, "rust", platform) | |
| 30 | + | |
| 31 | +def zig_dist(name, platform): | |
| 42 | 32 | """zig, which is the C and C++ compiler, the linker and the archiver.""" |
| 43 | - native.http_archive( | |
| 44 | - name = name, | |
| 45 | - urls = ["https://ziglang.org/download/{}/zig-{}-{}.tar.xz".format( | |
| 46 | - ZIG_VERSION, | |
| 47 | - triple, | |
| 48 | - ZIG_VERSION, | |
| 49 | - )], | |
| 50 | - sha256 = _ZIG_DIST[triple], | |
| 51 | - strip_prefix = "zig-{}-{}".format(triple, ZIG_VERSION), | |
| 52 | - type = "tar.xz", | |
| 53 | - visibility = ["PUBLIC"], | |
| 54 | - ) | |
| 33 | + _archive(name, "zig", platform) | |
| @@ -1,54 +1,33 @@ | |||
| 1 | # The compilers, as build artifacts rather than things found on the machine. | 1 | # The compilers, as build artifacts rather than things found on the machine. |
| 2 | # | 2 | # |
| 3 | -# scripts/*.dotslash pins the same tarballs for humans; these fetch them into | 3 | +# scripts/*.dotslash is where a tool's version and digest are written down, and |
| 4 | -# buck's own tree so that every action's inputs include the compiler that ran | 4 | +# it stays that way: generated.bzl beside this file is derived from those |
| 5 | -# it. Two things follow. An action's cache digest now covers the toolchain, so | 5 | +# manifests by scripts/dotslash-to-buck, so there is one place to bump a |
| 6 | -# a compiler change can no longer collide with the old entries — the failure | 6 | +# version and no second copy of a digest to drift. |
| 7 | -# that produced E0514 "please recompile that crate" against a shared cache. | ||
| 8 | -# And a remote worker can materialise the tools, which is what remote | ||
| 9 | -# execution needs and what a host path can never give it. | ||
| 10 | # | 7 | # |
| 11 | -# Keep the versions and digests here equal to the ones in scripts/. | 8 | +# buck cannot use the shims themselves. DotSlash resolves a tool at run time, |
| 12 | -RUST_VERSION = "1.98.0" | 9 | +# on the machine running it, against a cache in the user's home — so the tool |
| 13 | -RUST_DATE = "2026-08-20" | 10 | +# is not an input to any action, is not in any action's cache digest, and is |
| 14 | -ZIG_VERSION = "0.16.0" | 11 | +# not something a remote worker can be handed. Fetching the identical archive |
| 12 | +# by the identical digest is how buck gets those three properties without | ||
| 13 | +# taking the pinning away from DotSlash. | ||
| 14 | +load("//dist:generated.bzl", "DIST") | ||
| 15 | 15 | ||
| 16 | -_RUST_DIST = { | 16 | +def _archive(name, tool, platform): |
| 17 | - "x86_64-unknown-linux-gnu": "aa30409afa67bd1ada244cefd82c7980e6a65bc113bb978e934b2413c75e3900", | 17 | + entry = DIST[tool][platform] |
| 18 | - "aarch64-unknown-linux-gnu": "5fbb4282403046d52a4672765c6761a809bf9f33e699b17e6eb7a93ab7770cc3", | ||
| 19 | -} | ||
| 20 | - | ||
| 21 | -_ZIG_DIST = { | ||
| 22 | - "x86_64-linux": "70e49664a74374b48b51e6f3fdfbf437f6395d42509050588bd49abe52ba3d00", | ||
| 23 | - "aarch64-linux": "ea4b09bfb22ec6f6c6ceac57ab63efb6b46e17ab08d21f69f3a48b38e1534f17", | ||
| 24 | -} | ||
| 25 | - | ||
| 26 | -def rust_dist(name, triple): | ||
| 27 | - """The whole Rust dist tarball: rustc, rustdoc, cargo and the std components.""" | ||
| 28 | native.http_archive( | 18 | native.http_archive( |
| 29 | name = name, | 19 | name = name, |
| 30 | - urls = ["https://static.rust-lang.org/dist/{}/rust-{}-{}.tar.gz".format( | 20 | + urls = [entry["url"]], |
| 31 | - RUST_DATE, | 21 | + sha256 = entry["sha256"], |
| 32 | - RUST_VERSION, | 22 | + strip_prefix = entry["strip_prefix"], |
| 33 | - triple, | 23 | + type = entry["type"], |
| 34 | - )], | ||
| 35 | - sha256 = _RUST_DIST[triple], | ||
| 36 | - strip_prefix = "rust-{}-{}".format(RUST_VERSION, triple), | ||
| 37 | - type = "tar.gz", | ||
| 38 | visibility = ["PUBLIC"], | 24 | visibility = ["PUBLIC"], |
| 39 | ) | 25 | ) |
| 40 | 26 | ||
| 41 | -def zig_dist(name, triple): | 27 | +def rust_dist(name, platform): |
| 28 | + """The Rust dist tarball: rustc, rustdoc and the std components.""" | ||
| 29 | + _archive(name, "rust", platform) | ||
| 30 | + | ||
| 31 | +def zig_dist(name, platform): | ||
| 42 | """zig, which is the C and C++ compiler, the linker and the archiver.""" | 32 | """zig, which is the C and C++ compiler, the linker and the archiver.""" |
| 43 | - native.http_archive( | 33 | + _archive(name, "zig", platform) |
| 44 | - name = name, | ||
| 45 | - urls = ["https://ziglang.org/download/{}/zig-{}-{}.tar.xz".format( | ||
| 46 | - ZIG_VERSION, | ||
| 47 | - triple, | ||
| 48 | - ZIG_VERSION, | ||
| 49 | - )], | ||
| 50 | - sha256 = _ZIG_DIST[triple], | ||
| 51 | - strip_prefix = "zig-{}-{}".format(triple, ZIG_VERSION), | ||
| 52 | - type = "tar.xz", | ||
| 53 | - visibility = ["PUBLIC"], | ||
| 54 | - ) | ||
added
toolchains/dist/BUCK +7 -0 | new file mode 100644 | ||
| @@ -0,0 +1,7 @@ | ||
| 1 | +# The pinned toolchains, fetched by buck rather than by the shims. See | |
| 2 | +# ../dist.bzl; the versions and digests come from scripts/*.dotslash. | |
| 3 | +load("//:dist.bzl", "rust_dist", "zig_dist") | |
| 4 | + | |
| 5 | +rust_dist(name = "rust", platform = "linux-x86_64") | |
| 6 | + | |
| 7 | +zig_dist(name = "zig", platform = "linux-x86_64") | |
| new file mode 100644 | |||
| @@ -0,0 +1,7 @@ | |||
| 1 | +# The pinned toolchains, fetched by buck rather than by the shims. See | ||
| 2 | +# ../dist.bzl; the versions and digests come from scripts/*.dotslash. | ||
| 3 | +load("//:dist.bzl", "rust_dist", "zig_dist") | ||
| 4 | + | ||
| 5 | +rust_dist(name = "rust", platform = "linux-x86_64") | ||
| 6 | + | ||
| 7 | +zig_dist(name = "zig", platform = "linux-x86_64") | ||
added
toolchains/dist/generated.bzl +65 -0 | new file mode 100644 | ||
| @@ -0,0 +1,65 @@ | ||
| 1 | +# @generated by scripts/dotslash-to-buck — do not edit. | |
| 2 | +# | |
| 3 | +# The same archives scripts/*.dotslash pins, as facts buck can act on. | |
| 4 | +# Bump the DotSlash file and run `just sync-dist`. | |
| 5 | + | |
| 6 | +DIST = { | |
| 7 | + "rust": { | |
| 8 | + "linux-aarch64": { | |
| 9 | + "url": "https://static.rust-lang.org/dist/2026-08-20/rust-1.98.0-aarch64-unknown-linux-gnu.tar.gz", | |
| 10 | + "sha256": "5fbb4282403046d52a4672765c6761a809bf9f33e699b17e6eb7a93ab7770cc3", | |
| 11 | + "strip_prefix": "rust-1.98.0-aarch64-unknown-linux-gnu", | |
| 12 | + "type": "tar.gz" | |
| 13 | + }, | |
| 14 | + "linux-x86_64": { | |
| 15 | + "url": "https://static.rust-lang.org/dist/2026-08-20/rust-1.98.0-x86_64-unknown-linux-gnu.tar.gz", | |
| 16 | + "sha256": "aa30409afa67bd1ada244cefd82c7980e6a65bc113bb978e934b2413c75e3900", | |
| 17 | + "strip_prefix": "rust-1.98.0-x86_64-unknown-linux-gnu", | |
| 18 | + "type": "tar.gz" | |
| 19 | + }, | |
| 20 | + "macos-aarch64": { | |
| 21 | + "url": "https://static.rust-lang.org/dist/2026-08-20/rust-1.98.0-aarch64-apple-darwin.tar.gz", | |
| 22 | + "sha256": "026ec75bec81fb8c10b11df98f1336ee39f4bc11e04d6745dffe9fca81e5c0b5", | |
| 23 | + "strip_prefix": "rust-1.98.0-aarch64-apple-darwin", | |
| 24 | + "type": "tar.gz" | |
| 25 | + }, | |
| 26 | + "macos-x86_64": { | |
| 27 | + "url": "https://static.rust-lang.org/dist/2026-08-20/rust-1.98.0-x86_64-apple-darwin.tar.gz", | |
| 28 | + "sha256": "66f4e2e17275753deaba8437380de21072b56b5a083cf954bacb6376df0834fc", | |
| 29 | + "strip_prefix": "rust-1.98.0-x86_64-apple-darwin", | |
| 30 | + "type": "tar.gz" | |
| 31 | + } | |
| 32 | + }, | |
| 33 | + "zig": { | |
| 34 | + "linux-aarch64": { | |
| 35 | + "url": "https://ziglang.org/download/0.16.0/zig-aarch64-linux-0.16.0.tar.xz", | |
| 36 | + "sha256": "ea4b09bfb22ec6f6c6ceac57ab63efb6b46e17ab08d21f69f3a48b38e1534f17", | |
| 37 | + "strip_prefix": "zig-aarch64-linux-0.16.0", | |
| 38 | + "type": "tar.xz" | |
| 39 | + }, | |
| 40 | + "linux-x86_64": { | |
| 41 | + "url": "https://ziglang.org/download/0.16.0/zig-x86_64-linux-0.16.0.tar.xz", | |
| 42 | + "sha256": "70e49664a74374b48b51e6f3fdfbf437f6395d42509050588bd49abe52ba3d00", | |
| 43 | + "strip_prefix": "zig-x86_64-linux-0.16.0", | |
| 44 | + "type": "tar.xz" | |
| 45 | + }, | |
| 46 | + "macos-aarch64": { | |
| 47 | + "url": "https://ziglang.org/download/0.16.0/zig-aarch64-macos-0.16.0.tar.xz", | |
| 48 | + "sha256": "b23d70deaa879b5c2d486ed3316f7eaa53e84acf6fc9cc747de152450d401489", | |
| 49 | + "strip_prefix": "zig-aarch64-macos-0.16.0", | |
| 50 | + "type": "tar.xz" | |
| 51 | + }, | |
| 52 | + "macos-x86_64": { | |
| 53 | + "url": "https://ziglang.org/download/0.16.0/zig-x86_64-macos-0.16.0.tar.xz", | |
| 54 | + "sha256": "0387557ed1877bc6a2e1802c8391953baddba76081876301c522f52977b52ba7", | |
| 55 | + "strip_prefix": "zig-x86_64-macos-0.16.0", | |
| 56 | + "type": "tar.xz" | |
| 57 | + }, | |
| 58 | + "windows-x86_64": { | |
| 59 | + "url": "https://ziglang.org/download/0.16.0/zig-x86_64-windows-0.16.0.zip", | |
| 60 | + "sha256": "68659eb5f1e4eb1437a722f1dd889c5a322c9954607f5edcf337bc3684a75a7e", | |
| 61 | + "strip_prefix": "zig-x86_64-windows-0.16.0", | |
| 62 | + "type": "zip" | |
| 63 | + } | |
| 64 | + } | |
| 65 | +} | |
| new file mode 100644 | |||
| @@ -0,0 +1,65 @@ | |||
| 1 | +# @generated by scripts/dotslash-to-buck — do not edit. | ||
| 2 | +# | ||
| 3 | +# The same archives scripts/*.dotslash pins, as facts buck can act on. | ||
| 4 | +# Bump the DotSlash file and run `just sync-dist`. | ||
| 5 | + | ||
| 6 | +DIST = { | ||
| 7 | + "rust": { | ||
| 8 | + "linux-aarch64": { | ||
| 9 | + "url": "https://static.rust-lang.org/dist/2026-08-20/rust-1.98.0-aarch64-unknown-linux-gnu.tar.gz", | ||
| 10 | + "sha256": "5fbb4282403046d52a4672765c6761a809bf9f33e699b17e6eb7a93ab7770cc3", | ||
| 11 | + "strip_prefix": "rust-1.98.0-aarch64-unknown-linux-gnu", | ||
| 12 | + "type": "tar.gz" | ||
| 13 | + }, | ||
| 14 | + "linux-x86_64": { | ||
| 15 | + "url": "https://static.rust-lang.org/dist/2026-08-20/rust-1.98.0-x86_64-unknown-linux-gnu.tar.gz", | ||
| 16 | + "sha256": "aa30409afa67bd1ada244cefd82c7980e6a65bc113bb978e934b2413c75e3900", | ||
| 17 | + "strip_prefix": "rust-1.98.0-x86_64-unknown-linux-gnu", | ||
| 18 | + "type": "tar.gz" | ||
| 19 | + }, | ||
| 20 | + "macos-aarch64": { | ||
| 21 | + "url": "https://static.rust-lang.org/dist/2026-08-20/rust-1.98.0-aarch64-apple-darwin.tar.gz", | ||
| 22 | + "sha256": "026ec75bec81fb8c10b11df98f1336ee39f4bc11e04d6745dffe9fca81e5c0b5", | ||
| 23 | + "strip_prefix": "rust-1.98.0-aarch64-apple-darwin", | ||
| 24 | + "type": "tar.gz" | ||
| 25 | + }, | ||
| 26 | + "macos-x86_64": { | ||
| 27 | + "url": "https://static.rust-lang.org/dist/2026-08-20/rust-1.98.0-x86_64-apple-darwin.tar.gz", | ||
| 28 | + "sha256": "66f4e2e17275753deaba8437380de21072b56b5a083cf954bacb6376df0834fc", | ||
| 29 | + "strip_prefix": "rust-1.98.0-x86_64-apple-darwin", | ||
| 30 | + "type": "tar.gz" | ||
| 31 | + } | ||
| 32 | + }, | ||
| 33 | + "zig": { | ||
| 34 | + "linux-aarch64": { | ||
| 35 | + "url": "https://ziglang.org/download/0.16.0/zig-aarch64-linux-0.16.0.tar.xz", | ||
| 36 | + "sha256": "ea4b09bfb22ec6f6c6ceac57ab63efb6b46e17ab08d21f69f3a48b38e1534f17", | ||
| 37 | + "strip_prefix": "zig-aarch64-linux-0.16.0", | ||
| 38 | + "type": "tar.xz" | ||
| 39 | + }, | ||
| 40 | + "linux-x86_64": { | ||
| 41 | + "url": "https://ziglang.org/download/0.16.0/zig-x86_64-linux-0.16.0.tar.xz", | ||
| 42 | + "sha256": "70e49664a74374b48b51e6f3fdfbf437f6395d42509050588bd49abe52ba3d00", | ||
| 43 | + "strip_prefix": "zig-x86_64-linux-0.16.0", | ||
| 44 | + "type": "tar.xz" | ||
| 45 | + }, | ||
| 46 | + "macos-aarch64": { | ||
| 47 | + "url": "https://ziglang.org/download/0.16.0/zig-aarch64-macos-0.16.0.tar.xz", | ||
| 48 | + "sha256": "b23d70deaa879b5c2d486ed3316f7eaa53e84acf6fc9cc747de152450d401489", | ||
| 49 | + "strip_prefix": "zig-aarch64-macos-0.16.0", | ||
| 50 | + "type": "tar.xz" | ||
| 51 | + }, | ||
| 52 | + "macos-x86_64": { | ||
| 53 | + "url": "https://ziglang.org/download/0.16.0/zig-x86_64-macos-0.16.0.tar.xz", | ||
| 54 | + "sha256": "0387557ed1877bc6a2e1802c8391953baddba76081876301c522f52977b52ba7", | ||
| 55 | + "strip_prefix": "zig-x86_64-macos-0.16.0", | ||
| 56 | + "type": "tar.xz" | ||
| 57 | + }, | ||
| 58 | + "windows-x86_64": { | ||
| 59 | + "url": "https://ziglang.org/download/0.16.0/zig-x86_64-windows-0.16.0.zip", | ||
| 60 | + "sha256": "68659eb5f1e4eb1437a722f1dd889c5a322c9954607f5edcf337bc3684a75a7e", | ||
| 61 | + "strip_prefix": "zig-x86_64-windows-0.16.0", | ||
| 62 | + "type": "zip" | ||
| 63 | + } | ||
| 64 | + } | ||
| 65 | +} | ||
added
toolchains/rustc.sh +29 -0 | new file mode 100755 | ||
| @@ -0,0 +1,29 @@ | ||
| 1 | +#!/bin/sh | |
| 2 | +# rustc out of the dist tarball buck fetched, with the sysroot stitched on. | |
| 3 | +# | |
| 4 | +# The same job scripts/rustc does, said in a way buck can ship: the wrapper and | |
| 5 | +# the tarball are both inputs to every action that runs it, so the compiler's | |
| 6 | +# identity is part of the action's cache digest and a remote worker can be | |
| 7 | +# handed the whole thing. | |
| 8 | +# | |
| 9 | +# $1 is the directory the tarball unpacked into, $2 the target triple; buck | |
| 10 | +# passes both because neither is knowable from inside the script. | |
| 11 | +set -e | |
| 12 | +dist=$1 | |
| 13 | +triple=$2 | |
| 14 | +shift 2 | |
| 15 | + | |
| 16 | +# The dist tarball keeps its components in sibling directories, so the | |
| 17 | +# compiler's own default sysroot (rustc/) holds no libstd — it lives under | |
| 18 | +# rust-std-<triple>/. The codegen libraries are found by rpath relative to the | |
| 19 | +# binary, so only the sysroot has to be named. | |
| 20 | +# | |
| 21 | +# Callers that set their own win: buck's rustc_cfg action passes an empty | |
| 22 | +# --sysroot, and rustc refuses the option twice. | |
| 23 | +for arg do | |
| 24 | + case $arg in | |
| 25 | + --sysroot|--sysroot=*) exec "$dist/rustc/bin/rustc" "$@" ;; | |
| 26 | + esac | |
| 27 | +done | |
| 28 | + | |
| 29 | +exec "$dist/rustc/bin/rustc" --sysroot "$dist/rust-std-$triple" "$@" | |
| new file mode 100755 | |||
| @@ -0,0 +1,29 @@ | |||
| 1 | +#!/bin/sh | ||
| 2 | +# rustc out of the dist tarball buck fetched, with the sysroot stitched on. | ||
| 3 | +# | ||
| 4 | +# The same job scripts/rustc does, said in a way buck can ship: the wrapper and | ||
| 5 | +# the tarball are both inputs to every action that runs it, so the compiler's | ||
| 6 | +# identity is part of the action's cache digest and a remote worker can be | ||
| 7 | +# handed the whole thing. | ||
| 8 | +# | ||
| 9 | +# $1 is the directory the tarball unpacked into, $2 the target triple; buck | ||
| 10 | +# passes both because neither is knowable from inside the script. | ||
| 11 | +set -e | ||
| 12 | +dist=$1 | ||
| 13 | +triple=$2 | ||
| 14 | +shift 2 | ||
| 15 | + | ||
| 16 | +# The dist tarball keeps its components in sibling directories, so the | ||
| 17 | +# compiler's own default sysroot (rustc/) holds no libstd — it lives under | ||
| 18 | +# rust-std-<triple>/. The codegen libraries are found by rpath relative to the | ||
| 19 | +# binary, so only the sysroot has to be named. | ||
| 20 | +# | ||
| 21 | +# Callers that set their own win: buck's rustc_cfg action passes an empty | ||
| 22 | +# --sysroot, and rustc refuses the option twice. | ||
| 23 | +for arg do | ||
| 24 | + case $arg in | ||
| 25 | + --sysroot|--sysroot=*) exec "$dist/rustc/bin/rustc" "$@" ;; | ||
| 26 | + esac | ||
| 27 | +done | ||
| 28 | + | ||
| 29 | +exec "$dist/rustc/bin/rustc" --sysroot "$dist/rust-std-$triple" "$@" | ||