nandi/frq · dotslash-to-buck

nandi/frqpublic⑂ Fork 0

⑂ bffe879efd6e16d866307e451e9a3568976fcdb2

dotslash-to-buck · 92 lines · 4.0 KBBash    Blame HistoryRaw
Write the build in babashka, and pin the babashka   1
3exec "$(dirname "$0")/bb" "$0" "$@"
Build the APK with buck2, from pins rather than from a second checkout   5
Write the build in babashka, and pin the babashka   6;; Generate buck2 http_archive data from the DotSlash files beside this script.
8;; DotSlash is where a tool's version, URL and digest are written down. buck
9;; needs the same three facts to fetch that tool itself — it cannot use the
10;; shim, which resolves on the machine at run time and so is neither an action
11;; input nor anything a remote worker could be given. Rather than write them
12;; twice and let them drift, this reads the manifests and emits the table buck
13
15;;     just sync-dist        # after editing any scripts/*.dotslash
17;; The DotSlash format is JSON with a `//` shebang line on top: one entry per
18;; platform, each with a url, a size, a digest and the path of the binary
19;; inside the archive. Only the archive itself is wanted here; the path inside
20;; it is the shim's business, and the toolchain rules name what they need.
21(require '[babashka.fs :as fs]
22         '[cheshire.core :as json]
23         '[clojure.string :as str])
Build the APK with buck2, from pins rather than from a second checkout   24
Write the build in babashka, and pin the babashka   25(def here (fs/parent (fs/canonicalize *file*)))
26(def out-file (fs/path (fs/parent here) "toolchains" "dist" "generated.bzl"))
Build the APK with buck2, from pins rather than from a second checkout   27
Write the build in babashka, and pin the babashka   28;; The tools buck fetches for itself. buck2 is not among them: it is the thing
29;; doing the fetching. jolt is not either — it is fetched by DotSlash on the
30;; machine that runs the boot image compile, which may not be this one.
31
32  [;; DotSlash itself, so an action can fetch what it needs where it runs
33   ;; rather than being handed it from here. See android/BUCK.
34   "dotslash"
35   ;; jolt-native's release, so the APK's UI half and the Jolt side of the
36   ;; tree ABI are fetched rather than built out of a second checkout.
37   "libvidya-android"
Pin the media plane, now that there is a release to pin   38   ;; The media plane, out of the same tarball as libvidya — see its manifest
39   ;; for why the two are pinned together rather than independently.
40   "libjoltmoq-android"
Write the build in babashka, and pin the babashka   41   "glimmer-vidya"
42   "android-glue"])
Build the APK with buck2, from pins rather than from a second checkout   43
Write the build in babashka, and pin the babashka   44(defn load-manifest [stem]
45  (let [candidate (->> [(fs/path here (str stem ".dotslash")) (fs/path here stem)]
46                       (filter fs/exists?)
47                       first)]
48    (when-not candidate
49      (binding [*out* *err*] (println (str "no DotSlash manifest for " stem)))
50      (System/exit 1))
51    (let [text (slurp (str candidate))]
52      (json/parse-string (subs text (str/index-of text "{"))))))
Build the APK with buck2, from pins rather than from a second checkout   53
Write the build in babashka, and pin the babashka   54(defn platform-entry [entry]
55  (let [;; The path of the binary inside the archive starts with the directory
56        ;; the tarball unpacks into, which is what to strip. A flat archive —
57        ;; one binary, no directory around it — has nothing to strip, and
58        ;; naming the binary would strip the whole payload.
59        [head rest] (str/split (get entry "path") #"/" 2)]
60    (array-map "url" (get-in entry ["providers" 0 "url"])
61               "sha256" (get entry "digest")
62               "strip_prefix" (if rest head "")
63               "type" (get entry "format"))))
Build the APK with buck2, from pins rather than from a second checkout   64
Write the build in babashka, and pin the babashka   65
66  (into (array-map)
67        (for [stem (sort tools)]
68          [stem (into (array-map)
69                      (for [[platform entry] (sort-by key (get (load-manifest stem) "platforms"))
70                            ;; http_archive takes sha256 and nothing else.
71                            :when (= "sha256" (get entry "hash"))]
72                        [platform (platform-entry entry)]))])))
Build the APK with buck2, from pins rather than from a second checkout   73
Write the build in babashka, and pin the babashka   74;; Four-space indentation, like the Starlark around it: this file is read by
75;; people as often as by buck.
76(def pretty
77  (json/create-pretty-printer
78   (assoc json/default-pretty-print-options
79          :indentation "    "
80          :object-field-value-separator ": ")))
Build the APK with buck2, from pins rather than from a second checkout   81
Write the build in babashka, and pin the babashka   82(fs/create-dirs (fs/parent out-file))
83(spit (str out-file)
84      (str/join "\n"
85                ["# @generated by scripts/dotslash-to-buck — do not edit."
87                 "# The same archives scripts/*.dotslash pins, as facts buck can act on."
88                 "# Bump the DotSlash file and run `just sync-dist`."
90                 (str "DIST = " (json/generate-string dist {:pretty pretty}))
91                 ""]))
92(println (str "wrote " (fs/relativize (fs/parent here) out-file)))

Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	1	#!/bin/sh
	2	#_(
	3	exec "$(dirname "$0")/bb" "$0" "$@"
	4	)
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 18d ago	5
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	6	;; Generate buck2 http_archive data from the DotSlash files beside this script.
	7	;;
	8	;; DotSlash is where a tool's version, URL and digest are written down. buck
	9	;; needs the same three facts to fetch that tool itself — it cannot use the
	10	;; shim, which resolves on the machine at run time and so is neither an action
	11	;; input nor anything a remote worker could be given. Rather than write them
	12	;; twice and let them drift, this reads the manifests and emits the table buck
	13	;; reads.
	14	;;
	15	;; just sync-dist # after editing any scripts/*.dotslash
	16	;;
	17	;; The DotSlash format is JSON with a `//` shebang line on top: one entry per
	18	;; platform, each with a url, a size, a digest and the path of the binary
	19	;; inside the archive. Only the archive itself is wanted here; the path inside
	20	;; it is the shim's business, and the toolchain rules name what they need.
	21	(require '[babashka.fs :as fs]
	22	'[cheshire.core :as json]
	23	'[clojure.string :as str])
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 18d ago	24
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	25	(def here (fs/parent (fs/canonicalize file)))
	26	(def out-file (fs/path (fs/parent here) "toolchains" "dist" "generated.bzl"))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 18d ago	27
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	28	;; The tools buck fetches for itself. buck2 is not among them: it is the thing
	29	;; doing the fetching. jolt is not either — it is fetched by DotSlash on the
	30	;; machine that runs the boot image compile, which may not be this one.
	31	(def tools
	32	[;; DotSlash itself, so an action can fetch what it needs where it runs
	33	;; rather than being handed it from here. See android/BUCK.
	34	"dotslash"
	35	;; jolt-native's release, so the APK's UI half and the Jolt side of the
	36	;; tree ABI are fetched rather than built out of a second checkout.
	37	"libvidya-android"
Pin the media plane, now that there is a release to pin 03d5a6b nandi 18d ago	38	;; The media plane, out of the same tarball as libvidya — see its manifest
	39	;; for why the two are pinned together rather than independently.
	40	"libjoltmoq-android"
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	41	"glimmer-vidya"
	42	"android-glue"])
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 18d ago	43
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	44	(defn load-manifest [stem]
	45	(let [candidate (->> [(fs/path here (str stem ".dotslash")) (fs/path here stem)]
	46	(filter fs/exists?)
	47	first)]
	48	(when-not candidate
	49	(binding [out err] (println (str "no DotSlash manifest for " stem)))
	50	(System/exit 1))
	51	(let [text (slurp (str candidate))]
	52	(json/parse-string (subs text (str/index-of text "{"))))))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 18d ago	53
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	54	(defn platform-entry [entry]
	55	(let [;; The path of the binary inside the archive starts with the directory
	56	;; the tarball unpacks into, which is what to strip. A flat archive —
	57	;; one binary, no directory around it — has nothing to strip, and
	58	;; naming the binary would strip the whole payload.
	59	[head rest] (str/split (get entry "path") #"/" 2)]
	60	(array-map "url" (get-in entry ["providers" 0 "url"])
	61	"sha256" (get entry "digest")
	62	"strip_prefix" (if rest head "")
	63	"type" (get entry "format"))))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 18d ago	64
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	65	(def dist
	66	(into (array-map)
	67	(for [stem (sort tools)]
	68	[stem (into (array-map)
	69	(for [[platform entry] (sort-by key (get (load-manifest stem) "platforms"))
	70	;; http_archive takes sha256 and nothing else.
	71	:when (= "sha256" (get entry "hash"))]
	72	[platform (platform-entry entry)]))])))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 18d ago	73
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	74	;; Four-space indentation, like the Starlark around it: this file is read by
	75	;; people as often as by buck.
	76	(def pretty
	77	(json/create-pretty-printer
	78	(assoc json/default-pretty-print-options
	79	:indentation " "
	80	:object-field-value-separator ": ")))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 18d ago	81
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	82	(fs/create-dirs (fs/parent out-file))
	83	(spit (str out-file)
	84	(str/join "\n"
	85	["# @generated by scripts/dotslash-to-buck — do not edit."
	86	"#"
	87	"# The same archives scripts/*.dotslash pins, as facts buck can act on."
	88	"# Bump the DotSlash file and run `just sync-dist`."
	89	""
	90	(str "DIST = " (json/generate-string dist {:pretty pretty}))
	91	""]))
	92	(println (str "wrote " (fs/relativize (fs/parent here) out-file)))