nandi/frq · dotslash-to-buck

nandi/frqpublic⑂ Fork 0

⑂ 61622f79c17a3c16cda75ab261e6ed0d07606215

dotslash-to-buck · 89 lines · 3.9 KBBash    Blame HistoryRaw
Write the build in babashka, and pin the babashka   1
3exec "$(dirname "$0")/bb" "$0" "$@"
Build the APK with buck2, from pins rather than from a second checkout   5
Write the build in babashka, and pin the babashka   6;; Generate buck2 http_archive data from the DotSlash files beside this script.
8;; DotSlash is where a tool's version, URL and digest are written down. buck
9;; needs the same three facts to fetch that tool itself — it cannot use the
10;; shim, which resolves on the machine at run time and so is neither an action
11;; input nor anything a remote worker could be given. Rather than write them
12;; twice and let them drift, this reads the manifests and emits the table buck
13
15;;     just sync-dist        # after editing any scripts/*.dotslash
17;; The DotSlash format is JSON with a `//` shebang line on top: one entry per
18;; platform, each with a url, a size, a digest and the path of the binary
19;; inside the archive. Only the archive itself is wanted here; the path inside
20;; it is the shim's business, and the toolchain rules name what they need.
21(require '[babashka.fs :as fs]
22         '[cheshire.core :as json]
23         '[clojure.string :as str])
Build the APK with buck2, from pins rather than from a second checkout   24
Write the build in babashka, and pin the babashka   25(def here (fs/parent (fs/canonicalize *file*)))
26(def out-file (fs/path (fs/parent here) "toolchains" "dist" "generated.bzl"))
Build the APK with buck2, from pins rather than from a second checkout   27
Write the build in babashka, and pin the babashka   28;; The tools buck fetches for itself. buck2 is not among them: it is the thing
29;; doing the fetching. jolt is not either — it is fetched by DotSlash on the
30;; machine that runs the boot image compile, which may not be this one.
31
32  [;; DotSlash itself, so an action can fetch what it needs where it runs
33   ;; rather than being handed it from here. See android/BUCK.
34   "dotslash"
35   ;; jolt-native's release, so the APK's UI half and the Jolt side of the
36   ;; tree ABI are fetched rather than built out of a second checkout.
37   "libvidya-android"
38   "glimmer-vidya"
39   "android-glue"])
Build the APK with buck2, from pins rather than from a second checkout   40
Write the build in babashka, and pin the babashka   41(defn load-manifest [stem]
42  (let [candidate (->> [(fs/path here (str stem ".dotslash")) (fs/path here stem)]
43                       (filter fs/exists?)
44                       first)]
45    (when-not candidate
46      (binding [*out* *err*] (println (str "no DotSlash manifest for " stem)))
47      (System/exit 1))
48    (let [text (slurp (str candidate))]
49      (json/parse-string (subs text (str/index-of text "{"))))))
Build the APK with buck2, from pins rather than from a second checkout   50
Write the build in babashka, and pin the babashka   51(defn platform-entry [entry]
52  (let [;; The path of the binary inside the archive starts with the directory
53        ;; the tarball unpacks into, which is what to strip. A flat archive —
54        ;; one binary, no directory around it — has nothing to strip, and
55        ;; naming the binary would strip the whole payload.
56        [head rest] (str/split (get entry "path") #"/" 2)]
57    (array-map "url" (get-in entry ["providers" 0 "url"])
58               "sha256" (get entry "digest")
59               "strip_prefix" (if rest head "")
60               "type" (get entry "format"))))
Build the APK with buck2, from pins rather than from a second checkout   61
Write the build in babashka, and pin the babashka   62
63  (into (array-map)
64        (for [stem (sort tools)]
65          [stem (into (array-map)
66                      (for [[platform entry] (sort-by key (get (load-manifest stem) "platforms"))
67                            ;; http_archive takes sha256 and nothing else.
68                            :when (= "sha256" (get entry "hash"))]
69                        [platform (platform-entry entry)]))])))
Build the APK with buck2, from pins rather than from a second checkout   70
Write the build in babashka, and pin the babashka   71;; Four-space indentation, like the Starlark around it: this file is read by
72;; people as often as by buck.
73(def pretty
74  (json/create-pretty-printer
75   (assoc json/default-pretty-print-options
76          :indentation "    "
77          :object-field-value-separator ": ")))
Build the APK with buck2, from pins rather than from a second checkout   78
Write the build in babashka, and pin the babashka   79(fs/create-dirs (fs/parent out-file))
80(spit (str out-file)
81      (str/join "\n"
82                ["# @generated by scripts/dotslash-to-buck — do not edit."
84                 "# The same archives scripts/*.dotslash pins, as facts buck can act on."
85                 "# Bump the DotSlash file and run `just sync-dist`."
87                 (str "DIST = " (json/generate-string dist {:pretty pretty}))
88                 ""]))
89(println (str "wrote " (fs/relativize (fs/parent here) out-file)))

Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	1	#!/bin/sh
	2	#_(
	3	exec "$(dirname "$0")/bb" "$0" "$@"
	4	)
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 19d ago	5
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	6	;; Generate buck2 http_archive data from the DotSlash files beside this script.
	7	;;
	8	;; DotSlash is where a tool's version, URL and digest are written down. buck
	9	;; needs the same three facts to fetch that tool itself — it cannot use the
	10	;; shim, which resolves on the machine at run time and so is neither an action
	11	;; input nor anything a remote worker could be given. Rather than write them
	12	;; twice and let them drift, this reads the manifests and emits the table buck
	13	;; reads.
	14	;;
	15	;; just sync-dist # after editing any scripts/*.dotslash
	16	;;
	17	;; The DotSlash format is JSON with a `//` shebang line on top: one entry per
	18	;; platform, each with a url, a size, a digest and the path of the binary
	19	;; inside the archive. Only the archive itself is wanted here; the path inside
	20	;; it is the shim's business, and the toolchain rules name what they need.
	21	(require '[babashka.fs :as fs]
	22	'[cheshire.core :as json]
	23	'[clojure.string :as str])
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 19d ago	24
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	25	(def here (fs/parent (fs/canonicalize file)))
	26	(def out-file (fs/path (fs/parent here) "toolchains" "dist" "generated.bzl"))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 19d ago	27
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	28	;; The tools buck fetches for itself. buck2 is not among them: it is the thing
	29	;; doing the fetching. jolt is not either — it is fetched by DotSlash on the
	30	;; machine that runs the boot image compile, which may not be this one.
	31	(def tools
	32	[;; DotSlash itself, so an action can fetch what it needs where it runs
	33	;; rather than being handed it from here. See android/BUCK.
	34	"dotslash"
	35	;; jolt-native's release, so the APK's UI half and the Jolt side of the
	36	;; tree ABI are fetched rather than built out of a second checkout.
	37	"libvidya-android"
	38	"glimmer-vidya"
	39	"android-glue"])
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 19d ago	40
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	41	(defn load-manifest [stem]
	42	(let [candidate (->> [(fs/path here (str stem ".dotslash")) (fs/path here stem)]
	43	(filter fs/exists?)
	44	first)]
	45	(when-not candidate
	46	(binding [out err] (println (str "no DotSlash manifest for " stem)))
	47	(System/exit 1))
	48	(let [text (slurp (str candidate))]
	49	(json/parse-string (subs text (str/index-of text "{"))))))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 19d ago	50
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	51	(defn platform-entry [entry]
	52	(let [;; The path of the binary inside the archive starts with the directory
	53	;; the tarball unpacks into, which is what to strip. A flat archive —
	54	;; one binary, no directory around it — has nothing to strip, and
	55	;; naming the binary would strip the whole payload.
	56	[head rest] (str/split (get entry "path") #"/" 2)]
	57	(array-map "url" (get-in entry ["providers" 0 "url"])
	58	"sha256" (get entry "digest")
	59	"strip_prefix" (if rest head "")
	60	"type" (get entry "format"))))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 19d ago	61
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	62	(def dist
	63	(into (array-map)
	64	(for [stem (sort tools)]
	65	[stem (into (array-map)
	66	(for [[platform entry] (sort-by key (get (load-manifest stem) "platforms"))
	67	;; http_archive takes sha256 and nothing else.
	68	:when (= "sha256" (get entry "hash"))]
	69	[platform (platform-entry entry)]))])))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 19d ago	70
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	71	;; Four-space indentation, like the Starlark around it: this file is read by
	72	;; people as often as by buck.
	73	(def pretty
	74	(json/create-pretty-printer
	75	(assoc json/default-pretty-print-options
	76	:indentation " "
	77	:object-field-value-separator ": ")))
Build the APK with buck2, from pins rather than from a second checkout a71ef8f nandi 19d ago	78
Write the build in babashka, and pin the babashka 34832a8 nandi 18d ago	79	(fs/create-dirs (fs/parent out-file))
	80	(spit (str out-file)
	81	(str/join "\n"
	82	["# @generated by scripts/dotslash-to-buck — do not edit."
	83	"#"
	84	"# The same archives scripts/*.dotslash pins, as facts buck can act on."
	85	"# Bump the DotSlash file and run `just sync-dist`."
	86	""
	87	(str "DIST = " (json/generate-string dist {:pretty pretty}))
	88	""]))
	89	(println (str "wrote " (fs/relativize (fs/parent here) out-file)))