nandi/nimstatic · README.md

⑂ 6da556b6028125e359cbbe5aa3e62bda746e6cce

README.md · 173 lines · 6.3 KBmarkdown    Blame HistoryRaw
Rename to nimstatic, and detect dependencies from the source   1# nimstatic
muslkit: musl static libraries from Alpine, without Alpine   2
Rename to nimstatic, and detect dependencies from the source   3Fully static Nim binaries, dependencies and all.
muslkit: musl static libraries from Alpine, without Alpine   4
Rename to nimstatic, and detect dependencies from the source   6nimstatic app.nim
muslkit: musl static libraries from Alpine, without Alpine   7
Rename to nimstatic, and detect dependencies from the source   9That's the whole thing. nimstatic asks the Nim compiler what `app.nim` actually
10needs, fetches musl-built static libraries for it from Alpine's mirrors, and
11compiles against a sysroot it owns. No apk, no container, no root, no Nix.
muslkit: musl static libraries from Alpine, without Alpine   12
Rename to nimstatic, and detect dependencies from the source   14$ nimstatic freeqsay.nim -- -d:ssl
15probing freeqsay.nim …
16  ssl           dlopen  openssl-libs-static
17  crypto        dlopen  openssl-libs-static
18downloading openssl-libs-static 3.3.7-r1 (12734 KiB)
19
20wrote freeqsay (20687 KiB, static)
muslkit: musl static libraries from Alpine, without Alpine   21
Rename to nimstatic, and detect dependencies from the source   23The result runs anywhere with a Linux kernel — `ldd` says *not a dynamic
24executable* — and its HTTPS still works.
muslkit: musl static libraries from Alpine, without Alpine   25
Rename to nimstatic, and detect dependencies from the source   26## How it knows what you need
muslkit: musl static libraries from Alpine, without Alpine   27
Rename to nimstatic, and detect dependencies from the source   28Guessing from `import` lines would be wrong in both directions: a transitive
29import three modules deep still needs its library, and an import behind a
30`when` that never fires does not. So nimstatic asks the compiler instead. A
31`--compileOnly --genScript` probe produces the nimcache your real build would
32have, and that cache answers twice over:
muslkit: musl static libraries from Alpine, without Alpine   33
Rename to nimstatic, and detect dependencies from the source   34- **`<project>.json`** carries the link command, so every `-lfoo` is explicit.
35- **The generated C** contains the dynlib candidate strings Nim will `dlopen`
36  at runtime — `"libssl.so(.3|.1.1|…)"`. These never appear on a link line, and
37  they are exactly what breaks a static binary.
39Libraries musl already provides (`m`, `rt`, `dl`, `pthread`, …) are skipped.
40Anything else is looked up in a table of Alpine packages; whatever isn't mapped
41is reported rather than silently dropped:
44$ nimstatic detect app.nim
45library         how      alpine package
46sqlite3         link     sqlite-static
47ssl             dlopen   openssl-libs-static
48mystery         dlopen   (unmapped)
50unmapped: mystery
51search for one with: nimstatic search mystery
muslkit: musl static libraries from Alpine, without Alpine   52
Rename to nimstatic, and detect dependencies from the source   54Then `--map mystery=mystery-static` teaches it, for that build.
56## The two things that silently break static Nim
58Both are handled automatically; they're documented here because they cost
59everyone an afternoon at least once.
muslkit: musl static libraries from Alpine, without Alpine   60
61- **`-d:ssl` makes Nim `dlopen` libssl at runtime.** A static binary cannot,
62  and dies at startup with `could not load: libcrypto.so(...)` — even on code
Rename to nimstatic, and detect dependencies from the source   63  paths that never touch the network. Fix: `--dynlibOverride:ssl
64  --dynlibOverride:crypto` plus the archives on the link line. nimstatic emits
65  an override only for libraries it actually has an archive for, since an
66  override without one turns a runtime failure into a link failure.
muslkit: musl static libraries from Alpine, without Alpine   67- **OpenSSL 3 removed `SSL_get_peer_certificate`,** which Nim's wrapper still
Rename to nimstatic, and detect dependencies from the source   68  names, so the link dies on one undefined symbol. Fix:
muslkit: musl static libraries from Alpine, without Alpine   69  `-DSSL_get_peer_certificate=SSL_get1_peer_certificate`.
Rename to nimstatic, and detect dependencies from the source   71A static binary also carries no CA trust store, so set `SSL_CERT_FILE` on the
72host that runs it.
muslkit: musl static libraries from Alpine, without Alpine   73
Rename to nimstatic, and detect dependencies from the source   74
muslkit: musl static libraries from Alpine, without Alpine   75
Rename to nimstatic, and detect dependencies from the source   76
77nimstatic <file.nim> [-- <nim args>]   Detect, fetch, build static
78nimstatic detect <file.nim>            Show what it needs, change nothing
79nimstatic add <pkg>...                 Put packages in the sysroot by hand
80nimstatic list                         Show what the sysroot holds
81nimstatic libs                         Show the sysroot's static libraries
82nimstatic search <text>                Search Alpine's index
83nimstatic show <pkg>                   Index record for one package
84nimstatic nimflags [-l lib]            Print nim flags for the sysroot
85nimstatic ccflags [-l lib]             Print cc/clang flags
86nimstatic nimcfg [-o file]             Write a nim.cfg fragment
87nimstatic zigcc [-o file]              Write a `zig cc -target …-musl` wrapper
88nimstatic env                          Shell exports (PKG_CONFIG_*, NIMSTATIC_ROOT)
89nimstatic path                         Print the sysroot path
90nimstatic clean                        Remove the sysroot (cache is kept)
muslkit: musl static libraries from Alpine, without Alpine   92
Rename to nimstatic, and detect dependencies from the source   93Everything after `--` goes to the Nim compiler for **both** the probe and the
94build, so conditional imports resolve the same way twice:
muslkit: musl static libraries from Alpine, without Alpine   95
Rename to nimstatic, and detect dependencies from the source   96
97nimstatic app.nim -- -d:ssl -d:danger
100Build options: `-o/--output`, `-d/--debug` (skip `-d:release`), `-n/--dry-run`
101(print the command instead of running it), `--map lib=pkg`, `--pkg name`,
102`--cc path`, `--nim path`.
104Sysroot options: `-r/--root`, `-b/--branch` (default `v3.21`, `edge` for
105rolling), `-a/--arch`, `-m/--mirror`, `--repo main,community`, `-l/--lib`,
106`--no-deps`, `--refresh`, `-q/--quiet`.
108The sysroot defaults to `$XDG_DATA_HOME/nimstatic/sysroot` and honors
109`NIMSTATIC_ROOT`. Downloads cache under `$XDG_CACHE_HOME/nimstatic`, so the
110second build is offline and the index is re-fetched once a day.
112## Cross-compiling
114Same command, one flag — the sysroot, the packages and the zig target all
115follow `--arch`:
muslkit: musl static libraries from Alpine, without Alpine   116
Rename to nimstatic, and detect dependencies from the source   118nimstatic app.nim --arch aarch64 -o app-arm64
muslkit: musl static libraries from Alpine, without Alpine   119
Rename to nimstatic, and detect dependencies from the source   121## Requirements
123- **zig** on PATH — used as `zig cc -target x86_64-linux-musl`. Pass `--cc` to
124  use a musl cross-compiler you already have instead.
125- **tar** — an `.apk` is concatenated gzip streams, which GNU tar reads.
muslkit: musl static libraries from Alpine, without Alpine   126
Rename to nimstatic, and detect dependencies from the source   127## Using it as a library
muslkit: musl static libraries from Alpine, without Alpine   128
Rename to nimstatic, and detect dependencies from the source   130import nimstatic
132let d = detect("app.nim", ["-d:ssl"])
133echo packages(d)          # @["openssl-libs-static"]
134echo dynlibOverrides(d)   # @["ssl", "crypto"]
muslkit: musl static libraries from Alpine, without Alpine   135
136let remote = initRemote(branch = "edge")
Rename to nimstatic, and detect dependencies from the source   137for pkg in remote.fetchIndex().resolve(packages(d)):
muslkit: musl static libraries from Alpine, without Alpine   138  echo pkg.name, " ", pkg.version, " ", remote.fetchPackage(pkg)
Rename to nimstatic, and detect dependencies from the source   141Modules: `nimstatic/detect` (probe, parse, map), `nimstatic/build` (the
142one-command path), `nimstatic/index` (APKINDEX parsing, provides and dependency
143resolution), `nimstatic/repo` (mirror, cache, download, unpack),
144`nimstatic/sysroot` (manifest, static-lib discovery), `nimstatic/flags` (flag
145
147
149Packages come over HTTPS from the mirror and are unpacked as-is. nimstatic
150checks the size recorded in the index but does **not** verify Alpine's RSA
151signatures — apk's checksum field covers a package's control segment rather
152than the file, so a real check means implementing apk's signature format. Treat
153a sysroot as build input, not as a trust root. If that matters, pin a mirror
154
156
159nimble install
162or build in place with `nim c -d:ssl -o:nimstatic src/nimstatic.nim`.
muslkit: musl static libraries from Alpine, without Alpine   163
164
167nimble test
Rename to nimstatic, and detect dependencies from the source   17025 tests, all offline: soname parsing, link-command parsing, dynlib discovery
171in generated C, package mapping, APKINDEX parsing, dependency resolution
172through `so:`/`pkgconfig` provides, manifest round-trips and flag emission
173(including link order and the dropped-override case).

Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	1	# nimstatic
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	2
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	3	Fully static Nim binaries, dependencies and all.
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	4
	5	```bash
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	6	nimstatic app.nim
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	7	```
	8
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	9	That's the whole thing. nimstatic asks the Nim compiler what `app.nim` actually
	10	needs, fetches musl-built static libraries for it from Alpine's mirrors, and
	11	compiles against a sysroot it owns. No apk, no container, no root, no Nix.
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	12
	13	```
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	14	$ nimstatic freeqsay.nim -- -d:ssl
	15	probing freeqsay.nim …
	16	ssl dlopen openssl-libs-static
	17	crypto dlopen openssl-libs-static
	18	downloading openssl-libs-static 3.3.7-r1 (12734 KiB)
	19	building …
	20	wrote freeqsay (20687 KiB, static)
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	21	```
	22
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	23	The result runs anywhere with a Linux kernel — `ldd` says *not a dynamic
	24	executable* — and its HTTPS still works.
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	25
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	26	## How it knows what you need
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	27
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	28	Guessing from `import` lines would be wrong in both directions: a transitive
	29	import three modules deep still needs its library, and an import behind a
	30	`when` that never fires does not. So nimstatic asks the compiler instead. A
	31	`--compileOnly --genScript` probe produces the nimcache your real build would
	32	have, and that cache answers twice over:
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	33
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	34	- `<project>.json` carries the link command, so every `-lfoo` is explicit.
	35	- The generated C contains the dynlib candidate strings Nim will `dlopen`
	36	at runtime — `"libssl.so(.3\|.1.1\|…)"`. These never appear on a link line, and
	37	they are exactly what breaks a static binary.
	38
	39	Libraries musl already provides (`m`, `rt`, `dl`, `pthread`, …) are skipped.
	40	Anything else is looked up in a table of Alpine packages; whatever isn't mapped
	41	is reported rather than silently dropped:
	42
	43	```
	44	$ nimstatic detect app.nim
	45	library how alpine package
	46	sqlite3 link sqlite-static
	47	ssl dlopen openssl-libs-static
	48	mystery dlopen (unmapped)
	49
	50	unmapped: mystery
	51	search for one with: nimstatic search mystery
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	52	```
	53
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	54	Then `--map mystery=mystery-static` teaches it, for that build.
	55
	56	## The two things that silently break static Nim
	57
	58	Both are handled automatically; they're documented here because they cost
	59	everyone an afternoon at least once.
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	60
	61	- `-d:ssl` makes Nim `dlopen` libssl at runtime. A static binary cannot,
	62	and dies at startup with `could not load: libcrypto.so(...)` — even on code
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	63	paths that never touch the network. Fix: `--dynlibOverride:ssl
	64	--dynlibOverride:crypto` plus the archives on the link line. nimstatic emits
	65	an override only for libraries it actually has an archive for, since an
	66	override without one turns a runtime failure into a link failure.
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	67	- OpenSSL 3 removed `SSL_get_peer_certificate`, which Nim's wrapper still
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	68	names, so the link dies on one undefined symbol. Fix:
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	69	`-DSSL_get_peer_certificate=SSL_get1_peer_certificate`.
	70
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	71	A static binary also carries no CA trust store, so set `SSL_CERT_FILE` on the
	72	host that runs it.
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	73
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	74	## Usage
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	75
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	76	```
	77	nimstatic <file.nim> [-- <nim args>] Detect, fetch, build static
	78	nimstatic detect <file.nim> Show what it needs, change nothing
	79	nimstatic add <pkg>... Put packages in the sysroot by hand
	80	nimstatic list Show what the sysroot holds
	81	nimstatic libs Show the sysroot's static libraries
	82	nimstatic search <text> Search Alpine's index
	83	nimstatic show <pkg> Index record for one package
	84	nimstatic nimflags [-l lib] Print nim flags for the sysroot
	85	nimstatic ccflags [-l lib] Print cc/clang flags
	86	nimstatic nimcfg [-o file] Write a nim.cfg fragment
	87	nimstatic zigcc [-o file] Write a `zig cc -target …-musl` wrapper
	88	nimstatic env Shell exports (PKG_CONFIG_*, NIMSTATIC_ROOT)
	89	nimstatic path Print the sysroot path
	90	nimstatic clean Remove the sysroot (cache is kept)
	91	```
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	92
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	93	Everything after `--` goes to the Nim compiler for both the probe and the
	94	build, so conditional imports resolve the same way twice:
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	95
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	96	```bash
	97	nimstatic app.nim -- -d:ssl -d:danger
	98	```
	99
	100	Build options: `-o/--output`, `-d/--debug` (skip `-d:release`), `-n/--dry-run`
	101	(print the command instead of running it), `--map lib=pkg`, `--pkg name`,
	102	`--cc path`, `--nim path`.
	103
	104	Sysroot options: `-r/--root`, `-b/--branch` (default `v3.21`, `edge` for
	105	rolling), `-a/--arch`, `-m/--mirror`, `--repo main,community`, `-l/--lib`,
	106	`--no-deps`, `--refresh`, `-q/--quiet`.
	107
	108	The sysroot defaults to `$XDG_DATA_HOME/nimstatic/sysroot` and honors
	109	`NIMSTATIC_ROOT`. Downloads cache under `$XDG_CACHE_HOME/nimstatic`, so the
	110	second build is offline and the index is re-fetched once a day.
	111
	112	## Cross-compiling
	113
	114	Same command, one flag — the sysroot, the packages and the zig target all
	115	follow `--arch`:
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	116
	117	```bash
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	118	nimstatic app.nim --arch aarch64 -o app-arm64
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	119	```
	120
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	121	## Requirements
	122
	123	- zig on PATH — used as `zig cc -target x86_64-linux-musl`. Pass `--cc` to
	124	use a musl cross-compiler you already have instead.
	125	- tar — an `.apk` is concatenated gzip streams, which GNU tar reads.
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	126
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	127	## Using it as a library
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	128
	129	```nim
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	130	import nimstatic
	131
	132	let d = detect("app.nim", ["-d:ssl"])
	133	echo packages(d) # @["openssl-libs-static"]
	134	echo dynlibOverrides(d) # @["ssl", "crypto"]
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	135
	136	let remote = initRemote(branch = "edge")
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	137	for pkg in remote.fetchIndex().resolve(packages(d)):
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	138	echo pkg.name, " ", pkg.version, " ", remote.fetchPackage(pkg)
	139	```
	140
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	141	Modules: `nimstatic/detect` (probe, parse, map), `nimstatic/build` (the
	142	one-command path), `nimstatic/index` (APKINDEX parsing, provides and dependency
	143	resolution), `nimstatic/repo` (mirror, cache, download, unpack),
	144	`nimstatic/sysroot` (manifest, static-lib discovery), `nimstatic/flags` (flag
	145	emission).
	146
	147	## Trust
	148
	149	Packages come over HTTPS from the mirror and are unpacked as-is. nimstatic
	150	checks the size recorded in the index but does not verify Alpine's RSA
	151	signatures — apk's checksum field covers a package's control segment rather
	152	than the file, so a real check means implementing apk's signature format. Treat
	153	a sysroot as build input, not as a trust root. If that matters, pin a mirror
	154	you run.
	155
	156	## Install
	157
	158	```bash
	159	nimble install
	160	```
	161
	162	or build in place with `nim c -d:ssl -o:nimstatic src/nimstatic.nim`.
muslkit: musl static libraries from Alpine, without Alpine 5feb050 nandi 20h ago	163
	164	## Tests
	165
	166	```bash
	167	nimble test
	168	```
	169
Rename to nimstatic, and detect dependencies from the source 53cd879 nandi 19h ago	170	25 tests, all offline: soname parsing, link-command parsing, dynlib discovery
	171	in generated C, package mapping, APKINDEX parsing, dependency resolution
	172	through `so:`/`pkgconfig` provides, manifest round-trips and flag emission
	173	(including link order and the dropped-override case).