nandi/jolt-native · README.md

⑂ 4f920afa410da72bfdb7a07d7faa0264c7bb8a08

README.md · 179 lines · 8.6 KBmarkdown    Blame HistoryRaw
Lift freeq's AV media plane out of sleek   1# jolt-native
3Native capabilities for [jolt](https://github.com/jolt-lang/jolt), one shared
4object per capability.
6A jolt library binds a `.so` through `jolt.ffi/defcfn`: it declares typed
7foreign functions and gets back integers, doubles and borrowed UTF-8 strings.
8That is the whole vocabulary. These crates are the other side of that
9boundary — the things worth writing in Rust because writing them again in
10anything else would mean writing them badly.
Move the jolt libraries under glimmer-backends   13crates/jolt-abi                 the rules every library here keeps at its edge
14crates/jolt-vidya               the retained-tree C ABI  → libvidya.so
15crates/vidya-core               the egui semantic layer behind it (theme, widgets, fonts)
16crates/jolt-tui                 the same tree ABI, painted into a terminal → libjolttui.so
17crates/jolt-moq                 freeq's AV media plane   → libjoltmoq.so
18glimmer-backends/glimmer-vidya  the jolt side of libvidya: glimmer's backend
glimmer-cosmic: a libcosmic backend for glimmer (spike)   19crates/jolt-cosmic              the same tree ABI, painted by libcosmic → libjoltcosmic.so
20glimmer-backends/glimmer-cosmic glimmer's backend over libjoltcosmic (a spike)
Write dvui's shape in jolt, on SDL3, with no shared object  Veronika Winters 21jvui                            no .so at all: dvui's shape, in jolt, on SDL3
22glimmer-backends/glimmer-jvui   glimmer's backend over jvui
wayland-bar-compositor: a wlroots compositor in jolt, for panels on X11   23wayland-bar-compositor          no .so either: a wlroots compositor in jolt, for panels on X11
Lift freeq's AV media plane out of sleek   24
Paint the same tree into a terminal, for the machines with no window   26All three objects land in one `target/release`, so a consumer points
Bring vidya in   27`LD_LIBRARY_PATH` at one directory and names in `:jolt/native` only the ones it
28actually wants.
Lift freeq's AV media plane out of sleek   30## Why a workspace and not one library with features
32Because cargo features are additive *within* a library. A single
33`libjoltnative.so` whose `graphics` and `moq` features selected the parts would
34link egui **and** QUIC, Opus, H.264, cpal and V4L2 into one object, so a chat
35client that is not in a call would carry the whole media plane in order to open
36a window. The feature unification that comes with it has bitten this code
37before: enabling eframe's `wayland`/`x11` features for a desktop build pulled
38them into the Android one and made the activity exit on launch.
40A workspace keeps the parts of a monorepo worth having — one CI, one release,
41shared conventions in `jolt-abi` — while each consumer links only what it asked
42for. jolt's `:jolt/native` is a vector; naming two objects there is no harder
43than naming one.
45## Building
Point the docs at the flake   47The flake is the toolchain. `nix develop` is where the compiler, the C
48toolchain, PipeWire's headers and the kernel headers bindgen wants come from,
49so **nothing needs to be installed** beyond nix itself — no `cc` on PATH, no
50rustup, no NDK.
Lift freeq's AV media plane out of sleek   52
Point the docs at the flake   53nix develop                 # then:
54just build                  # release .so files in target/release
55just test                   # the whole workspace
56just libdir                 # where to point LD_LIBRARY_PATH
Lift freeq's AV media plane out of sleek   57
Point the docs at the flake   59`just build` is the edit loop; the nix outputs are the artifact, built against
60a pinned compiler and a pinned crate graph:
Lift freeq's AV media plane out of sleek   61
Point the docs at the flake   62
63nix build                   # all three objects  → result/lib
64nix build .#android         # both device objects → result/lib/arm64-v8a
65nix flake check             # fmt, clippy, tests, all three objects
Lift freeq's AV media plane out of sleek   67
Point the docs at the flake   68Two things are worth knowing about:
70* **bindgen runs libclang directly**, so `v4l2r` needs to be pointed at
71  `linux/videodev2.h`. The flake sets `V4L2R_VIDEODEV2_H_PATH` and
72  `BINDGEN_EXTRA_CLANG_ARGS` from nixpkgs' `linuxHeaders` in both the devShell
73  and the build, which is why building inside `nix develop` rather than a bare
74  shell is the supported path.
Lift freeq's AV media plane out of sleek   75* **PipeWire** is a real system dependency — `libspa-sys` wants `pkg-config`
Point the docs at the flake   76  and PipeWire's headers, and the flake supplies both. It is behind the
77  default-on `pipewire` feature of `jolt-moq`; `just test-minimal` builds
78  without it, for a machine outside the flake. Turning it off costs device
79  *names*: only ALSA PCMs remain visible ("pipewire", "sysdefault") rather than
80  the real sources a browser would offer, which makes picking a microphone
81  guesswork.
83### Android
86just ffi-android            # nix build .#android
89Same sources and same `Cargo.lock` as the desktop objects; only the target
90configuration moves. The NDK comes from nixpkgs, so this needs nothing
91installed either. The camera wants two things from the APK that the build
92cannot supply: a `CameraCapture` class in its `classes.dex`, and a call to
93`joltmoq_android_init` — see `android/jolt_main.c`.
Lift freeq's AV media plane out of sleek   94
95## jolt-moq
97freeq's audio and video calls, extracted from
98[sleek](https://github.com/codegod100/sleek), which reached them by calling
99Rust from Rust.
101A freeq call has two halves, and only one of them is here.
103**Signaling stayed behind.** A call is opened, joined and left over IRC
104TAGMSGs — `+freeq.at/av-start` and its siblings — and the server broadcasts
105`+freeq.at/av-state` back. Any client that speaks IRC already has everything it
106needs for that, in whatever language it speaks IRC in; crossing an FFI boundary
107to send a TAGMSG would be worse than not crossing one. This library begins once
108the session id and the SFU token are known.
110**The media plane came across**, because it is not a thing to write twice: MoQ
111over QUIC, Opus, H.264, camera capture and the SFU's own dial rules come to
112some three thousand lines that would say the same thing again in another
113language and be wrong in different places.
115`crates/jolt-moq/include/joltmoq.h` is the surface, and says more about each
116call than this does. The shape of it:
118* **Nothing calls back.** Status is drained with `joltmoq_poll_status`, video
119  with `joltmoq_frame_poll`, both from whatever thread the caller paints on. A
120  callback into a foreign runtime from a tokio worker is a rule about threads
121  that the caller has to keep and that nothing can check.
122* **Frames are borrowed, not copied.** `joltmoq_frame_rgba` points into the
123  decoder's own buffer until the next poll. A frame is a megabyte or two;
124  copying it out so the caller can hand it straight to a texture upload would
125  be two copies a frame for nothing.
126* **One call at a time.** There is one microphone, so there is one session.
128### Its relationship to sleek
Let the media plane cross to the phone, camera and all   130`av.rs`, `av_media.rs`, `v4l2cam.rs`, `android_camera.rs` and `nv12_orient.rs`
131came from sleek's `android/src`. They are kept close to their originals so changes can still be moved between the
Lift freeq's AV media plane out of sleek   132two by eye. What was dropped on the way was the signaling-and-presentation
133half — `ChannelCall`, `LocalCall`, `apply_av_state`, `av_state_message` —
134which is sleek's own app state and belongs to a client, not to a media plane.
135Dropping it also dropped the `freeq-sdk` dependency entirely.
137Sleek still has its own copy. Pointing it at this crate instead is the obvious
138next step and has not been taken yet; until it is, a fix made in one place
139needs making in the other.
Let the media plane cross to the phone, camera and all   141`android_camera.rs` is the one that did not come across unchanged. Sleek is a
142single shared object, so its Camera2 bridge reads the `JavaVM` and the Activity
143straight out of the `AndroidApp` that android-activity handed it. Here the glue
144is in `libvidya.so` and the media plane is not, so the handles arrive through
145`joltmoq_android_init` instead — see `android_jni.rs` for why the usual bridge,
146`ndk_context`, cannot cross two cdylibs. Everything below that line is sleek's.
Bring vidya in   148
150The theme layer, the widget set and the retained-tree ABI that glimmer paints
151through. Imported from its own repo rather than depended on: egui is the only
152backend anyone wants now, so the indirection that let `libvidya` be swapped for
153a raylib or cimgui build of the same symbols was paying for a choice nobody was
154going to make.
156`crates/vidya-core` is the egui layer — and the 4MB under its `assets/` is not
157incidental: the Twemoji pack and the symbols font are `include_bytes!`d into
158the library, because a shared object has no host app to install a font set for
Move the jolt libraries under glimmer-backends   161`glimmer-backends/glimmer-vidya` is the other side of that ABI, and is jolt
162rather than Rust. A consumer takes it as a `:local/root` dep and `libvidya.so` on the
Bring vidya in   163loader path — frq's `deps.edn` is the worked example.
Lift freeq's AV media plane out of sleek   165## Adding a crate
167One crate, one `.so`, one capability. Depend on `jolt-abi` and keep its three
168rules — catch panics at every entry point, lend strings from a `Scratch` rather
169than handing out memory to free, and let the caller ask rather than calling it
170back. A consumer then names your object in `:jolt/native` alongside whichever
171others it wants, and pays for nothing else.
Say which licence, in the place people look for it   172
173
175MIT — the same as the projects this code came from: `crates/vidya-*` and
Move the jolt libraries under glimmer-backends   176`glimmer-backends/glimmer-vidya` from [vidya](https://tangled.org/nandi.uk/vidya),
Say which licence, in the place people look for it   177`crates/jolt-moq`'s media plane from
178[sleek](https://github.com/codegod100/sleek). `third-party/cpal` is a vendored
179pin of [cpal](https://github.com/RustAudio/cpal) and keeps its own licence.

Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	1	# jolt-native
	2
	3	Native capabilities for [jolt](https://github.com/jolt-lang/jolt), one shared
	4	object per capability.
	5
	6	A jolt library binds a `.so` through `jolt.ffi/defcfn`: it declares typed
	7	foreign functions and gets back integers, doubles and borrowed UTF-8 strings.
	8	That is the whole vocabulary. These crates are the other side of that
	9	boundary — the things worth writing in Rust because writing them again in
	10	anything else would mean writing them badly.
	11
	12	```
Move the jolt libraries under glimmer-backends 19df0d8 nandi 11d ago	13	crates/jolt-abi the rules every library here keeps at its edge
	14	crates/jolt-vidya the retained-tree C ABI → libvidya.so
	15	crates/vidya-core the egui semantic layer behind it (theme, widgets, fonts)
	16	crates/jolt-tui the same tree ABI, painted into a terminal → libjolttui.so
	17	crates/jolt-moq freeq's AV media plane → libjoltmoq.so
	18	glimmer-backends/glimmer-vidya the jolt side of libvidya: glimmer's backend
glimmer-cosmic: a libcosmic backend for glimmer (spike) 6a3304d nandi 8d ago	19	crates/jolt-cosmic the same tree ABI, painted by libcosmic → libjoltcosmic.so
	20	glimmer-backends/glimmer-cosmic glimmer's backend over libjoltcosmic (a spike)
Write dvui's shape in jolt, on SDL3, with no shared object 109c7e4 Veronika Winters 9d ago	21	jvui no .so at all: dvui's shape, in jolt, on SDL3
	22	glimmer-backends/glimmer-jvui glimmer's backend over jvui
wayland-bar-compositor: a wlroots compositor in jolt, for panels on X11 1d16366 nandi 8d ago	23	wayland-bar-compositor no .so either: a wlroots compositor in jolt, for panels on X11
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	24	```
	25
Paint the same tree into a terminal, for the machines with no window a7f6202 nandi 17d ago	26	All three objects land in one `target/release`, so a consumer points
Bring vidya in cfd3e36 nandi 19d ago	27	`LD_LIBRARY_PATH` at one directory and names in `:jolt/native` only the ones it
	28	actually wants.
	29
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	30	## Why a workspace and not one library with features
	31
	32	Because cargo features are additive within a library. A single
	33	`libjoltnative.so` whose `graphics` and `moq` features selected the parts would
	34	link egui and QUIC, Opus, H.264, cpal and V4L2 into one object, so a chat
	35	client that is not in a call would carry the whole media plane in order to open
	36	a window. The feature unification that comes with it has bitten this code
	37	before: enabling eframe's `wayland`/`x11` features for a desktop build pulled
	38	them into the Android one and made the activity exit on launch.
	39
	40	A workspace keeps the parts of a monorepo worth having — one CI, one release,
	41	shared conventions in `jolt-abi` — while each consumer links only what it asked
	42	for. jolt's `:jolt/native` is a vector; naming two objects there is no harder
	43	than naming one.
	44
	45	## Building
	46
Point the docs at the flake d682bfa nandi 13d ago	47	The flake is the toolchain. `nix develop` is where the compiler, the C
	48	toolchain, PipeWire's headers and the kernel headers bindgen wants come from,
	49	so nothing needs to be installed beyond nix itself — no `cc` on PATH, no
	50	rustup, no NDK.
	51
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	52	```bash
Point the docs at the flake d682bfa nandi 13d ago	53	nix develop # then:
	54	just build # release .so files in target/release
	55	just test # the whole workspace
	56	just libdir # where to point LD_LIBRARY_PATH
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	57	```
	58
Point the docs at the flake d682bfa nandi 13d ago	59	`just build` is the edit loop; the nix outputs are the artifact, built against
Point the docs at the flake d682bfa nandi 13d ago	60	a pinned compiler and a pinned crate graph:
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	61
Point the docs at the flake d682bfa nandi 13d ago	62	```bash
	63	nix build # all three objects → result/lib
	64	nix build .#android # both device objects → result/lib/arm64-v8a
	65	nix flake check # fmt, clippy, tests, all three objects
	66	```
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	67
Point the docs at the flake d682bfa nandi 13d ago	68	Two things are worth knowing about:
	69
	70	* bindgen runs libclang directly, so `v4l2r` needs to be pointed at
	71	`linux/videodev2.h`. The flake sets `V4L2R_VIDEODEV2_H_PATH` and
	72	`BINDGEN_EXTRA_CLANG_ARGS` from nixpkgs' `linuxHeaders` in both the devShell
	73	and the build, which is why building inside `nix develop` rather than a bare
	74	shell is the supported path.
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	75	* PipeWire is a real system dependency — `libspa-sys` wants `pkg-config`
Point the docs at the flake d682bfa nandi 13d ago	76	and PipeWire's headers, and the flake supplies both. It is behind the
	77	default-on `pipewire` feature of `jolt-moq`; `just test-minimal` builds
	78	without it, for a machine outside the flake. Turning it off costs device
	79	names: only ALSA PCMs remain visible ("pipewire", "sysdefault") rather than
	80	the real sources a browser would offer, which makes picking a microphone
	81	guesswork.
	82
	83	### Android
	84
	85	```bash
	86	just ffi-android # nix build .#android
	87	```
	88
	89	Same sources and same `Cargo.lock` as the desktop objects; only the target
	90	configuration moves. The NDK comes from nixpkgs, so this needs nothing
	91	installed either. The camera wants two things from the APK that the build
	92	cannot supply: a `CameraCapture` class in its `classes.dex`, and a call to
	93	`joltmoq_android_init` — see `android/jolt_main.c`.
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	94
	95	## jolt-moq
	96
	97	freeq's audio and video calls, extracted from
	98	[sleek](https://github.com/codegod100/sleek), which reached them by calling
	99	Rust from Rust.
	100
	101	A freeq call has two halves, and only one of them is here.
	102
	103	Signaling stayed behind. A call is opened, joined and left over IRC
	104	TAGMSGs — `+freeq.at/av-start` and its siblings — and the server broadcasts
	105	`+freeq.at/av-state` back. Any client that speaks IRC already has everything it
	106	needs for that, in whatever language it speaks IRC in; crossing an FFI boundary
	107	to send a TAGMSG would be worse than not crossing one. This library begins once
	108	the session id and the SFU token are known.
	109
	110	The media plane came across, because it is not a thing to write twice: MoQ
	111	over QUIC, Opus, H.264, camera capture and the SFU's own dial rules come to
	112	some three thousand lines that would say the same thing again in another
	113	language and be wrong in different places.
	114
	115	`crates/jolt-moq/include/joltmoq.h` is the surface, and says more about each
	116	call than this does. The shape of it:
	117
	118	* Nothing calls back. Status is drained with `joltmoq_poll_status`, video
	119	with `joltmoq_frame_poll`, both from whatever thread the caller paints on. A
	120	callback into a foreign runtime from a tokio worker is a rule about threads
	121	that the caller has to keep and that nothing can check.
	122	* Frames are borrowed, not copied. `joltmoq_frame_rgba` points into the
	123	decoder's own buffer until the next poll. A frame is a megabyte or two;
	124	copying it out so the caller can hand it straight to a texture upload would
	125	be two copies a frame for nothing.
	126	* One call at a time. There is one microphone, so there is one session.
	127
	128	### Its relationship to sleek
	129
Let the media plane cross to the phone, camera and all fd0e21a nandi 18d ago	130	`av.rs`, `av_media.rs`, `v4l2cam.rs`, `android_camera.rs` and `nv12_orient.rs`
	131	came from sleek's `android/src`. They are kept close to their originals so changes can still be moved between the
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	132	two by eye. What was dropped on the way was the signaling-and-presentation
	133	half — `ChannelCall`, `LocalCall`, `apply_av_state`, `av_state_message` —
	134	which is sleek's own app state and belongs to a client, not to a media plane.
	135	Dropping it also dropped the `freeq-sdk` dependency entirely.
	136
	137	Sleek still has its own copy. Pointing it at this crate instead is the obvious
	138	next step and has not been taken yet; until it is, a fix made in one place
	139	needs making in the other.
	140
Let the media plane cross to the phone, camera and all fd0e21a nandi 18d ago	141	`android_camera.rs` is the one that did not come across unchanged. Sleek is a
	142	single shared object, so its Camera2 bridge reads the `JavaVM` and the Activity
	143	straight out of the `AndroidApp` that android-activity handed it. Here the glue
	144	is in `libvidya.so` and the media plane is not, so the handles arrive through
	145	`joltmoq_android_init` instead — see `android_jni.rs` for why the usual bridge,
	146	`ndk_context`, cannot cross two cdylibs. Everything below that line is sleek's.
	147
Bring vidya in cfd3e36 nandi 19d ago	148	## vidya
	149
	150	The theme layer, the widget set and the retained-tree ABI that glimmer paints
	151	through. Imported from its own repo rather than depended on: egui is the only
	152	backend anyone wants now, so the indirection that let `libvidya` be swapped for
	153	a raylib or cimgui build of the same symbols was paying for a choice nobody was
	154	going to make.
	155
	156	`crates/vidya-core` is the egui layer — and the 4MB under its `assets/` is not
	157	incidental: the Twemoji pack and the symbols font are `include_bytes!`d into
	158	the library, because a shared object has no host app to install a font set for
	159	it.
	160
Move the jolt libraries under glimmer-backends 19df0d8 nandi 11d ago	161	`glimmer-backends/glimmer-vidya` is the other side of that ABI, and is jolt
	162	rather than Rust. A consumer takes it as a `:local/root` dep and `libvidya.so` on the
Bring vidya in cfd3e36 nandi 19d ago	163	loader path — frq's `deps.edn` is the worked example.
Bring vidya in cfd3e36 nandi 19d ago	164
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	165	## Adding a crate
	166
	167	One crate, one `.so`, one capability. Depend on `jolt-abi` and keep its three
	168	rules — catch panics at every entry point, lend strings from a `Scratch` rather
	169	than handing out memory to free, and let the caller ask rather than calling it
	170	back. A consumer then names your object in `:jolt/native` alongside whichever
	171	others it wants, and pays for nothing else.
Say which licence, in the place people look for it fc16d8e nandi 19d ago	172
	173	## License
	174
	175	MIT — the same as the projects this code came from: `crates/vidya-*` and
Move the jolt libraries under glimmer-backends 19df0d8 nandi 11d ago	176	`glimmer-backends/glimmer-vidya` from [vidya](https://tangled.org/nandi.uk/vidya),
Say which licence, in the place people look for it fc16d8e nandi 19d ago	177	`crates/jolt-moq`'s media plane from
	178	[sleek](https://github.com/codegod100/sleek). `third-party/cpal` is a vendored
	179	pin of [cpal](https://github.com/RustAudio/cpal) and keeps its own licence.