nandi/jolt-native · README.md

⑂ dce285fb5a5ec1f331b8afa7b2bdc4ed5e1bbd46

README.md · 150 lines · 7.3 KBmarkdown    Blame HistoryRaw
Lift freeq's AV media plane out of sleek   1# jolt-native
3Native capabilities for [jolt](https://github.com/jolt-lang/jolt), one shared
4object per capability.
6A jolt library binds a `.so` through `jolt.ffi/defcfn`: it declares typed
7foreign functions and gets back integers, doubles and borrowed UTF-8 strings.
8That is the whole vocabulary. These crates are the other side of that
9boundary — the things worth writing in Rust because writing them again in
10anything else would mean writing them badly.
Bring vidya in   13crates/jolt-abi     the rules every library here keeps at its edge
14crates/jolt-vidya   the retained-tree C ABI  → libvidya.so
15crates/vidya-core   the egui semantic layer behind it (theme, widgets, fonts)
Paint the same tree into a terminal, for the machines with no window   16crates/jolt-tui     the same tree ABI, painted into a terminal → libjolttui.so
Bring vidya in   17crates/jolt-moq     freeq's AV media plane   → libjoltmoq.so
18jolt/glimmer-vidya  the jolt side of libvidya: glimmer's backend
Lift freeq's AV media plane out of sleek   19
Paint the same tree into a terminal, for the machines with no window   21All three objects land in one `target/release`, so a consumer points
Bring vidya in   22`LD_LIBRARY_PATH` at one directory and names in `:jolt/native` only the ones it
23actually wants.
Lift freeq's AV media plane out of sleek   25## Why a workspace and not one library with features
27Because cargo features are additive *within* a library. A single
28`libjoltnative.so` whose `graphics` and `moq` features selected the parts would
29link egui **and** QUIC, Opus, H.264, cpal and V4L2 into one object, so a chat
30client that is not in a call would carry the whole media plane in order to open
31a window. The feature unification that comes with it has bitten this code
32before: enabling eframe's `wayland`/`x11` features for a desktop build pulled
33them into the Android one and made the activity exit on launch.
35A workspace keeps the parts of a monorepo worth having — one CI, one release,
36shared conventions in `jolt-abi` — while each consumer links only what it asked
37for. jolt's `:jolt/native` is a vector; naming two objects there is no harder
38than naming one.
40## Building
43just build      # release .so files in target/release
44just test       # the whole workspace
45just libdir     # where to point LD_LIBRARY_PATH
48Everything is compiled by `zig cc`, which brings its own glibc sysroot and
49Linux headers, so **no system C or C++ toolchain is needed** — this repo builds
50on a machine with no `cc` on its PATH at all. The pinned tools come from
51DotSlash files in `scripts/`, fetched on first use; `dotslash` must be on PATH.
53Two things sit outside that promise and are worth knowing about:
55* **bindgen runs libclang directly** and does not inherit zig's header search
56  path, so `v4l2r` cannot find `linux/videodev2.h`. The justfile points it at
57  zig's bundled copies. That is what `scripts/zig-include` is for, and why
58  building through `just` rather than bare `cargo` is the supported path.
59* **PipeWire** is a real system dependency — `libspa-sys` wants `pkg-config`
60  and PipeWire's headers. It is behind the default-on `pipewire` feature of
61  `jolt-moq`; `just test-minimal` builds without it. Turning it off costs
62  device *names*: only ALSA PCMs remain visible ("pipewire", "sysdefault")
63  rather than the real sources a browser would offer, which makes picking a
64  microphone guesswork.
66## jolt-moq
68freeq's audio and video calls, extracted from
69[sleek](https://github.com/codegod100/sleek), which reached them by calling
70Rust from Rust.
72A freeq call has two halves, and only one of them is here.
74**Signaling stayed behind.** A call is opened, joined and left over IRC
75TAGMSGs — `+freeq.at/av-start` and its siblings — and the server broadcasts
76`+freeq.at/av-state` back. Any client that speaks IRC already has everything it
77needs for that, in whatever language it speaks IRC in; crossing an FFI boundary
78to send a TAGMSG would be worse than not crossing one. This library begins once
79the session id and the SFU token are known.
81**The media plane came across**, because it is not a thing to write twice: MoQ
82over QUIC, Opus, H.264, camera capture and the SFU's own dial rules come to
83some three thousand lines that would say the same thing again in another
84language and be wrong in different places.
86`crates/jolt-moq/include/joltmoq.h` is the surface, and says more about each
87call than this does. The shape of it:
89* **Nothing calls back.** Status is drained with `joltmoq_poll_status`, video
90  with `joltmoq_frame_poll`, both from whatever thread the caller paints on. A
91  callback into a foreign runtime from a tokio worker is a rule about threads
92  that the caller has to keep and that nothing can check.
93* **Frames are borrowed, not copied.** `joltmoq_frame_rgba` points into the
94  decoder's own buffer until the next poll. A frame is a megabyte or two;
95  copying it out so the caller can hand it straight to a texture upload would
96  be two copies a frame for nothing.
97* **One call at a time.** There is one microphone, so there is one session.
99### Its relationship to sleek
Let the media plane cross to the phone, camera and all   101`av.rs`, `av_media.rs`, `v4l2cam.rs`, `android_camera.rs` and `nv12_orient.rs`
102came from sleek's `android/src`. They are kept close to their originals so changes can still be moved between the
Lift freeq's AV media plane out of sleek   103two by eye. What was dropped on the way was the signaling-and-presentation
104half — `ChannelCall`, `LocalCall`, `apply_av_state`, `av_state_message` —
105which is sleek's own app state and belongs to a client, not to a media plane.
106Dropping it also dropped the `freeq-sdk` dependency entirely.
108Sleek still has its own copy. Pointing it at this crate instead is the obvious
109next step and has not been taken yet; until it is, a fix made in one place
110needs making in the other.
Let the media plane cross to the phone, camera and all   112`android_camera.rs` is the one that did not come across unchanged. Sleek is a
113single shared object, so its Camera2 bridge reads the `JavaVM` and the Activity
114straight out of the `AndroidApp` that android-activity handed it. Here the glue
115is in `libvidya.so` and the media plane is not, so the handles arrive through
116`joltmoq_android_init` instead — see `android_jni.rs` for why the usual bridge,
117`ndk_context`, cannot cross two cdylibs. Everything below that line is sleek's.
Bring vidya in   119
121The theme layer, the widget set and the retained-tree ABI that glimmer paints
122through. Imported from its own repo rather than depended on: egui is the only
123backend anyone wants now, so the indirection that let `libvidya` be swapped for
124a raylib or cimgui build of the same symbols was paying for a choice nobody was
125going to make.
127`crates/vidya-core` is the egui layer — and the 4MB under its `assets/` is not
128incidental: the Twemoji pack and the symbols font are `include_bytes!`d into
129the library, because a shared object has no host app to install a font set for
132`jolt/glimmer-vidya` is the other side of that ABI, and is jolt rather than
133Rust. A consumer takes it as a `:local/root` dep and `libvidya.so` on the
134loader path — frq's `deps.edn` is the worked example.
Lift freeq's AV media plane out of sleek   136## Adding a crate
138One crate, one `.so`, one capability. Depend on `jolt-abi` and keep its three
139rules — catch panics at every entry point, lend strings from a `Scratch` rather
140than handing out memory to free, and let the caller ask rather than calling it
141back. A consumer then names your object in `:jolt/native` alongside whichever
142others it wants, and pays for nothing else.
Say which licence, in the place people look for it   143
144
146MIT — the same as the projects this code came from: `crates/vidya-*` and
147`jolt/glimmer-vidya` from [vidya](https://tangled.org/nandi.uk/vidya),
148`crates/jolt-moq`'s media plane from
149[sleek](https://github.com/codegod100/sleek). `third-party/cpal` is a vendored
150pin of [cpal](https://github.com/RustAudio/cpal) and keeps its own licence.

Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	1	# jolt-native
	2
	3	Native capabilities for [jolt](https://github.com/jolt-lang/jolt), one shared
	4	object per capability.
	5
	6	A jolt library binds a `.so` through `jolt.ffi/defcfn`: it declares typed
	7	foreign functions and gets back integers, doubles and borrowed UTF-8 strings.
	8	That is the whole vocabulary. These crates are the other side of that
	9	boundary — the things worth writing in Rust because writing them again in
	10	anything else would mean writing them badly.
	11
	12	```
Bring vidya in cfd3e36 nandi 19d ago	13	crates/jolt-abi the rules every library here keeps at its edge
	14	crates/jolt-vidya the retained-tree C ABI → libvidya.so
	15	crates/vidya-core the egui semantic layer behind it (theme, widgets, fonts)
Paint the same tree into a terminal, for the machines with no window a7f6202 nandi 17d ago	16	crates/jolt-tui the same tree ABI, painted into a terminal → libjolttui.so
Bring vidya in cfd3e36 nandi 19d ago	17	crates/jolt-moq freeq's AV media plane → libjoltmoq.so
Bring vidya in cfd3e36 nandi 19d ago	18	jolt/glimmer-vidya the jolt side of libvidya: glimmer's backend
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	19	```
	20
Paint the same tree into a terminal, for the machines with no window a7f6202 nandi 17d ago	21	All three objects land in one `target/release`, so a consumer points
Bring vidya in cfd3e36 nandi 19d ago	22	`LD_LIBRARY_PATH` at one directory and names in `:jolt/native` only the ones it
	23	actually wants.
	24
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	25	## Why a workspace and not one library with features
	26
	27	Because cargo features are additive within a library. A single
	28	`libjoltnative.so` whose `graphics` and `moq` features selected the parts would
	29	link egui and QUIC, Opus, H.264, cpal and V4L2 into one object, so a chat
	30	client that is not in a call would carry the whole media plane in order to open
	31	a window. The feature unification that comes with it has bitten this code
	32	before: enabling eframe's `wayland`/`x11` features for a desktop build pulled
	33	them into the Android one and made the activity exit on launch.
	34
	35	A workspace keeps the parts of a monorepo worth having — one CI, one release,
	36	shared conventions in `jolt-abi` — while each consumer links only what it asked
	37	for. jolt's `:jolt/native` is a vector; naming two objects there is no harder
	38	than naming one.
	39
	40	## Building
	41
	42	```bash
	43	just build # release .so files in target/release
	44	just test # the whole workspace
	45	just libdir # where to point LD_LIBRARY_PATH
	46	```
	47
	48	Everything is compiled by `zig cc`, which brings its own glibc sysroot and
	49	Linux headers, so no system C or C++ toolchain is needed — this repo builds
	50	on a machine with no `cc` on its PATH at all. The pinned tools come from
	51	DotSlash files in `scripts/`, fetched on first use; `dotslash` must be on PATH.
	52
	53	Two things sit outside that promise and are worth knowing about:
	54
	55	* bindgen runs libclang directly and does not inherit zig's header search
	56	path, so `v4l2r` cannot find `linux/videodev2.h`. The justfile points it at
	57	zig's bundled copies. That is what `scripts/zig-include` is for, and why
	58	building through `just` rather than bare `cargo` is the supported path.
	59	* PipeWire is a real system dependency — `libspa-sys` wants `pkg-config`
	60	and PipeWire's headers. It is behind the default-on `pipewire` feature of
	61	`jolt-moq`; `just test-minimal` builds without it. Turning it off costs
	62	device names: only ALSA PCMs remain visible ("pipewire", "sysdefault")
	63	rather than the real sources a browser would offer, which makes picking a
	64	microphone guesswork.
	65
	66	## jolt-moq
	67
	68	freeq's audio and video calls, extracted from
	69	[sleek](https://github.com/codegod100/sleek), which reached them by calling
	70	Rust from Rust.
	71
	72	A freeq call has two halves, and only one of them is here.
	73
	74	Signaling stayed behind. A call is opened, joined and left over IRC
	75	TAGMSGs — `+freeq.at/av-start` and its siblings — and the server broadcasts
	76	`+freeq.at/av-state` back. Any client that speaks IRC already has everything it
	77	needs for that, in whatever language it speaks IRC in; crossing an FFI boundary
	78	to send a TAGMSG would be worse than not crossing one. This library begins once
	79	the session id and the SFU token are known.
	80
	81	The media plane came across, because it is not a thing to write twice: MoQ
	82	over QUIC, Opus, H.264, camera capture and the SFU's own dial rules come to
	83	some three thousand lines that would say the same thing again in another
	84	language and be wrong in different places.
	85
	86	`crates/jolt-moq/include/joltmoq.h` is the surface, and says more about each
	87	call than this does. The shape of it:
	88
89	* Nothing calls back. Status is drained with `joltmoq_poll_status`, video
90	with `joltmoq_frame_poll`, both from whatever thread the caller paints on. A
91	callback into a foreign runtime from a tokio worker is a rule about threads
92	that the caller has to keep and that nothing can check.
93	* Frames are borrowed, not copied. `joltmoq_frame_rgba` points into the
94	decoder's own buffer until the next poll. A frame is a megabyte or two;
95	copying it out so the caller can hand it straight to a texture upload would
96	be two copies a frame for nothing.
97	* One call at a time. There is one microphone, so there is one session.
98
99	### Its relationship to sleek
100
Let the media plane cross to the phone, camera and all fd0e21a nandi 18d ago	101	`av.rs`, `av_media.rs`, `v4l2cam.rs`, `android_camera.rs` and `nv12_orient.rs`
	102	came from sleek's `android/src`. They are kept close to their originals so changes can still be moved between the
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	103	two by eye. What was dropped on the way was the signaling-and-presentation
	104	half — `ChannelCall`, `LocalCall`, `apply_av_state`, `av_state_message` —
	105	which is sleek's own app state and belongs to a client, not to a media plane.
	106	Dropping it also dropped the `freeq-sdk` dependency entirely.
	107
	108	Sleek still has its own copy. Pointing it at this crate instead is the obvious
	109	next step and has not been taken yet; until it is, a fix made in one place
	110	needs making in the other.
	111
Let the media plane cross to the phone, camera and all fd0e21a nandi 18d ago	112	`android_camera.rs` is the one that did not come across unchanged. Sleek is a
	113	single shared object, so its Camera2 bridge reads the `JavaVM` and the Activity
	114	straight out of the `AndroidApp` that android-activity handed it. Here the glue
	115	is in `libvidya.so` and the media plane is not, so the handles arrive through
	116	`joltmoq_android_init` instead — see `android_jni.rs` for why the usual bridge,
	117	`ndk_context`, cannot cross two cdylibs. Everything below that line is sleek's.
	118
Bring vidya in cfd3e36 nandi 19d ago	119	## vidya
	120
	121	The theme layer, the widget set and the retained-tree ABI that glimmer paints
	122	through. Imported from its own repo rather than depended on: egui is the only
	123	backend anyone wants now, so the indirection that let `libvidya` be swapped for
	124	a raylib or cimgui build of the same symbols was paying for a choice nobody was
	125	going to make.
	126
	127	`crates/vidya-core` is the egui layer — and the 4MB under its `assets/` is not
	128	incidental: the Twemoji pack and the symbols font are `include_bytes!`d into
	129	the library, because a shared object has no host app to install a font set for
	130	it.
	131
	132	`jolt/glimmer-vidya` is the other side of that ABI, and is jolt rather than
	133	Rust. A consumer takes it as a `:local/root` dep and `libvidya.so` on the
	134	loader path — frq's `deps.edn` is the worked example.
	135
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	136	## Adding a crate
	137
	138	One crate, one `.so`, one capability. Depend on `jolt-abi` and keep its three
	139	rules — catch panics at every entry point, lend strings from a `Scratch` rather
	140	than handing out memory to free, and let the caller ask rather than calling it
	141	back. A consumer then names your object in `:jolt/native` alongside whichever
	142	others it wants, and pays for nothing else.
Say which licence, in the place people look for it fc16d8e nandi 19d ago	143
	144	## License
	145
	146	MIT — the same as the projects this code came from: `crates/vidya-*` and
	147	`jolt/glimmer-vidya` from [vidya](https://tangled.org/nandi.uk/vidya),
	148	`crates/jolt-moq`'s media plane from
	149	[sleek](https://github.com/codegod100/sleek). `third-party/cpal` is a vendored
	150	pin of [cpal](https://github.com/RustAudio/cpal) and keeps its own licence.