nandi/jolt-native · README.md

⑂ 90f8b89576cb60fadfc57e138ac4499fcf1d2213

README.md · 110 lines · 5.3 KBmarkdown    Blame HistoryRaw
Lift freeq's AV media plane out of sleek   1# jolt-native
3Native capabilities for [jolt](https://github.com/jolt-lang/jolt), one shared
4object per capability.
6A jolt library binds a `.so` through `jolt.ffi/defcfn`: it declares typed
7foreign functions and gets back integers, doubles and borrowed UTF-8 strings.
8That is the whole vocabulary. These crates are the other side of that
9boundary — the things worth writing in Rust because writing them again in
10anything else would mean writing them badly.
13crates/jolt-abi   the rules every library here keeps at its edge
14crates/jolt-moq   freeq's AV media plane — MoQ over QUIC, Opus, H.264, capture
17## Why a workspace and not one library with features
19Because cargo features are additive *within* a library. A single
20`libjoltnative.so` whose `graphics` and `moq` features selected the parts would
21link egui **and** QUIC, Opus, H.264, cpal and V4L2 into one object, so a chat
22client that is not in a call would carry the whole media plane in order to open
23a window. The feature unification that comes with it has bitten this code
24before: enabling eframe's `wayland`/`x11` features for a desktop build pulled
25them into the Android one and made the activity exit on launch.
27A workspace keeps the parts of a monorepo worth having — one CI, one release,
28shared conventions in `jolt-abi` — while each consumer links only what it asked
29for. jolt's `:jolt/native` is a vector; naming two objects there is no harder
30than naming one.
32## Building
35just build      # release .so files in target/release
36just test       # the whole workspace
37just libdir     # where to point LD_LIBRARY_PATH
40Everything is compiled by `zig cc`, which brings its own glibc sysroot and
41Linux headers, so **no system C or C++ toolchain is needed** — this repo builds
42on a machine with no `cc` on its PATH at all. The pinned tools come from
43DotSlash files in `scripts/`, fetched on first use; `dotslash` must be on PATH.
45Two things sit outside that promise and are worth knowing about:
47* **bindgen runs libclang directly** and does not inherit zig's header search
48  path, so `v4l2r` cannot find `linux/videodev2.h`. The justfile points it at
49  zig's bundled copies. That is what `scripts/zig-include` is for, and why
50  building through `just` rather than bare `cargo` is the supported path.
51* **PipeWire** is a real system dependency — `libspa-sys` wants `pkg-config`
52  and PipeWire's headers. It is behind the default-on `pipewire` feature of
53  `jolt-moq`; `just test-minimal` builds without it. Turning it off costs
54  device *names*: only ALSA PCMs remain visible ("pipewire", "sysdefault")
55  rather than the real sources a browser would offer, which makes picking a
56  microphone guesswork.
58## jolt-moq
60freeq's audio and video calls, extracted from
61[sleek](https://github.com/codegod100/sleek), which reached them by calling
62Rust from Rust.
64A freeq call has two halves, and only one of them is here.
66**Signaling stayed behind.** A call is opened, joined and left over IRC
67TAGMSGs — `+freeq.at/av-start` and its siblings — and the server broadcasts
68`+freeq.at/av-state` back. Any client that speaks IRC already has everything it
69needs for that, in whatever language it speaks IRC in; crossing an FFI boundary
70to send a TAGMSG would be worse than not crossing one. This library begins once
71the session id and the SFU token are known.
73**The media plane came across**, because it is not a thing to write twice: MoQ
74over QUIC, Opus, H.264, camera capture and the SFU's own dial rules come to
75some three thousand lines that would say the same thing again in another
76language and be wrong in different places.
78`crates/jolt-moq/include/joltmoq.h` is the surface, and says more about each
79call than this does. The shape of it:
81* **Nothing calls back.** Status is drained with `joltmoq_poll_status`, video
82  with `joltmoq_frame_poll`, both from whatever thread the caller paints on. A
83  callback into a foreign runtime from a tokio worker is a rule about threads
84  that the caller has to keep and that nothing can check.
85* **Frames are borrowed, not copied.** `joltmoq_frame_rgba` points into the
86  decoder's own buffer until the next poll. A frame is a megabyte or two;
87  copying it out so the caller can hand it straight to a texture upload would
88  be two copies a frame for nothing.
89* **One call at a time.** There is one microphone, so there is one session.
91### Its relationship to sleek
93`av.rs`, `av_media.rs` and `v4l2cam.rs` came from sleek's `android/src`. They
94are kept close to their originals so changes can still be moved between the
95two by eye. What was dropped on the way was the signaling-and-presentation
96half — `ChannelCall`, `LocalCall`, `apply_av_state`, `av_state_message` —
97which is sleek's own app state and belongs to a client, not to a media plane.
98Dropping it also dropped the `freeq-sdk` dependency entirely.
100Sleek still has its own copy. Pointing it at this crate instead is the obvious
101next step and has not been taken yet; until it is, a fix made in one place
102needs making in the other.
104## Adding a crate
106One crate, one `.so`, one capability. Depend on `jolt-abi` and keep its three
107rules — catch panics at every entry point, lend strings from a `Scratch` rather
108than handing out memory to free, and let the caller ask rather than calling it
109back. A consumer then names your object in `:jolt/native` alongside whichever
110others it wants, and pays for nothing else.

Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	1	# jolt-native
	2
	3	Native capabilities for [jolt](https://github.com/jolt-lang/jolt), one shared
	4	object per capability.
	5
	6	A jolt library binds a `.so` through `jolt.ffi/defcfn`: it declares typed
	7	foreign functions and gets back integers, doubles and borrowed UTF-8 strings.
	8	That is the whole vocabulary. These crates are the other side of that
	9	boundary — the things worth writing in Rust because writing them again in
	10	anything else would mean writing them badly.
	11
	12	```
	13	crates/jolt-abi the rules every library here keeps at its edge
	14	crates/jolt-moq freeq's AV media plane — MoQ over QUIC, Opus, H.264, capture
	15	```
	16
	17	## Why a workspace and not one library with features
	18
	19	Because cargo features are additive within a library. A single
	20	`libjoltnative.so` whose `graphics` and `moq` features selected the parts would
	21	link egui and QUIC, Opus, H.264, cpal and V4L2 into one object, so a chat
	22	client that is not in a call would carry the whole media plane in order to open
	23	a window. The feature unification that comes with it has bitten this code
	24	before: enabling eframe's `wayland`/`x11` features for a desktop build pulled
	25	them into the Android one and made the activity exit on launch.
	26
	27	A workspace keeps the parts of a monorepo worth having — one CI, one release,
	28	shared conventions in `jolt-abi` — while each consumer links only what it asked
	29	for. jolt's `:jolt/native` is a vector; naming two objects there is no harder
	30	than naming one.
	31
	32	## Building
	33
	34	```bash
	35	just build # release .so files in target/release
	36	just test # the whole workspace
	37	just libdir # where to point LD_LIBRARY_PATH
	38	```
	39
	40	Everything is compiled by `zig cc`, which brings its own glibc sysroot and
	41	Linux headers, so no system C or C++ toolchain is needed — this repo builds
	42	on a machine with no `cc` on its PATH at all. The pinned tools come from
	43	DotSlash files in `scripts/`, fetched on first use; `dotslash` must be on PATH.
	44
	45	Two things sit outside that promise and are worth knowing about:
	46
	47	* bindgen runs libclang directly and does not inherit zig's header search
	48	path, so `v4l2r` cannot find `linux/videodev2.h`. The justfile points it at
	49	zig's bundled copies. That is what `scripts/zig-include` is for, and why
	50	building through `just` rather than bare `cargo` is the supported path.
	51	* PipeWire is a real system dependency — `libspa-sys` wants `pkg-config`
	52	and PipeWire's headers. It is behind the default-on `pipewire` feature of
	53	`jolt-moq`; `just test-minimal` builds without it. Turning it off costs
	54	device names: only ALSA PCMs remain visible ("pipewire", "sysdefault")
	55	rather than the real sources a browser would offer, which makes picking a
	56	microphone guesswork.
	57
	58	## jolt-moq
	59
	60	freeq's audio and video calls, extracted from
	61	[sleek](https://github.com/codegod100/sleek), which reached them by calling
	62	Rust from Rust.
	63
	64	A freeq call has two halves, and only one of them is here.
65
66	Signaling stayed behind. A call is opened, joined and left over IRC
67	TAGMSGs — `+freeq.at/av-start` and its siblings — and the server broadcasts
68	`+freeq.at/av-state` back. Any client that speaks IRC already has everything it
69	needs for that, in whatever language it speaks IRC in; crossing an FFI boundary
70	to send a TAGMSG would be worse than not crossing one. This library begins once
71	the session id and the SFU token are known.
72
73	The media plane came across, because it is not a thing to write twice: MoQ
74	over QUIC, Opus, H.264, camera capture and the SFU's own dial rules come to
75	some three thousand lines that would say the same thing again in another
76	language and be wrong in different places.
77
78	`crates/jolt-moq/include/joltmoq.h` is the surface, and says more about each
79	call than this does. The shape of it:
80
81	* Nothing calls back. Status is drained with `joltmoq_poll_status`, video
82	with `joltmoq_frame_poll`, both from whatever thread the caller paints on. A
83	callback into a foreign runtime from a tokio worker is a rule about threads
84	that the caller has to keep and that nothing can check.
85	* Frames are borrowed, not copied. `joltmoq_frame_rgba` points into the
86	decoder's own buffer until the next poll. A frame is a megabyte or two;
87	copying it out so the caller can hand it straight to a texture upload would
88	be two copies a frame for nothing.
89	* One call at a time. There is one microphone, so there is one session.
90
91	### Its relationship to sleek
92
93	`av.rs`, `av_media.rs` and `v4l2cam.rs` came from sleek's `android/src`. They
94	are kept close to their originals so changes can still be moved between the
95	two by eye. What was dropped on the way was the signaling-and-presentation
96	half — `ChannelCall`, `LocalCall`, `apply_av_state`, `av_state_message` —
97	which is sleek's own app state and belongs to a client, not to a media plane.
98	Dropping it also dropped the `freeq-sdk` dependency entirely.
99
100	Sleek still has its own copy. Pointing it at this crate instead is the obvious
101	next step and has not been taken yet; until it is, a fix made in one place
102	needs making in the other.
103
104	## Adding a crate
105
106	One crate, one `.so`, one capability. Depend on `jolt-abi` and keep its three
107	rules — catch panics at every entry point, lend strings from a `Scratch` rather
108	than handing out memory to free, and let the caller ask rather than calling it
109	back. A consumer then names your object in `:jolt/native` alongside whichever
110	others it wants, and pays for nothing else.