nandi/jolt-native · README.md

⑂ cfd3e3677bed92e80cfe447469a249cfdc0b1401

README.md · 134 lines · 6.4 KBmarkdown    Blame HistoryRaw
Lift freeq's AV media plane out of sleek   1# jolt-native
3Native capabilities for [jolt](https://github.com/jolt-lang/jolt), one shared
4object per capability.
6A jolt library binds a `.so` through `jolt.ffi/defcfn`: it declares typed
7foreign functions and gets back integers, doubles and borrowed UTF-8 strings.
8That is the whole vocabulary. These crates are the other side of that
9boundary — the things worth writing in Rust because writing them again in
10anything else would mean writing them badly.
Bring vidya in   13crates/jolt-abi     the rules every library here keeps at its edge
14crates/jolt-vidya   the retained-tree C ABI  → libvidya.so
15crates/vidya-core   the egui semantic layer behind it (theme, widgets, fonts)
16crates/jolt-moq     freeq's AV media plane   → libjoltmoq.so
17jolt/glimmer-vidya  the jolt side of libvidya: glimmer's backend
Lift freeq's AV media plane out of sleek   18
Bring vidya in   20Both objects land in one `target/release`, so a consumer points
21`LD_LIBRARY_PATH` at one directory and names in `:jolt/native` only the ones it
22actually wants.
Lift freeq's AV media plane out of sleek   24## Why a workspace and not one library with features
26Because cargo features are additive *within* a library. A single
27`libjoltnative.so` whose `graphics` and `moq` features selected the parts would
28link egui **and** QUIC, Opus, H.264, cpal and V4L2 into one object, so a chat
29client that is not in a call would carry the whole media plane in order to open
30a window. The feature unification that comes with it has bitten this code
31before: enabling eframe's `wayland`/`x11` features for a desktop build pulled
32them into the Android one and made the activity exit on launch.
34A workspace keeps the parts of a monorepo worth having — one CI, one release,
35shared conventions in `jolt-abi` — while each consumer links only what it asked
36for. jolt's `:jolt/native` is a vector; naming two objects there is no harder
37than naming one.
39## Building
42just build      # release .so files in target/release
43just test       # the whole workspace
44just libdir     # where to point LD_LIBRARY_PATH
47Everything is compiled by `zig cc`, which brings its own glibc sysroot and
48Linux headers, so **no system C or C++ toolchain is needed** — this repo builds
49on a machine with no `cc` on its PATH at all. The pinned tools come from
50DotSlash files in `scripts/`, fetched on first use; `dotslash` must be on PATH.
52Two things sit outside that promise and are worth knowing about:
54* **bindgen runs libclang directly** and does not inherit zig's header search
55  path, so `v4l2r` cannot find `linux/videodev2.h`. The justfile points it at
56  zig's bundled copies. That is what `scripts/zig-include` is for, and why
57  building through `just` rather than bare `cargo` is the supported path.
58* **PipeWire** is a real system dependency — `libspa-sys` wants `pkg-config`
59  and PipeWire's headers. It is behind the default-on `pipewire` feature of
60  `jolt-moq`; `just test-minimal` builds without it. Turning it off costs
61  device *names*: only ALSA PCMs remain visible ("pipewire", "sysdefault")
62  rather than the real sources a browser would offer, which makes picking a
63  microphone guesswork.
65## jolt-moq
67freeq's audio and video calls, extracted from
68[sleek](https://github.com/codegod100/sleek), which reached them by calling
69Rust from Rust.
71A freeq call has two halves, and only one of them is here.
73**Signaling stayed behind.** A call is opened, joined and left over IRC
74TAGMSGs — `+freeq.at/av-start` and its siblings — and the server broadcasts
75`+freeq.at/av-state` back. Any client that speaks IRC already has everything it
76needs for that, in whatever language it speaks IRC in; crossing an FFI boundary
77to send a TAGMSG would be worse than not crossing one. This library begins once
78the session id and the SFU token are known.
80**The media plane came across**, because it is not a thing to write twice: MoQ
81over QUIC, Opus, H.264, camera capture and the SFU's own dial rules come to
82some three thousand lines that would say the same thing again in another
83language and be wrong in different places.
85`crates/jolt-moq/include/joltmoq.h` is the surface, and says more about each
86call than this does. The shape of it:
88* **Nothing calls back.** Status is drained with `joltmoq_poll_status`, video
89  with `joltmoq_frame_poll`, both from whatever thread the caller paints on. A
90  callback into a foreign runtime from a tokio worker is a rule about threads
91  that the caller has to keep and that nothing can check.
92* **Frames are borrowed, not copied.** `joltmoq_frame_rgba` points into the
93  decoder's own buffer until the next poll. A frame is a megabyte or two;
94  copying it out so the caller can hand it straight to a texture upload would
95  be two copies a frame for nothing.
96* **One call at a time.** There is one microphone, so there is one session.
98### Its relationship to sleek
100`av.rs`, `av_media.rs` and `v4l2cam.rs` came from sleek's `android/src`. They
101are kept close to their originals so changes can still be moved between the
102two by eye. What was dropped on the way was the signaling-and-presentation
103half — `ChannelCall`, `LocalCall`, `apply_av_state`, `av_state_message` —
104which is sleek's own app state and belongs to a client, not to a media plane.
105Dropping it also dropped the `freeq-sdk` dependency entirely.
107Sleek still has its own copy. Pointing it at this crate instead is the obvious
108next step and has not been taken yet; until it is, a fix made in one place
109needs making in the other.
Bring vidya in   111
113The theme layer, the widget set and the retained-tree ABI that glimmer paints
114through. Imported from its own repo rather than depended on: egui is the only
115backend anyone wants now, so the indirection that let `libvidya` be swapped for
116a raylib or cimgui build of the same symbols was paying for a choice nobody was
117going to make.
119`crates/vidya-core` is the egui layer — and the 4MB under its `assets/` is not
120incidental: the Twemoji pack and the symbols font are `include_bytes!`d into
121the library, because a shared object has no host app to install a font set for
124`jolt/glimmer-vidya` is the other side of that ABI, and is jolt rather than
125Rust. A consumer takes it as a `:local/root` dep and `libvidya.so` on the
126loader path — frq's `deps.edn` is the worked example.
Lift freeq's AV media plane out of sleek   128## Adding a crate
130One crate, one `.so`, one capability. Depend on `jolt-abi` and keep its three
131rules — catch panics at every entry point, lend strings from a `Scratch` rather
132than handing out memory to free, and let the caller ask rather than calling it
133back. A consumer then names your object in `:jolt/native` alongside whichever
134others it wants, and pays for nothing else.

Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	1	# jolt-native
	2
	3	Native capabilities for [jolt](https://github.com/jolt-lang/jolt), one shared
	4	object per capability.
	5
	6	A jolt library binds a `.so` through `jolt.ffi/defcfn`: it declares typed
	7	foreign functions and gets back integers, doubles and borrowed UTF-8 strings.
	8	That is the whole vocabulary. These crates are the other side of that
	9	boundary — the things worth writing in Rust because writing them again in
	10	anything else would mean writing them badly.
	11
	12	```
Bring vidya in cfd3e36 nandi 19d ago	13	crates/jolt-abi the rules every library here keeps at its edge
	14	crates/jolt-vidya the retained-tree C ABI → libvidya.so
	15	crates/vidya-core the egui semantic layer behind it (theme, widgets, fonts)
	16	crates/jolt-moq freeq's AV media plane → libjoltmoq.so
	17	jolt/glimmer-vidya the jolt side of libvidya: glimmer's backend
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	18	```
	19
Bring vidya in cfd3e36 nandi 19d ago	20	Both objects land in one `target/release`, so a consumer points
	21	`LD_LIBRARY_PATH` at one directory and names in `:jolt/native` only the ones it
	22	actually wants.
	23
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	24	## Why a workspace and not one library with features
	25
	26	Because cargo features are additive within a library. A single
	27	`libjoltnative.so` whose `graphics` and `moq` features selected the parts would
	28	link egui and QUIC, Opus, H.264, cpal and V4L2 into one object, so a chat
	29	client that is not in a call would carry the whole media plane in order to open
	30	a window. The feature unification that comes with it has bitten this code
	31	before: enabling eframe's `wayland`/`x11` features for a desktop build pulled
	32	them into the Android one and made the activity exit on launch.
	33
	34	A workspace keeps the parts of a monorepo worth having — one CI, one release,
	35	shared conventions in `jolt-abi` — while each consumer links only what it asked
	36	for. jolt's `:jolt/native` is a vector; naming two objects there is no harder
	37	than naming one.
	38
	39	## Building
	40
	41	```bash
	42	just build # release .so files in target/release
	43	just test # the whole workspace
	44	just libdir # where to point LD_LIBRARY_PATH
	45	```
	46
	47	Everything is compiled by `zig cc`, which brings its own glibc sysroot and
	48	Linux headers, so no system C or C++ toolchain is needed — this repo builds
	49	on a machine with no `cc` on its PATH at all. The pinned tools come from
	50	DotSlash files in `scripts/`, fetched on first use; `dotslash` must be on PATH.
	51
	52	Two things sit outside that promise and are worth knowing about:
	53
	54	* bindgen runs libclang directly and does not inherit zig's header search
	55	path, so `v4l2r` cannot find `linux/videodev2.h`. The justfile points it at
	56	zig's bundled copies. That is what `scripts/zig-include` is for, and why
	57	building through `just` rather than bare `cargo` is the supported path.
	58	* PipeWire is a real system dependency — `libspa-sys` wants `pkg-config`
	59	and PipeWire's headers. It is behind the default-on `pipewire` feature of
	60	`jolt-moq`; `just test-minimal` builds without it. Turning it off costs
	61	device names: only ALSA PCMs remain visible ("pipewire", "sysdefault")
	62	rather than the real sources a browser would offer, which makes picking a
	63	microphone guesswork.
	64
	65	## jolt-moq
	66
	67	freeq's audio and video calls, extracted from
	68	[sleek](https://github.com/codegod100/sleek), which reached them by calling
	69	Rust from Rust.
	70
	71	A freeq call has two halves, and only one of them is here.
	72
	73	Signaling stayed behind. A call is opened, joined and left over IRC
	74	TAGMSGs — `+freeq.at/av-start` and its siblings — and the server broadcasts
	75	`+freeq.at/av-state` back. Any client that speaks IRC already has everything it
	76	needs for that, in whatever language it speaks IRC in; crossing an FFI boundary
	77	to send a TAGMSG would be worse than not crossing one. This library begins once
	78	the session id and the SFU token are known.
	79
	80	The media plane came across, because it is not a thing to write twice: MoQ
	81	over QUIC, Opus, H.264, camera capture and the SFU's own dial rules come to
	82	some three thousand lines that would say the same thing again in another
	83	language and be wrong in different places.
	84
	85	`crates/jolt-moq/include/joltmoq.h` is the surface, and says more about each
	86	call than this does. The shape of it:
	87
88	* Nothing calls back. Status is drained with `joltmoq_poll_status`, video
89	with `joltmoq_frame_poll`, both from whatever thread the caller paints on. A
90	callback into a foreign runtime from a tokio worker is a rule about threads
91	that the caller has to keep and that nothing can check.
92	* Frames are borrowed, not copied. `joltmoq_frame_rgba` points into the
93	decoder's own buffer until the next poll. A frame is a megabyte or two;
94	copying it out so the caller can hand it straight to a texture upload would
95	be two copies a frame for nothing.
96	* One call at a time. There is one microphone, so there is one session.
97
98	### Its relationship to sleek
99
100	`av.rs`, `av_media.rs` and `v4l2cam.rs` came from sleek's `android/src`. They
101	are kept close to their originals so changes can still be moved between the
102	two by eye. What was dropped on the way was the signaling-and-presentation
103	half — `ChannelCall`, `LocalCall`, `apply_av_state`, `av_state_message` —
104	which is sleek's own app state and belongs to a client, not to a media plane.
105	Dropping it also dropped the `freeq-sdk` dependency entirely.
106
107	Sleek still has its own copy. Pointing it at this crate instead is the obvious
108	next step and has not been taken yet; until it is, a fix made in one place
109	needs making in the other.
110
Bring vidya in cfd3e36 nandi 19d ago	111	## vidya
	112
	113	The theme layer, the widget set and the retained-tree ABI that glimmer paints
	114	through. Imported from its own repo rather than depended on: egui is the only
	115	backend anyone wants now, so the indirection that let `libvidya` be swapped for
	116	a raylib or cimgui build of the same symbols was paying for a choice nobody was
	117	going to make.
	118
	119	`crates/vidya-core` is the egui layer — and the 4MB under its `assets/` is not
	120	incidental: the Twemoji pack and the symbols font are `include_bytes!`d into
	121	the library, because a shared object has no host app to install a font set for
	122	it.
	123
	124	`jolt/glimmer-vidya` is the other side of that ABI, and is jolt rather than
	125	Rust. A consumer takes it as a `:local/root` dep and `libvidya.so` on the
	126	loader path — frq's `deps.edn` is the worked example.
	127
Lift freeq's AV media plane out of sleek 90f8b89 nandi 19d ago	128	## Adding a crate
	129
	130	One crate, one `.so`, one capability. Depend on `jolt-abi` and keep its three
	131	rules — catch panics at every entry point, lend strings from a `Scratch` rather
	132	than handing out memory to free, and let the caller ask rather than calling it
	133	back. A consumer then names your object in `:jolt/native` alongside whichever
	134	others it wants, and pays for nothing else.