rickub
Sign in Get started
nandi/frqpublic⑂ Fork 0
⑂ 87e5be9
Branches
claude/add-remote-repository-47d12e main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/frq.git
git clone ssh://git@rickub.com/nandi/frq.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Bind the C libraries a call actually needs

Opus, H.264, the camera and the audio devices, as C rather than as somebody's
bindings to C. libmoq_ffi keeps the one piece with no C answer -- MoQ over
QUIC -- and everything else now comes from libraries that have had a stable
ABI for twenty years.

frq.codec.opus is libopus, flat and unremarkable. PCM crosses as [pointer
length] and never becomes a jolt value: a 20ms stereo frame is 1920 samples
and building a vector of them fifty times a second is work with nothing to
show for it. frame-size counts samples PER CHANNEL, which is the mistake
worth naming -- at stereo the interleaved count is also a legal frame size,
so nothing raises and the audio simply runs fast.

frq.codec.h264 is openh264 one step removed, and the step is not optional.
openh264's C API is a vtable: ISVCEncoder is `const ISVCEncoderVtbl*` and
every method is a function pointer hanging off the object. jolt.ffi cannot
call one -- Chez fixes a foreign procedure's types when it COMPILES it, and
the target has to be a literal C symbol name. So c/frq_h264.c walks the
vtable and exports five plain symbols. It also flattens openh264's
SFrameBSInfo, whose layers and NAL counts are its business rather than ours.
A hundred lines of C against a library nixpkgs already has; the contrast with
the 1062-crate build it replaces is the whole point. The alternative was
x264, whose C API is flat and whose licence is not ours to change.

frq.capture.v4l2 is ioctls, so it names no library at all. Every VIDIOC_
request number encodes the size of the struct it carries, which makes a
layout one byte wrong not a misread field but a request the kernel has never
heard of -- ENOTTY for an ioctl that plainly exists. Every size and offset is
therefore checked against a C program compiled on this kernel's own headers.
Fifteen of fifteen agree. There is no camera in this container, so the
capture path itself is unexercised and the smoke test says so rather than
implying otherwise.

frq.capture.alsa binds the small API: snd_pcm_set_params does in one call
what hw_params does in thirty accessors, and interleaved S16 at a fixed rate
is what Opus wants on one side and a device gives on the other. Reads are
counted in FRAMES; an overrun is recovered rather than raised, because a
dropped buffer is a thing a call survives. Exercised against ALSA's `null`
PCM, which is always there and needs no hardware.

Frames stay borrowed throughout, in both directions -- a captured buffer goes
to the encoder as the driver's pointer and is requeued in a finally, the way
a decoded one already reaches a texture.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nandi committed 2026-09-10T04:00:56-07:00 Browse files
87e5be9 parent: f518938
added c/frq_h264.c +132 -0
new file mode 100644
@@ -0,0 +1,132 @@
1+/* A flat C face for openh264's encoder.
2+ *
3+ * WHY THIS EXISTS. openh264's C API is not flat. `ISVCEncoder` is
4+ * `const ISVCEncoderVtbl*` — a pointer to a table of function pointers — so
5+ * calling Initialize or EncodeFrame means dereferencing the object, reading a
6+ * slot, and calling through it. jolt.ffi cannot do that: Chez fixes a foreign
7+ * procedure's types when it COMPILES it, and the target has to be a literal C
8+ * symbol name rather than a function pointer (see jolt/ffi.clj, "the target
9+ * must be a literal C symbol name"). So the vtable is walked here, in C, and
10+ * what jolt binds is the five plain symbols below.
11+ *
12+ * It is deliberately thin. No policy, no buffering beyond what openh264's own
13+ * output demands, and no decisions that belong in frq — the shim exists to
14+ * change a calling convention, not to be a video pipeline.
15+ *
16+ * THE ONE THING IT DOES DO is flatten the output. openh264 hands back an
17+ * SFrameBSInfo describing up to MAX_LAYER_NUM_OF_FRAME layers, each with its
18+ * own NAL count and a shared bitstream buffer. A caller wanting one Annex B
19+ * frame has to walk that; doing it in jolt would mean reading nested C structs
20+ * whose layout is openh264's business. It is copied into one contiguous
21+ * buffer owned by the encoder handle and handed over as a borrowed span,
22+ * valid until the next encode — which is exactly the contract frq.av already
23+ * has for a video frame.
24+ */
25+#include <stdlib.h>
26+#include <string.h>
27+#include <stdint.h>
28+#include <wels/codec_api.h>
29+#include <wels/codec_app_def.h>
30+
31+typedef struct {
32+ ISVCEncoder *enc;
33+ unsigned char *out; /* flattened Annex B, grown as needed */
34+ size_t out_cap;
35+ int width, height;
36+} frq_h264;
37+
38+/* Answers 0 on success, or openh264's own non-zero return. */
39+int frq_h264_open(int width, int height, int fps, int bitrate, void **handle) {
40+ ISVCEncoder *enc = NULL;
41+ frq_h264 *h;
42+ SEncParamBase p;
43+ int rc;
44+
45+ *handle = NULL;
46+ rc = WelsCreateSVCEncoder(&enc);
47+ if (rc != 0 || enc == NULL) return rc ? rc : -1;
48+
49+ memset(&p, 0, sizeof(p));
50+ p.iUsageType = CAMERA_VIDEO_REAL_TIME;
51+ p.iPicWidth = width;
52+ p.iPicHeight = height;
53+ p.iTargetBitrate = bitrate;
54+ p.fMaxFrameRate = (float)fps;
55+
56+ rc = (*enc)->Initialize(enc, &p);
57+ if (rc != 0) { WelsDestroySVCEncoder(enc); return rc; }
58+
59+ h = (frq_h264 *)calloc(1, sizeof(frq_h264));
60+ if (h == NULL) { (*enc)->Uninitialize(enc); WelsDestroySVCEncoder(enc); return -1; }
61+ h->enc = enc; h->width = width; h->height = height;
62+ *handle = h;
63+ return 0;
64+}
65+
66+/* Encode one I420 frame.
67+ *
68+ * `i420` is width*height luma followed by two (width/2)*(height/2) planes.
69+ * On success answers 0 and sets *out / *out_len to a BORROWED span, valid
70+ * until the next call on this handle. *keyframe says whether it is an IDR.
71+ * A frame openh264 chose to skip answers 0 with *out_len == 0. */
72+int frq_h264_encode(void *handle, const unsigned char *i420, long long pts_us,
73+ const unsigned char **out, int *out_len, int *keyframe) {
74+ frq_h264 *h = (frq_h264 *)handle;
75+ SSourcePicture pic;
76+ SFrameBSInfo info;
77+ int rc, i, j, total = 0, off = 0;
78+
79+ *out = NULL; *out_len = 0; *keyframe = 0;
80+
81+ memset(&pic, 0, sizeof(pic));
82+ pic.iPicWidth = h->width;
83+ pic.iPicHeight = h->height;
84+ pic.iColorFormat = videoFormatI420;
85+ pic.iStride[0] = h->width;
86+ pic.iStride[1] = h->width / 2;
87+ pic.iStride[2] = h->width / 2;
88+ pic.pData[0] = (unsigned char *)i420;
89+ pic.pData[1] = pic.pData[0] + h->width * h->height;
90+ pic.pData[2] = pic.pData[1] + (h->width / 2) * (h->height / 2);
91+ pic.uiTimeStamp = pts_us / 1000; /* openh264 counts milliseconds */
92+
93+ memset(&info, 0, sizeof(info));
94+ rc = (*h->enc)->EncodeFrame(h->enc, &pic, &info);
95+ if (rc != cmResultSuccess) return rc;
96+ if (info.eFrameType == videoFrameTypeSkip) return 0;
97+
98+ for (i = 0; i < info.iLayerNum; i++)
99+ for (j = 0; j < info.sLayerInfo[i].iNalCount; j++)
100+ total += info.sLayerInfo[i].pNalLengthInByte[j];
101+
102+ if ((size_t)total > h->out_cap) {
103+ unsigned char *grown = (unsigned char *)realloc(h->out, (size_t)total);
104+ if (grown == NULL) return -1;
105+ h->out = grown; h->out_cap = (size_t)total;
106+ }
107+ for (i = 0; i < info.iLayerNum; i++) {
108+ int n = 0, k;
109+ for (k = 0; k < info.sLayerInfo[i].iNalCount; k++)
110+ n += info.sLayerInfo[i].pNalLengthInByte[k];
111+ memcpy(h->out + off, info.sLayerInfo[i].pBsBuf, (size_t)n);
112+ off += n;
113+ }
114+
115+ *out = h->out;
116+ *out_len = total;
117+ *keyframe = (info.eFrameType == videoFrameTypeIDR);
118+ return 0;
119+}
120+
121+int frq_h264_force_keyframe(void *handle) {
122+ frq_h264 *h = (frq_h264 *)handle;
123+ return (*h->enc)->ForceIntraFrame(h->enc, true);
124+}
125+
126+void frq_h264_close(void *handle) {
127+ frq_h264 *h = (frq_h264 *)handle;
128+ if (h == NULL) return;
129+ if (h->enc) { (*h->enc)->Uninitialize(h->enc); WelsDestroySVCEncoder(h->enc); }
130+ free(h->out);
131+ free(h);
132+}
new file mode 100644
@@ -0,0 +1,132 @@
1+/* A flat C face for openh264's encoder.
2+ *
3+ * WHY THIS EXISTS. openh264's C API is not flat. `ISVCEncoder` is
4+ * `const ISVCEncoderVtbl*` — a pointer to a table of function pointers — so
5+ * calling Initialize or EncodeFrame means dereferencing the object, reading a
6+ * slot, and calling through it. jolt.ffi cannot do that: Chez fixes a foreign
7+ * procedure's types when it COMPILES it, and the target has to be a literal C
8+ * symbol name rather than a function pointer (see jolt/ffi.clj, "the target
9+ * must be a literal C symbol name"). So the vtable is walked here, in C, and
10+ * what jolt binds is the five plain symbols below.
11+ *
12+ * It is deliberately thin. No policy, no buffering beyond what openh264's own
13+ * output demands, and no decisions that belong in frq — the shim exists to
14+ * change a calling convention, not to be a video pipeline.
15+ *
16+ * THE ONE THING IT DOES DO is flatten the output. openh264 hands back an
17+ * SFrameBSInfo describing up to MAX_LAYER_NUM_OF_FRAME layers, each with its
18+ * own NAL count and a shared bitstream buffer. A caller wanting one Annex B
19+ * frame has to walk that; doing it in jolt would mean reading nested C structs
20+ * whose layout is openh264's business. It is copied into one contiguous
21+ * buffer owned by the encoder handle and handed over as a borrowed span,
22+ * valid until the next encode — which is exactly the contract frq.av already
23+ * has for a video frame.
24+ */
25+#include <stdlib.h>
26+#include <string.h>
27+#include <stdint.h>
28+#include <wels/codec_api.h>
29+#include <wels/codec_app_def.h>
30+
31+typedef struct {
32+ ISVCEncoder *enc;
33+ unsigned char *out; /* flattened Annex B, grown as needed */
34+ size_t out_cap;
35+ int width, height;
36+} frq_h264;
37+
38+/* Answers 0 on success, or openh264's own non-zero return. */
39+int frq_h264_open(int width, int height, int fps, int bitrate, void **handle) {
40+ ISVCEncoder *enc = NULL;
41+ frq_h264 *h;
42+ SEncParamBase p;
43+ int rc;
44+
45+ *handle = NULL;
46+ rc = WelsCreateSVCEncoder(&enc);
47+ if (rc != 0 || enc == NULL) return rc ? rc : -1;
48+
49+ memset(&p, 0, sizeof(p));
50+ p.iUsageType = CAMERA_VIDEO_REAL_TIME;
51+ p.iPicWidth = width;
52+ p.iPicHeight = height;
53+ p.iTargetBitrate = bitrate;
54+ p.fMaxFrameRate = (float)fps;
55+
56+ rc = (*enc)->Initialize(enc, &p);
57+ if (rc != 0) { WelsDestroySVCEncoder(enc); return rc; }
58+
59+ h = (frq_h264 *)calloc(1, sizeof(frq_h264));
60+ if (h == NULL) { (*enc)->Uninitialize(enc); WelsDestroySVCEncoder(enc); return -1; }
61+ h->enc = enc; h->width = width; h->height = height;
62+ *handle = h;
63+ return 0;
64+}
65+
66+/* Encode one I420 frame.
67+ *
68+ * `i420` is width*height luma followed by two (width/2)*(height/2) planes.
69+ * On success answers 0 and sets *out / *out_len to a BORROWED span, valid
70+ * until the next call on this handle. *keyframe says whether it is an IDR.
71+ * A frame openh264 chose to skip answers 0 with *out_len == 0. */
72+int frq_h264_encode(void *handle, const unsigned char *i420, long long pts_us,
73+ const unsigned char **out, int *out_len, int *keyframe) {
74+ frq_h264 *h = (frq_h264 *)handle;
75+ SSourcePicture pic;
76+ SFrameBSInfo info;
77+ int rc, i, j, total = 0, off = 0;
78+
79+ *out = NULL; *out_len = 0; *keyframe = 0;
80+
81+ memset(&pic, 0, sizeof(pic));
82+ pic.iPicWidth = h->width;
83+ pic.iPicHeight = h->height;
84+ pic.iColorFormat = videoFormatI420;
85+ pic.iStride[0] = h->width;
86+ pic.iStride[1] = h->width / 2;
87+ pic.iStride[2] = h->width / 2;
88+ pic.pData[0] = (unsigned char *)i420;
89+ pic.pData[1] = pic.pData[0] + h->width * h->height;
90+ pic.pData[2] = pic.pData[1] + (h->width / 2) * (h->height / 2);
91+ pic.uiTimeStamp = pts_us / 1000; /* openh264 counts milliseconds */
92+
93+ memset(&info, 0, sizeof(info));
94+ rc = (*h->enc)->EncodeFrame(h->enc, &pic, &info);
95+ if (rc != cmResultSuccess) return rc;
96+ if (info.eFrameType == videoFrameTypeSkip) return 0;
97+
98+ for (i = 0; i < info.iLayerNum; i++)
99+ for (j = 0; j < info.sLayerInfo[i].iNalCount; j++)
100+ total += info.sLayerInfo[i].pNalLengthInByte[j];
101+
102+ if ((size_t)total > h->out_cap) {
103+ unsigned char *grown = (unsigned char *)realloc(h->out, (size_t)total);
104+ if (grown == NULL) return -1;
105+ h->out = grown; h->out_cap = (size_t)total;
106+ }
107+ for (i = 0; i < info.iLayerNum; i++) {
108+ int n = 0, k;
109+ for (k = 0; k < info.sLayerInfo[i].iNalCount; k++)
110+ n += info.sLayerInfo[i].pNalLengthInByte[k];
111+ memcpy(h->out + off, info.sLayerInfo[i].pBsBuf, (size_t)n);
112+ off += n;
113+ }
114+
115+ *out = h->out;
116+ *out_len = total;
117+ *keyframe = (info.eFrameType == videoFrameTypeIDR);
118+ return 0;
119+}
120+
121+int frq_h264_force_keyframe(void *handle) {
122+ frq_h264 *h = (frq_h264 *)handle;
123+ return (*h->enc)->ForceIntraFrame(h->enc, true);
124+}
125+
126+void frq_h264_close(void *handle) {
127+ frq_h264 *h = (frq_h264 *)handle;
128+ if (h == NULL) return;
129+ if (h->enc) { (*h->enc)->Uninitialize(h->enc); WelsDestroySVCEncoder(h->enc); }
130+ free(h->out);
131+ free(h);
132+}
modified deps.edn +32 -1
@@ -51,7 +51,38 @@
5151 ;; No :darwin: :systems is x86_64-linux and aarch64-linux, so
5252 ;; a .dylib named here would name a file nothing fetches.
5353 {:name "moq_ffi"
54- :linux ["libmoq_ffi.so"]}]
54+ :linux ["libmoq_ffi.so"]}
55+
56+ ;; The codecs, as C libraries rather than as somebody's
57+ ;; bindings to them. Opus first; openh264 beside it when the
58+ ;; video half lands.
59+ ;;
60+ ;; The SONAME, not the bare .so: a `libopus.so` is the -dev
61+ ;; symlink and is not what a runtime closure carries. The
62+ ;; loader looks these up by name in one directory, which the
63+ ;; flake's `nativeAll` is.
64+ {:name "opus"
65+ :linux ["libopus.so.0"]
66+ :darwin ["libopus.0.dylib"]}
67+
68+ ;; H.264, one step removed. openh264's C API is a vtable —
69+ ;; `ISVCEncoder` is `const ISVCEncoderVtbl*` — and jolt.ffi
70+ ;; can only call a literal C symbol, never a function
71+ ;; pointer. c/frq_h264.c walks the vtable and exports flat
72+ ;; symbols; this is that. libopenh264 itself comes along as
73+ ;; its DT_NEEDED and is never named here.
74+ {:name "frqh264"
75+ :linux ["libfrqh264.so"]
76+ :darwin ["libfrqh264.dylib"]}
77+
78+ ;; Audio devices. No :darwin — CoreAudio is the other side of
79+ ;; that door and is not this library.
80+ ;;
81+ ;; V4L2 is deliberately absent from this list: the camera is
82+ ;; ioctls against libc and the kernel, so there is nothing to
83+ ;; load. See frq.capture.v4l2.
84+ {:name "asound"
85+ :linux ["libasound.so.2"]}]
5586
5687 :aliases {:frq {:main-opts ["-m" "frq.app"]}
5788
@@ -51,7 +51,38 @@
51 ;; No :darwin: :systems is x86_64-linux and aarch64-linux, so51 ;; No :darwin: :systems is x86_64-linux and aarch64-linux, so
52 ;; a .dylib named here would name a file nothing fetches.52 ;; a .dylib named here would name a file nothing fetches.
53 {:name "moq_ffi"53 {:name "moq_ffi"
54- :linux ["libmoq_ffi.so"]}]54+ :linux ["libmoq_ffi.so"]}
55+
56+ ;; The codecs, as C libraries rather than as somebody's
57+ ;; bindings to them. Opus first; openh264 beside it when the
58+ ;; video half lands.
59+ ;;
60+ ;; The SONAME, not the bare .so: a `libopus.so` is the -dev
61+ ;; symlink and is not what a runtime closure carries. The
62+ ;; loader looks these up by name in one directory, which the
63+ ;; flake's `nativeAll` is.
64+ {:name "opus"
65+ :linux ["libopus.so.0"]
66+ :darwin ["libopus.0.dylib"]}
67+
68+ ;; H.264, one step removed. openh264's C API is a vtable —
69+ ;; `ISVCEncoder` is `const ISVCEncoderVtbl*` — and jolt.ffi
70+ ;; can only call a literal C symbol, never a function
71+ ;; pointer. c/frq_h264.c walks the vtable and exports flat
72+ ;; symbols; this is that. libopenh264 itself comes along as
73+ ;; its DT_NEEDED and is never named here.
74+ {:name "frqh264"
75+ :linux ["libfrqh264.so"]
76+ :darwin ["libfrqh264.dylib"]}
77+
78+ ;; Audio devices. No :darwin — CoreAudio is the other side of
79+ ;; that door and is not this library.
80+ ;;
81+ ;; V4L2 is deliberately absent from this list: the camera is
82+ ;; ioctls against libc and the kernel, so there is nothing to
83+ ;; load. See frq.capture.v4l2.
84+ {:name "asound"
85+ :linux ["libasound.so.2"]}]
55 86
56 :aliases {:frq {:main-opts ["-m" "frq.app"]}87 :aliases {:frq {:main-opts ["-m" "frq.app"]}
57 88
modified flake.nix +41 -2
@@ -240,9 +240,48 @@
240240 # from two places — jolt-native's flake, and the moq-ffi release — so
241241 # they are joined rather than the path being made a list, which the
242242 # loader does not take.
243+ # The C codecs, from nixpkgs. libmoq_ffi carries the transport and
244+ # nothing else — moq-ffi's `audio` and `video` features would have
245+ # brought Opus and H.264 with them, at the price of compiling a
246+ # 1062-crate workspace — so the codecs are linked here instead,
247+ # where they have always lived.
248+ #
249+ # Named in :jolt/native, so the loader resolves them the same way it
250+ # resolves libvidya: by name, out of one directory.
251+ # A flat C face for openh264, because openh264 has none. Its
252+ # `ISVCEncoder` is `const ISVCEncoderVtbl*` — every method is a
253+ # function pointer in a vtable — and jolt.ffi cannot call one: Chez
254+ # fixes a foreign procedure's types when it compiles it, and the
255+ # target must be a literal C symbol name. So the vtable is walked in
256+ # c/frq_h264.c and jolt binds the five plain symbols it exports.
257+ #
258+ # One translation unit against a library nixpkgs already has. It is
259+ # a calling convention adapter, not a second media plane, and the
260+ # distinction from the moq-ffi build it replaces is the whole point:
261+ # this compiles one .c file, not a 1062-crate workspace.
262+ frqH264 = pkgs.stdenv.mkDerivation {
263+ pname = "frq-h264";
264+ version = "0.1";
265+ src = ./c;
266+ nativeBuildInputs = [ pkgs.pkg-config ];
267+ buildInputs = [ pkgs.openh264 ];
268+ buildPhase = ''
269+ $CC -O2 -fPIC -shared frq_h264.c -o libfrqh264.so \
270+ $(pkg-config --cflags --libs openh264)
271+ '';
272+ installPhase = ''
273+ mkdir -p $out/lib && cp libfrqh264.so $out/lib/
274+ '';
275+ };
276+
277+ # openh264 is here for frqH264's DT_NEEDED; alsa-lib for capture
278+ # and playback. V4L2 needs nothing: it is ioctls against libc and
279+ # the kernel, so there is no library to name.
280+ codecs = [ pkgs.libopus pkgs.openh264 frqH264 pkgs.alsa-lib ];
281+
243282 nativeAll = pkgs.symlinkJoin {
244283 name = "frq-native";
245- paths = [ native moqFfi ];
284+ paths = [ native moqFfi ] ++ codecs;
246285 };
247286
248287 # Jolt itself: Clojure on Chez, built the way its own flake builds it.
@@ -397,7 +436,7 @@
397436 };
398437 in
399438 {
400- inherit native moqFfi nativeAll frq;
439+ inherit native moqFfi frqH264 nativeAll frq;
401440 inherit tui;
402441 jolt = joltRuntime;
403442 default = frq;
@@ -240,9 +240,48 @@
240 # from two places — jolt-native's flake, and the moq-ffi release — so240 # from two places — jolt-native's flake, and the moq-ffi release — so
241 # they are joined rather than the path being made a list, which the241 # they are joined rather than the path being made a list, which the
242 # loader does not take.242 # loader does not take.
243+ # The C codecs, from nixpkgs. libmoq_ffi carries the transport and
244+ # nothing else — moq-ffi's `audio` and `video` features would have
245+ # brought Opus and H.264 with them, at the price of compiling a
246+ # 1062-crate workspace — so the codecs are linked here instead,
247+ # where they have always lived.
248+ #
249+ # Named in :jolt/native, so the loader resolves them the same way it
250+ # resolves libvidya: by name, out of one directory.
251+ # A flat C face for openh264, because openh264 has none. Its
252+ # `ISVCEncoder` is `const ISVCEncoderVtbl*` — every method is a
253+ # function pointer in a vtable — and jolt.ffi cannot call one: Chez
254+ # fixes a foreign procedure's types when it compiles it, and the
255+ # target must be a literal C symbol name. So the vtable is walked in
256+ # c/frq_h264.c and jolt binds the five plain symbols it exports.
257+ #
258+ # One translation unit against a library nixpkgs already has. It is
259+ # a calling convention adapter, not a second media plane, and the
260+ # distinction from the moq-ffi build it replaces is the whole point:
261+ # this compiles one .c file, not a 1062-crate workspace.
262+ frqH264 = pkgs.stdenv.mkDerivation {
263+ pname = "frq-h264";
264+ version = "0.1";
265+ src = ./c;
266+ nativeBuildInputs = [ pkgs.pkg-config ];
267+ buildInputs = [ pkgs.openh264 ];
268+ buildPhase = ''
269+ $CC -O2 -fPIC -shared frq_h264.c -o libfrqh264.so \
270+ $(pkg-config --cflags --libs openh264)
271+ '';
272+ installPhase = ''
273+ mkdir -p $out/lib && cp libfrqh264.so $out/lib/
274+ '';
275+ };
276+
277+ # openh264 is here for frqH264's DT_NEEDED; alsa-lib for capture
278+ # and playback. V4L2 needs nothing: it is ioctls against libc and
279+ # the kernel, so there is no library to name.
280+ codecs = [ pkgs.libopus pkgs.openh264 frqH264 pkgs.alsa-lib ];
281+
243 nativeAll = pkgs.symlinkJoin {282 nativeAll = pkgs.symlinkJoin {
244 name = "frq-native";283 name = "frq-native";
245- paths = [ native moqFfi ];284+ paths = [ native moqFfi ] ++ codecs;
246 };285 };
247 286
248 # Jolt itself: Clojure on Chez, built the way its own flake builds it.287 # Jolt itself: Clojure on Chez, built the way its own flake builds it.
@@ -397,7 +436,7 @@
397 };436 };
398 in437 in
399 {438 {
400- inherit native moqFfi nativeAll frq;439+ inherit native moqFfi frqH264 nativeAll frq;
401 inherit tui;440 inherit tui;
402 jolt = joltRuntime;441 jolt = joltRuntime;
403 default = frq;442 default = frq;
added src/frq/capture/alsa.clj +88 -0
new file mode 100644
@@ -0,0 +1,88 @@
1+(ns frq.capture.alsa
2+ "Audio devices, through libasound.
3+
4+ The last of the four C libraries the media plane needs, and the least
5+ eventful: ALSA has a flat C API, no vtable and no ioctl arithmetic. What it
6+ does have is two APIs, and this binds the small one — `snd_pcm_set_params`
7+ configures format, access, channels, rate, resampling and latency in a
8+ single call, where the general path is a `snd_pcm_hw_params_t` allocated by
9+ the library and poked field by field through thirty accessors. The small one
10+ is enough for a call: interleaved S16 at a fixed rate is what Opus wants on
11+ one side and what a device gives on the other.
12+
13+ READS ARE BLOCKING and counted in FRAMES, not bytes and not samples. A
14+ frame is one sample per channel, so 960 frames of stereo S16 is 3840 bytes;
15+ passing a byte count asks for four times the audio and blocks for four
16+ times as long, which looks like a slow device rather than a bug.
17+
18+ RECOVERY IS EXPECTED. An overrun on capture is normal on a busy machine and
19+ is not a failure — `snd_pcm_recover` puts the stream back and the next read
20+ continues. `read!` does that itself and reports the loss rather than
21+ raising, because a dropped buffer is a thing a call survives."
22+ (:require [jolt.ffi :as ffi]))
23+
24+(def ^:const format-s16-le 2)
25+(def ^:const access-rw-interleaved 3)
26+(def streams {:playback 0 :capture 1})
27+
28+(ffi/defcfn raw-open "snd_pcm_open" [:pointer :string :int :int] :int)
29+(ffi/defcfn raw-close "snd_pcm_close" [:pointer] :int)
30+(ffi/defcfn raw-set-params "snd_pcm_set_params"
31+ [:pointer :int :int :uint :uint :int :uint] :int)
32+(ffi/defcfn raw-readi "snd_pcm_readi" [:pointer :pointer :uint64] :int64)
33+(ffi/defcfn raw-writei "snd_pcm_writei" [:pointer :pointer :uint64] :int64)
34+(ffi/defcfn raw-prepare "snd_pcm_prepare" [:pointer] :int)
35+(ffi/defcfn raw-recover "snd_pcm_recover" [:pointer :int :int] :int)
36+(ffi/defcfn raw-drain "snd_pcm_drain" [:pointer] :int)
37+(ffi/defcfn strerror "snd_strerror" [:int] :string)
38+
39+(defn- check! [rc what]
40+ (if (neg? rc)
41+ (throw (ex-info (str "alsa: " what ": " (strerror rc)) {:code rc :op what}))
42+ rc))
43+
44+(defn open-pcm
45+ "Open a PCM by ALSA name — \"default\", \"hw:1,0\", or \"null\" for a device
46+ that swallows everything and always exists.
47+
48+ `latency-us` is what ALSA is asked to aim for; it picks buffer and period
49+ sizes to suit and may not hit it exactly."
50+ [name stream {:keys [rate channels latency-us]
51+ :or {rate 48000 channels 1 latency-us 20000}}]
52+ (ffi/with-arena [a]
53+ (let [out (ffi/alloc a 8)
54+ dir (or (streams stream)
55+ (throw (ex-info "unknown pcm stream" {:got stream})))]
56+ (check! (raw-open out name dir 0) (str "open " name))
57+ (let [pcm (ffi/read out :pointer)]
58+ (check! (raw-set-params pcm format-s16-le access-rw-interleaved
59+ channels rate 1 latency-us)
60+ "set_params")
61+ pcm))))
62+
63+(defn read!
64+ "Read up to `frames` frames of interleaved S16 into `buf`.
65+
66+ Answers {:frames n} on a normal read, or {:frames n :recovered true} when
67+ an overrun was absorbed. Frames, not bytes — see the namespace docstring."
68+ [pcm buf frames]
69+ (let [n (raw-readi pcm buf frames)]
70+ (if (neg? n)
71+ (do (check! (raw-recover pcm n 1) "recover")
72+ (let [n2 (raw-readi pcm buf frames)]
73+ {:frames (max 0 (check! n2 "readi")) :recovered true}))
74+ {:frames n})))
75+
76+(defn write!
77+ "Write `frames` frames of interleaved S16 from `buf`."
78+ [pcm buf frames]
79+ (let [n (raw-writei pcm buf frames)]
80+ (if (neg? n)
81+ (do (check! (raw-recover pcm n 1) "recover")
82+ {:frames (max 0 (check! (raw-writei pcm buf frames) "writei"))
83+ :recovered true})
84+ {:frames n})))
85+
86+(defn prepare! [pcm] (check! (raw-prepare pcm) "prepare") nil)
87+(defn drain! [pcm] (check! (raw-drain pcm) "drain") nil)
88+(defn close! [pcm] (raw-close pcm) nil)
new file mode 100644
@@ -0,0 +1,88 @@
1+(ns frq.capture.alsa
2+ "Audio devices, through libasound.
3+
4+ The last of the four C libraries the media plane needs, and the least
5+ eventful: ALSA has a flat C API, no vtable and no ioctl arithmetic. What it
6+ does have is two APIs, and this binds the small one — `snd_pcm_set_params`
7+ configures format, access, channels, rate, resampling and latency in a
8+ single call, where the general path is a `snd_pcm_hw_params_t` allocated by
9+ the library and poked field by field through thirty accessors. The small one
10+ is enough for a call: interleaved S16 at a fixed rate is what Opus wants on
11+ one side and what a device gives on the other.
12+
13+ READS ARE BLOCKING and counted in FRAMES, not bytes and not samples. A
14+ frame is one sample per channel, so 960 frames of stereo S16 is 3840 bytes;
15+ passing a byte count asks for four times the audio and blocks for four
16+ times as long, which looks like a slow device rather than a bug.
17+
18+ RECOVERY IS EXPECTED. An overrun on capture is normal on a busy machine and
19+ is not a failure — `snd_pcm_recover` puts the stream back and the next read
20+ continues. `read!` does that itself and reports the loss rather than
21+ raising, because a dropped buffer is a thing a call survives."
22+ (:require [jolt.ffi :as ffi]))
23+
24+(def ^:const format-s16-le 2)
25+(def ^:const access-rw-interleaved 3)
26+(def streams {:playback 0 :capture 1})
27+
28+(ffi/defcfn raw-open "snd_pcm_open" [:pointer :string :int :int] :int)
29+(ffi/defcfn raw-close "snd_pcm_close" [:pointer] :int)
30+(ffi/defcfn raw-set-params "snd_pcm_set_params"
31+ [:pointer :int :int :uint :uint :int :uint] :int)
32+(ffi/defcfn raw-readi "snd_pcm_readi" [:pointer :pointer :uint64] :int64)
33+(ffi/defcfn raw-writei "snd_pcm_writei" [:pointer :pointer :uint64] :int64)
34+(ffi/defcfn raw-prepare "snd_pcm_prepare" [:pointer] :int)
35+(ffi/defcfn raw-recover "snd_pcm_recover" [:pointer :int :int] :int)
36+(ffi/defcfn raw-drain "snd_pcm_drain" [:pointer] :int)
37+(ffi/defcfn strerror "snd_strerror" [:int] :string)
38+
39+(defn- check! [rc what]
40+ (if (neg? rc)
41+ (throw (ex-info (str "alsa: " what ": " (strerror rc)) {:code rc :op what}))
42+ rc))
43+
44+(defn open-pcm
45+ "Open a PCM by ALSA name — \"default\", \"hw:1,0\", or \"null\" for a device
46+ that swallows everything and always exists.
47+
48+ `latency-us` is what ALSA is asked to aim for; it picks buffer and period
49+ sizes to suit and may not hit it exactly."
50+ [name stream {:keys [rate channels latency-us]
51+ :or {rate 48000 channels 1 latency-us 20000}}]
52+ (ffi/with-arena [a]
53+ (let [out (ffi/alloc a 8)
54+ dir (or (streams stream)
55+ (throw (ex-info "unknown pcm stream" {:got stream})))]
56+ (check! (raw-open out name dir 0) (str "open " name))
57+ (let [pcm (ffi/read out :pointer)]
58+ (check! (raw-set-params pcm format-s16-le access-rw-interleaved
59+ channels rate 1 latency-us)
60+ "set_params")
61+ pcm))))
62+
63+(defn read!
64+ "Read up to `frames` frames of interleaved S16 into `buf`.
65+
66+ Answers {:frames n} on a normal read, or {:frames n :recovered true} when
67+ an overrun was absorbed. Frames, not bytes — see the namespace docstring."
68+ [pcm buf frames]
69+ (let [n (raw-readi pcm buf frames)]
70+ (if (neg? n)
71+ (do (check! (raw-recover pcm n 1) "recover")
72+ (let [n2 (raw-readi pcm buf frames)]
73+ {:frames (max 0 (check! n2 "readi")) :recovered true}))
74+ {:frames n})))
75+
76+(defn write!
77+ "Write `frames` frames of interleaved S16 from `buf`."
78+ [pcm buf frames]
79+ (let [n (raw-writei pcm buf frames)]
80+ (if (neg? n)
81+ (do (check! (raw-recover pcm n 1) "recover")
82+ {:frames (max 0 (check! (raw-writei pcm buf frames) "writei"))
83+ :recovered true})
84+ {:frames n})))
85+
86+(defn prepare! [pcm] (check! (raw-prepare pcm) "prepare") nil)
87+(defn drain! [pcm] (check! (raw-drain pcm) "drain") nil)
88+(defn close! [pcm] (raw-close pcm) nil)
added src/frq/capture/v4l2.clj +264 -0
new file mode 100644
@@ -0,0 +1,264 @@
1+(ns frq.capture.v4l2
2+ "The camera, behind an ioctl.
3+
4+ V4L2 is the one part of the media plane that was never going to need a
5+ library: it is `open`, `ioctl`, `mmap` and a handful of structs, all of it
6+ in libc and the kernel. What it needs instead is EXACTNESS. Every request
7+ number below encodes the size of the struct it carries, so a layout that is
8+ one byte wrong does not read a wrong field — it makes a request number the
9+ kernel has never heard of, and the driver answers ENOTTY for an ioctl that
10+ plainly exists.
11+
12+ That is why `frq.capture.v4l2-test/check-layouts` compares every size and
13+ offset here against what a C compiler says about the running kernel's
14+ headers, rather than trusting that they were transcribed correctly.
15+
16+ A CAPTURED FRAME IS BORROWED. `with-frame` hands the mmap'd buffer straight
17+ to its callback and requeues it afterwards; the pointer is the driver's, it
18+ is valid until the buffer goes back, and nothing here copies it. That is
19+ `frq.av`'s rule arriving from the other end — capture buffer to encoder as a
20+ pointer, the way the decoder's buffer already reaches a texture."
21+ (:require [jolt.ffi :as ffi]))
22+
23+;; --- libc --------------------------------------------------------------------
24+
25+(ffi/defcfn c-open "open" [:string :int :& :int] :int)
26+(ffi/defcfn c-close "close" [:int] :int)
27+;; ioctl's third argument is whatever the request says it is; for every
28+;; request here it is a pointer, and the declared tail costs no compile.
29+(ffi/defcfn c-ioctl "ioctl" [:int :uint64 :& :pointer] :int)
30+(ffi/defcfn c-mmap "mmap" [:pointer :uint64 :int :int :int :int64] :pointer)
31+(ffi/defcfn c-munmap "munmap" [:pointer :uint64] :int)
32+
33+(def ^:const o-rdwr 2)
34+(def ^:const prot-read 1)
35+(def ^:const prot-write 2)
36+(def ^:const map-shared 1)
37+
38+;; --- the requests ------------------------------------------------------------
39+;; _IOC(dir, type, nr, size) = dir<<30 | size<<16 | 'V'<<8 | nr, with dir 1
40+;; for write, 2 for read and 3 for both. The size in there is the struct's,
41+;; which is why the layouts below are checked rather than assumed.
42+
43+(def ^:const VIDIOC_QUERYCAP 2154321408)
44+(def ^:const VIDIOC_S_FMT 3234878981)
45+(def ^:const VIDIOC_REQBUFS 3222558216)
46+(def ^:const VIDIOC_QUERYBUF 3227014665)
47+(def ^:const VIDIOC_QBUF 3227014671)
48+(def ^:const VIDIOC_DQBUF 3227014673)
49+(def ^:const VIDIOC_STREAMON 1074026002)
50+(def ^:const VIDIOC_STREAMOFF 1074026003)
51+
52+(def ^:const buf-type-video-capture 1)
53+(def ^:const memory-mmap 1)
54+(def ^:const field-none 1)
55+(def ^:const cap-video-capture 1)
56+(def ^:const cap-streaming 67108864)
57+
58+(def pixel-formats
59+ "V4L2 fourccs, as the kernel packs them."
60+ {:yuyv 1448695129 :mjpeg 1196444237 :yuv420 842093913})
61+
62+;; --- the structs -------------------------------------------------------------
63+;; Padded to the kernel's sizes rather than described field by field: what
64+;; matters is the total size (it is in the request number) and the offsets of
65+;; the fields actually read. A union is spelled as the reserved block it
66+;; occupies, which is what `v4l2_format` mostly is.
67+
68+(def capability
69+ (ffi/layout [:struct [[:driver [:array :uint8 16]]
70+ [:card [:array :uint8 32]]
71+ [:bus-info [:array :uint8 32]]
72+ [:version :uint32]
73+ [:capabilities :uint32]
74+ [:device-caps :uint32]
75+ [:reserved [:array :uint32 3]]]]))
76+
77+(def format-pix
78+ ;; v4l2_format is 208 bytes: a type, four bytes of padding, then a union
79+ ;; whose largest member decides the rest. Only the pix arm is described;
80+ ;; the tail is the union's remaining bytes.
81+ (ffi/layout [:struct [[:type :uint32]
82+ [:pad :uint32]
83+ [:width :uint32]
84+ [:height :uint32]
85+ [:pixelformat :uint32]
86+ [:field :uint32]
87+ [:bytesperline :uint32]
88+ [:sizeimage :uint32]
89+ [:colorspace :uint32]
90+ [:priv :uint32]
91+ [:flags :uint32]
92+ [:enc :uint32]
93+ [:quantization :uint32]
94+ [:xfer-func :uint32]
95+ [:rest [:array :uint8 152]]]]))
96+
97+(def requestbuffers
98+ (ffi/layout [:struct [[:count :uint32]
99+ [:type :uint32]
100+ [:memory :uint32]
101+ [:capabilities :uint32]
102+ [:flags :uint8]
103+ [:reserved [:array :uint8 3]]]]))
104+
105+(def buffer
106+ ;; 88 bytes. `timestamp` is a struct timeval at 24, `m` is a union at 64
107+ ;; whose first member is the mmap offset, and `memory` sits at 60.
108+ (ffi/layout [:struct [[:index :uint32]
109+ [:type :uint32]
110+ [:bytesused :uint32]
111+ [:flags :uint32]
112+ [:field :uint32]
113+ [:pad0 :uint32]
114+ [:tv-sec :int64]
115+ [:tv-usec :int64]
116+ [:timecode [:array :uint8 16]]
117+ [:sequence :uint32]
118+ [:memory :uint32]
119+ [:offset :uint32]
120+ [:pad1 :uint32]
121+ [:length :uint32]
122+ [:reserved2 :uint32]
123+ [:request-fd :int32]
124+ [:pad2 :uint32]]]))
125+
126+;; --- opening -----------------------------------------------------------------
127+
128+(defn- ioctl! [fd req p what]
129+ (let [rc (c-ioctl fd req p)]
130+ (when (neg? rc)
131+ (throw (ex-info (str "v4l2: " what " failed") {:errno (ffi/errno) :op what})))
132+ rc))
133+
134+(defn open-device
135+ "Open a camera and answer its fd."
136+ [path]
137+ (let [fd (c-open path o-rdwr)]
138+ (when (neg? fd)
139+ (throw (ex-info (str "v4l2: cannot open " path) {:errno (ffi/errno) :path path})))
140+ fd))
141+
142+(defn capabilities
143+ "What the device says it can do. `:capture?` and `:streaming?` are the two
144+ that decide whether the rest of this namespace applies to it."
145+ [fd]
146+ (ffi/with-arena [a]
147+ (let [p (ffi/alloc a (ffi/layout-size capability))]
148+ (ioctl! fd VIDIOC_QUERYCAP p "QUERYCAP")
149+ (let [caps (ffi/read-field p capability [:capabilities])
150+ dev (ffi/read-field p capability [:device-caps])
151+ ;; device_caps describes THIS node; capabilities describes the
152+ ;; whole device, which on a multi-node camera is not the same
153+ ;; thing and is the usual reason a /dev/video1 refuses to stream.
154+ eff (if (zero? dev) caps dev)]
155+ {:capabilities caps
156+ :device-caps dev
157+ :capture? (pos? (bit-and eff cap-video-capture))
158+ :streaming? (pos? (bit-and eff cap-streaming))}))))
159+
160+(defn set-format!
161+ "Ask for a size and pixel format; answers what the driver actually chose.
162+
163+ V4L2 negotiates rather than obeys — a driver may answer a different size or
164+ a different format entirely, and the returned map is the truth."
165+ [fd width height pixel-format]
166+ (ffi/with-arena [a]
167+ (let [p (ffi/alloc a (ffi/layout-size format-pix))
168+ fourcc (or (pixel-formats pixel-format) pixel-format)]
169+ (ffi/write p format-pix {:type buf-type-video-capture :pad 0
170+ :width width :height height
171+ :pixelformat fourcc :field field-none
172+ :bytesperline 0 :sizeimage 0 :colorspace 0
173+ :priv 0 :flags 0 :enc 0 :quantization 0
174+ :xfer-func 0 :rest (vec (repeat 152 0))})
175+ (ioctl! fd VIDIOC_S_FMT p "S_FMT")
176+ {:width (ffi/read-field p format-pix [:width])
177+ :height (ffi/read-field p format-pix [:height])
178+ :pixelformat (ffi/read-field p format-pix [:pixelformat])
179+ :bytesperline (ffi/read-field p format-pix [:bytesperline])
180+ :sizeimage (ffi/read-field p format-pix [:sizeimage])})))
181+
182+;; --- buffers -----------------------------------------------------------------
183+
184+(defn request-buffers!
185+ "Ask the driver for `n` mmap buffers; answers how many it granted."
186+ [fd n]
187+ (ffi/with-arena [a]
188+ (let [p (ffi/alloc a (ffi/layout-size requestbuffers))]
189+ (ffi/write p requestbuffers {:count n :type buf-type-video-capture
190+ :memory memory-mmap :capabilities 0
191+ :flags 0 :reserved [0 0 0]})
192+ (ioctl! fd VIDIOC_REQBUFS p "REQBUFS")
193+ (ffi/read-field p requestbuffers [:count]))))
194+
195+(defn- blank-buffer [p index]
196+ (ffi/write p buffer {:index index :type buf-type-video-capture :bytesused 0
197+ :flags 0 :field 0 :pad0 0 :tv-sec 0 :tv-usec 0
198+ :timecode (vec (repeat 16 0)) :sequence 0
199+ :memory memory-mmap :offset 0 :pad1 0 :length 0
200+ :reserved2 0 :request-fd 0 :pad2 0}))
201+
202+(defn map-buffers!
203+ "QUERYBUF then mmap each buffer; answers a vector of {:ptr :len :index}."
204+ [fd n]
205+ (ffi/with-arena [a]
206+ (let [p (ffi/alloc a (ffi/layout-size buffer))]
207+ (mapv (fn [i]
208+ (blank-buffer p i)
209+ (ioctl! fd VIDIOC_QUERYBUF p "QUERYBUF")
210+ (let [len (ffi/read-field p buffer [:length])
211+ off (ffi/read-field p buffer [:offset])
212+ ptr (c-mmap ffi/null len (bit-or prot-read prot-write)
213+ map-shared fd off)]
214+ (when (= ptr -1)
215+ (throw (ex-info "v4l2: mmap failed" {:errno (ffi/errno) :index i})))
216+ {:index i :ptr ptr :len len}))
217+ (range n)))))
218+
219+(defn queue!
220+ "Hand a buffer back to the driver."
221+ [fd index]
222+ (ffi/with-arena [a]
223+ (let [p (ffi/alloc a (ffi/layout-size buffer))]
224+ (blank-buffer p index)
225+ (ioctl! fd VIDIOC_QBUF p "QBUF")))
226+ nil)
227+
228+(defn stream-on! [fd]
229+ (ffi/with-arena [a]
230+ (let [t (ffi/alloc a 4)]
231+ (ffi/write t :uint32 buf-type-video-capture)
232+ (ioctl! fd VIDIOC_STREAMON t "STREAMON")))
233+ nil)
234+
235+(defn stream-off! [fd]
236+ (ffi/with-arena [a]
237+ (let [t (ffi/alloc a 4)]
238+ (ffi/write t :uint32 buf-type-video-capture)
239+ (ioctl! fd VIDIOC_STREAMOFF t "STREAMOFF")))
240+ nil)
241+
242+(defn with-frame
243+ "Dequeue a frame, hand it to `f` as [pointer length], and requeue it.
244+
245+ The pointer is the driver's mmap'd buffer, valid only until the requeue —
246+ which is why the buffer goes back in a `finally` and why `f` is called
247+ rather than the span being answered. Copying it here would be the one copy
248+ frq.av exists to avoid."
249+ [fd buffers f]
250+ (ffi/with-arena [a]
251+ (let [p (ffi/alloc a (ffi/layout-size buffer))]
252+ (blank-buffer p 0)
253+ (ioctl! fd VIDIOC_DQBUF p "DQBUF")
254+ (let [i (ffi/read-field p buffer [:index])
255+ n (ffi/read-field p buffer [:bytesused])
256+ buf (nth buffers i)]
257+ (try
258+ (f (:ptr buf) n)
259+ (finally (queue! fd i)))))))
260+
261+(defn close-device! [fd buffers]
262+ (doseq [{:keys [ptr len]} buffers] (c-munmap ptr len))
263+ (c-close fd)
264+ nil)
new file mode 100644
@@ -0,0 +1,264 @@
1+(ns frq.capture.v4l2
2+ "The camera, behind an ioctl.
3+
4+ V4L2 is the one part of the media plane that was never going to need a
5+ library: it is `open`, `ioctl`, `mmap` and a handful of structs, all of it
6+ in libc and the kernel. What it needs instead is EXACTNESS. Every request
7+ number below encodes the size of the struct it carries, so a layout that is
8+ one byte wrong does not read a wrong field — it makes a request number the
9+ kernel has never heard of, and the driver answers ENOTTY for an ioctl that
10+ plainly exists.
11+
12+ That is why `frq.capture.v4l2-test/check-layouts` compares every size and
13+ offset here against what a C compiler says about the running kernel's
14+ headers, rather than trusting that they were transcribed correctly.
15+
16+ A CAPTURED FRAME IS BORROWED. `with-frame` hands the mmap'd buffer straight
17+ to its callback and requeues it afterwards; the pointer is the driver's, it
18+ is valid until the buffer goes back, and nothing here copies it. That is
19+ `frq.av`'s rule arriving from the other end — capture buffer to encoder as a
20+ pointer, the way the decoder's buffer already reaches a texture."
21+ (:require [jolt.ffi :as ffi]))
22+
23+;; --- libc --------------------------------------------------------------------
24+
25+(ffi/defcfn c-open "open" [:string :int :& :int] :int)
26+(ffi/defcfn c-close "close" [:int] :int)
27+;; ioctl's third argument is whatever the request says it is; for every
28+;; request here it is a pointer, and the declared tail costs no compile.
29+(ffi/defcfn c-ioctl "ioctl" [:int :uint64 :& :pointer] :int)
30+(ffi/defcfn c-mmap "mmap" [:pointer :uint64 :int :int :int :int64] :pointer)
31+(ffi/defcfn c-munmap "munmap" [:pointer :uint64] :int)
32+
33+(def ^:const o-rdwr 2)
34+(def ^:const prot-read 1)
35+(def ^:const prot-write 2)
36+(def ^:const map-shared 1)
37+
38+;; --- the requests ------------------------------------------------------------
39+;; _IOC(dir, type, nr, size) = dir<<30 | size<<16 | 'V'<<8 | nr, with dir 1
40+;; for write, 2 for read and 3 for both. The size in there is the struct's,
41+;; which is why the layouts below are checked rather than assumed.
42+
43+(def ^:const VIDIOC_QUERYCAP 2154321408)
44+(def ^:const VIDIOC_S_FMT 3234878981)
45+(def ^:const VIDIOC_REQBUFS 3222558216)
46+(def ^:const VIDIOC_QUERYBUF 3227014665)
47+(def ^:const VIDIOC_QBUF 3227014671)
48+(def ^:const VIDIOC_DQBUF 3227014673)
49+(def ^:const VIDIOC_STREAMON 1074026002)
50+(def ^:const VIDIOC_STREAMOFF 1074026003)
51+
52+(def ^:const buf-type-video-capture 1)
53+(def ^:const memory-mmap 1)
54+(def ^:const field-none 1)
55+(def ^:const cap-video-capture 1)
56+(def ^:const cap-streaming 67108864)
57+
58+(def pixel-formats
59+ "V4L2 fourccs, as the kernel packs them."
60+ {:yuyv 1448695129 :mjpeg 1196444237 :yuv420 842093913})
61+
62+;; --- the structs -------------------------------------------------------------
63+;; Padded to the kernel's sizes rather than described field by field: what
64+;; matters is the total size (it is in the request number) and the offsets of
65+;; the fields actually read. A union is spelled as the reserved block it
66+;; occupies, which is what `v4l2_format` mostly is.
67+
68+(def capability
69+ (ffi/layout [:struct [[:driver [:array :uint8 16]]
70+ [:card [:array :uint8 32]]
71+ [:bus-info [:array :uint8 32]]
72+ [:version :uint32]
73+ [:capabilities :uint32]
74+ [:device-caps :uint32]
75+ [:reserved [:array :uint32 3]]]]))
76+
77+(def format-pix
78+ ;; v4l2_format is 208 bytes: a type, four bytes of padding, then a union
79+ ;; whose largest member decides the rest. Only the pix arm is described;
80+ ;; the tail is the union's remaining bytes.
81+ (ffi/layout [:struct [[:type :uint32]
82+ [:pad :uint32]
83+ [:width :uint32]
84+ [:height :uint32]
85+ [:pixelformat :uint32]
86+ [:field :uint32]
87+ [:bytesperline :uint32]
88+ [:sizeimage :uint32]
89+ [:colorspace :uint32]
90+ [:priv :uint32]
91+ [:flags :uint32]
92+ [:enc :uint32]
93+ [:quantization :uint32]
94+ [:xfer-func :uint32]
95+ [:rest [:array :uint8 152]]]]))
96+
97+(def requestbuffers
98+ (ffi/layout [:struct [[:count :uint32]
99+ [:type :uint32]
100+ [:memory :uint32]
101+ [:capabilities :uint32]
102+ [:flags :uint8]
103+ [:reserved [:array :uint8 3]]]]))
104+
105+(def buffer
106+ ;; 88 bytes. `timestamp` is a struct timeval at 24, `m` is a union at 64
107+ ;; whose first member is the mmap offset, and `memory` sits at 60.
108+ (ffi/layout [:struct [[:index :uint32]
109+ [:type :uint32]
110+ [:bytesused :uint32]
111+ [:flags :uint32]
112+ [:field :uint32]
113+ [:pad0 :uint32]
114+ [:tv-sec :int64]
115+ [:tv-usec :int64]
116+ [:timecode [:array :uint8 16]]
117+ [:sequence :uint32]
118+ [:memory :uint32]
119+ [:offset :uint32]
120+ [:pad1 :uint32]
121+ [:length :uint32]
122+ [:reserved2 :uint32]
123+ [:request-fd :int32]
124+ [:pad2 :uint32]]]))
125+
126+;; --- opening -----------------------------------------------------------------
127+
128+(defn- ioctl! [fd req p what]
129+ (let [rc (c-ioctl fd req p)]
130+ (when (neg? rc)
131+ (throw (ex-info (str "v4l2: " what " failed") {:errno (ffi/errno) :op what})))
132+ rc))
133+
134+(defn open-device
135+ "Open a camera and answer its fd."
136+ [path]
137+ (let [fd (c-open path o-rdwr)]
138+ (when (neg? fd)
139+ (throw (ex-info (str "v4l2: cannot open " path) {:errno (ffi/errno) :path path})))
140+ fd))
141+
142+(defn capabilities
143+ "What the device says it can do. `:capture?` and `:streaming?` are the two
144+ that decide whether the rest of this namespace applies to it."
145+ [fd]
146+ (ffi/with-arena [a]
147+ (let [p (ffi/alloc a (ffi/layout-size capability))]
148+ (ioctl! fd VIDIOC_QUERYCAP p "QUERYCAP")
149+ (let [caps (ffi/read-field p capability [:capabilities])
150+ dev (ffi/read-field p capability [:device-caps])
151+ ;; device_caps describes THIS node; capabilities describes the
152+ ;; whole device, which on a multi-node camera is not the same
153+ ;; thing and is the usual reason a /dev/video1 refuses to stream.
154+ eff (if (zero? dev) caps dev)]
155+ {:capabilities caps
156+ :device-caps dev
157+ :capture? (pos? (bit-and eff cap-video-capture))
158+ :streaming? (pos? (bit-and eff cap-streaming))}))))
159+
160+(defn set-format!
161+ "Ask for a size and pixel format; answers what the driver actually chose.
162+
163+ V4L2 negotiates rather than obeys — a driver may answer a different size or
164+ a different format entirely, and the returned map is the truth."
165+ [fd width height pixel-format]
166+ (ffi/with-arena [a]
167+ (let [p (ffi/alloc a (ffi/layout-size format-pix))
168+ fourcc (or (pixel-formats pixel-format) pixel-format)]
169+ (ffi/write p format-pix {:type buf-type-video-capture :pad 0
170+ :width width :height height
171+ :pixelformat fourcc :field field-none
172+ :bytesperline 0 :sizeimage 0 :colorspace 0
173+ :priv 0 :flags 0 :enc 0 :quantization 0
174+ :xfer-func 0 :rest (vec (repeat 152 0))})
175+ (ioctl! fd VIDIOC_S_FMT p "S_FMT")
176+ {:width (ffi/read-field p format-pix [:width])
177+ :height (ffi/read-field p format-pix [:height])
178+ :pixelformat (ffi/read-field p format-pix [:pixelformat])
179+ :bytesperline (ffi/read-field p format-pix [:bytesperline])
180+ :sizeimage (ffi/read-field p format-pix [:sizeimage])})))
181+
182+;; --- buffers -----------------------------------------------------------------
183+
184+(defn request-buffers!
185+ "Ask the driver for `n` mmap buffers; answers how many it granted."
186+ [fd n]
187+ (ffi/with-arena [a]
188+ (let [p (ffi/alloc a (ffi/layout-size requestbuffers))]
189+ (ffi/write p requestbuffers {:count n :type buf-type-video-capture
190+ :memory memory-mmap :capabilities 0
191+ :flags 0 :reserved [0 0 0]})
192+ (ioctl! fd VIDIOC_REQBUFS p "REQBUFS")
193+ (ffi/read-field p requestbuffers [:count]))))
194+
195+(defn- blank-buffer [p index]
196+ (ffi/write p buffer {:index index :type buf-type-video-capture :bytesused 0
197+ :flags 0 :field 0 :pad0 0 :tv-sec 0 :tv-usec 0
198+ :timecode (vec (repeat 16 0)) :sequence 0
199+ :memory memory-mmap :offset 0 :pad1 0 :length 0
200+ :reserved2 0 :request-fd 0 :pad2 0}))
201+
202+(defn map-buffers!
203+ "QUERYBUF then mmap each buffer; answers a vector of {:ptr :len :index}."
204+ [fd n]
205+ (ffi/with-arena [a]
206+ (let [p (ffi/alloc a (ffi/layout-size buffer))]
207+ (mapv (fn [i]
208+ (blank-buffer p i)
209+ (ioctl! fd VIDIOC_QUERYBUF p "QUERYBUF")
210+ (let [len (ffi/read-field p buffer [:length])
211+ off (ffi/read-field p buffer [:offset])
212+ ptr (c-mmap ffi/null len (bit-or prot-read prot-write)
213+ map-shared fd off)]
214+ (when (= ptr -1)
215+ (throw (ex-info "v4l2: mmap failed" {:errno (ffi/errno) :index i})))
216+ {:index i :ptr ptr :len len}))
217+ (range n)))))
218+
219+(defn queue!
220+ "Hand a buffer back to the driver."
221+ [fd index]
222+ (ffi/with-arena [a]
223+ (let [p (ffi/alloc a (ffi/layout-size buffer))]
224+ (blank-buffer p index)
225+ (ioctl! fd VIDIOC_QBUF p "QBUF")))
226+ nil)
227+
228+(defn stream-on! [fd]
229+ (ffi/with-arena [a]
230+ (let [t (ffi/alloc a 4)]
231+ (ffi/write t :uint32 buf-type-video-capture)
232+ (ioctl! fd VIDIOC_STREAMON t "STREAMON")))
233+ nil)
234+
235+(defn stream-off! [fd]
236+ (ffi/with-arena [a]
237+ (let [t (ffi/alloc a 4)]
238+ (ffi/write t :uint32 buf-type-video-capture)
239+ (ioctl! fd VIDIOC_STREAMOFF t "STREAMOFF")))
240+ nil)
241+
242+(defn with-frame
243+ "Dequeue a frame, hand it to `f` as [pointer length], and requeue it.
244+
245+ The pointer is the driver's mmap'd buffer, valid only until the requeue —
246+ which is why the buffer goes back in a `finally` and why `f` is called
247+ rather than the span being answered. Copying it here would be the one copy
248+ frq.av exists to avoid."
249+ [fd buffers f]
250+ (ffi/with-arena [a]
251+ (let [p (ffi/alloc a (ffi/layout-size buffer))]
252+ (blank-buffer p 0)
253+ (ioctl! fd VIDIOC_DQBUF p "DQBUF")
254+ (let [i (ffi/read-field p buffer [:index])
255+ n (ffi/read-field p buffer [:bytesused])
256+ buf (nth buffers i)]
257+ (try
258+ (f (:ptr buf) n)
259+ (finally (queue! fd i)))))))
260+
261+(defn close-device! [fd buffers]
262+ (doseq [{:keys [ptr len]} buffers] (c-munmap ptr len))
263+ (c-close fd)
264+ nil)
added src/frq/codec/h264.clj +84 -0
new file mode 100644
@@ -0,0 +1,84 @@
1+(ns frq.codec.h264
2+ "H.264 encoding, through openh264.
3+
4+ Not through openh264 DIRECTLY, and the reason is a calling convention.
5+ openh264's C API is not flat: `ISVCEncoder` is `const ISVCEncoderVtbl*`, so
6+ every method — Initialize, EncodeFrame, Uninitialize — is a function pointer
7+ read out of a table hanging off the object. jolt.ffi cannot call one. Chez
8+ fixes a foreign procedure's argument and result types when it COMPILES it,
9+ which is also why the target has to be a literal C symbol name rather than
10+ an address; `jolt/ffi.clj` says so in as many words.
11+
12+ So `c/frq_h264.c` walks the vtable and exports five plain symbols, and this
13+ namespace binds those. The shim is a hundred lines that change a calling
14+ convention; it holds no policy and makes no decisions that belong here.
15+
16+ It does flatten the output, because that part cannot sensibly live in jolt:
17+ openh264 answers an `SFrameBSInfo` of layers, each with its own NAL count
18+ over a shared buffer, and walking that from here would mean reading nested
19+ C structs whose layout is openh264's business rather than ours.
20+
21+ A FRAME IS BORROWED, both ways. `encode!` takes a pointer to I420 and
22+ answers a span into the encoder's own buffer, valid until the next call on
23+ the same encoder. That is `frq.av`'s existing contract for video — decoder
24+ buffer to texture as a pointer, never copied on this side — and it is why
25+ nothing here turns a picture into a jolt value."
26+ (:require [jolt.ffi :as ffi]))
27+
28+(ffi/defcfn raw-open "frq_h264_open" [:int :int :int :int :pointer] :int)
29+(ffi/defcfn raw-encode "frq_h264_encode"
30+ [:pointer :pointer :int64 :pointer :pointer :pointer] :int)
31+(ffi/defcfn raw-force-keyframe "frq_h264_force_keyframe" [:pointer] :int)
32+(ffi/defcfn raw-close "frq_h264_close" [:pointer] :void)
33+
34+(defn i420-size
35+ "Bytes in one I420 frame: a luma plane, then two at quarter resolution."
36+ [width height]
37+ (+ (* width height) (* 2 (quot (* width height) 4))))
38+
39+(defn encoder
40+ "Open an encoder. `bitrate` is bits per second.
41+
42+ openh264 validates here rather than at the first frame, so an impossible
43+ size or bitrate raises now."
44+ [{:keys [width height fps bitrate] :or {fps 30 bitrate 1000000}}]
45+ (ffi/with-arena [a]
46+ (let [out (ffi/alloc a 8)
47+ rc (raw-open width height fps bitrate out)]
48+ (when-not (zero? rc)
49+ (throw (ex-info "openh264: could not open an encoder"
50+ {:code rc :width width :height height
51+ :fps fps :bitrate bitrate})))
52+ (let [h (ffi/read out :pointer)]
53+ (when (ffi/null? h)
54+ (throw (ex-info "openh264: encoder handle is NULL" {})))
55+ h))))
56+
57+(defn encode!
58+ "Encode one I420 frame and hand the result to `use-frame`.
59+
60+ `i420` is a pointer to `(i420-size w h)` bytes. `use-frame` is called with
61+ [pointer length keyframe?] and its value is answered; the span is the
62+ encoder's own buffer and is valid only for the duration of that call.
63+
64+ A frame openh264 chose to skip calls `use-frame` with a zero length rather
65+ than raising — a skip is a decision, not a failure."
66+ [enc i420 pts-us use-frame]
67+ (ffi/with-arena [a]
68+ (let [out (ffi/alloc a 8)
69+ len (ffi/alloc a 4)
70+ key (ffi/alloc a 4)
71+ rc (raw-encode enc i420 pts-us out len key)]
72+ (when-not (zero? rc)
73+ (throw (ex-info "openh264: encode failed" {:code rc})))
74+ (use-frame (ffi/read out :pointer)
75+ (ffi/read len :int32)
76+ (not (zero? (ffi/read key :int32)))))))
77+
78+(defn force-keyframe!
79+ "Make the next frame an IDR — what a newly arrived subscriber needs."
80+ [enc]
81+ (raw-force-keyframe enc)
82+ nil)
83+
84+(defn close! [enc] (raw-close enc) nil)
new file mode 100644
@@ -0,0 +1,84 @@
1+(ns frq.codec.h264
2+ "H.264 encoding, through openh264.
3+
4+ Not through openh264 DIRECTLY, and the reason is a calling convention.
5+ openh264's C API is not flat: `ISVCEncoder` is `const ISVCEncoderVtbl*`, so
6+ every method — Initialize, EncodeFrame, Uninitialize — is a function pointer
7+ read out of a table hanging off the object. jolt.ffi cannot call one. Chez
8+ fixes a foreign procedure's argument and result types when it COMPILES it,
9+ which is also why the target has to be a literal C symbol name rather than
10+ an address; `jolt/ffi.clj` says so in as many words.
11+
12+ So `c/frq_h264.c` walks the vtable and exports five plain symbols, and this
13+ namespace binds those. The shim is a hundred lines that change a calling
14+ convention; it holds no policy and makes no decisions that belong here.
15+
16+ It does flatten the output, because that part cannot sensibly live in jolt:
17+ openh264 answers an `SFrameBSInfo` of layers, each with its own NAL count
18+ over a shared buffer, and walking that from here would mean reading nested
19+ C structs whose layout is openh264's business rather than ours.
20+
21+ A FRAME IS BORROWED, both ways. `encode!` takes a pointer to I420 and
22+ answers a span into the encoder's own buffer, valid until the next call on
23+ the same encoder. That is `frq.av`'s existing contract for video — decoder
24+ buffer to texture as a pointer, never copied on this side — and it is why
25+ nothing here turns a picture into a jolt value."
26+ (:require [jolt.ffi :as ffi]))
27+
28+(ffi/defcfn raw-open "frq_h264_open" [:int :int :int :int :pointer] :int)
29+(ffi/defcfn raw-encode "frq_h264_encode"
30+ [:pointer :pointer :int64 :pointer :pointer :pointer] :int)
31+(ffi/defcfn raw-force-keyframe "frq_h264_force_keyframe" [:pointer] :int)
32+(ffi/defcfn raw-close "frq_h264_close" [:pointer] :void)
33+
34+(defn i420-size
35+ "Bytes in one I420 frame: a luma plane, then two at quarter resolution."
36+ [width height]
37+ (+ (* width height) (* 2 (quot (* width height) 4))))
38+
39+(defn encoder
40+ "Open an encoder. `bitrate` is bits per second.
41+
42+ openh264 validates here rather than at the first frame, so an impossible
43+ size or bitrate raises now."
44+ [{:keys [width height fps bitrate] :or {fps 30 bitrate 1000000}}]
45+ (ffi/with-arena [a]
46+ (let [out (ffi/alloc a 8)
47+ rc (raw-open width height fps bitrate out)]
48+ (when-not (zero? rc)
49+ (throw (ex-info "openh264: could not open an encoder"
50+ {:code rc :width width :height height
51+ :fps fps :bitrate bitrate})))
52+ (let [h (ffi/read out :pointer)]
53+ (when (ffi/null? h)
54+ (throw (ex-info "openh264: encoder handle is NULL" {})))
55+ h))))
56+
57+(defn encode!
58+ "Encode one I420 frame and hand the result to `use-frame`.
59+
60+ `i420` is a pointer to `(i420-size w h)` bytes. `use-frame` is called with
61+ [pointer length keyframe?] and its value is answered; the span is the
62+ encoder's own buffer and is valid only for the duration of that call.
63+
64+ A frame openh264 chose to skip calls `use-frame` with a zero length rather
65+ than raising — a skip is a decision, not a failure."
66+ [enc i420 pts-us use-frame]
67+ (ffi/with-arena [a]
68+ (let [out (ffi/alloc a 8)
69+ len (ffi/alloc a 4)
70+ key (ffi/alloc a 4)
71+ rc (raw-encode enc i420 pts-us out len key)]
72+ (when-not (zero? rc)
73+ (throw (ex-info "openh264: encode failed" {:code rc})))
74+ (use-frame (ffi/read out :pointer)
75+ (ffi/read len :int32)
76+ (not (zero? (ffi/read key :int32)))))))
77+
78+(defn force-keyframe!
79+ "Make the next frame an IDR — what a newly arrived subscriber needs."
80+ [enc]
81+ (raw-force-keyframe enc)
82+ nil)
83+
84+(defn close! [enc] (raw-close enc) nil)
added src/frq/codec/opus.clj +144 -0
new file mode 100644
@@ -0,0 +1,144 @@
1+(ns frq.codec.opus
2+ "Opus, bound to libopus.
3+
4+ This is what the media plane's audio half becomes on this side of the port.
5+ libmoq_ffi carries MoQ over QUIC and nothing else — moq-ffi's `audio`
6+ feature, which would have brought Opus with it, costs a 1062-crate build —
7+ so the codec is linked where it has always lived, in a C library with a flat
8+ API and a twenty-year-old ABI.
9+
10+ Two things about that API shape the binding.
11+
12+ **PCM is int16, interleaved, and never a jolt value.** `encode!` and
13+ `decode!` take and answer [pointer length] spans of foreign memory, the same
14+ as `frq.moq.media`'s frame path. A 20ms stereo frame at 48kHz is 1920
15+ samples; turning that into a jolt vector twice per frame, fifty times a
16+ second, is work with nothing to show for it, and `frq.av`'s rule already
17+ says audio and video move as pointers.
18+
19+ **A frame is not any length you like.** Opus encodes exactly 2.5, 5, 10, 20,
20+ 40 or 60ms, and `frame-size` is a count of samples PER CHANNEL, not bytes
21+ and not interleaved samples. Handing it the interleaved count is the classic
22+ mistake: at stereo it asks for twice the duration, which is a valid frame
23+ size, so nothing raises and the audio simply runs fast."
24+ (:require [jolt.ffi :as ffi]))
25+
26+;; --- constants ---------------------------------------------------------------
27+
28+(def ^:const ok 0)
29+
30+(def applications
31+ "What the encoder is being asked to optimise for. :voip is what a call
32+ wants — it favours speech intelligibility over musical fidelity."
33+ {:voip 2048 :audio 2049 :low-delay 2051})
34+
35+(def ^:private errors
36+ {0 :ok -1 :bad-arg -2 :buffer-too-small -3 :internal-error
37+ -4 :invalid-packet -5 :unimplemented -6 :invalid-state -7 :alloc-fail})
38+
39+(def ^:const set-bitrate-request 4002)
40+
41+;; Sample counts PER CHANNEL for one frame at 48kHz, by frame duration.
42+(def frame-samples-48k
43+ {2.5 120, 5 240, 10 480, 20 960, 40 1920, 60 2880})
44+
45+;; --- the entry points --------------------------------------------------------
46+
47+(ffi/defcfn raw-encoder-create "opus_encoder_create"
48+ [:int32 :int :int :pointer] :pointer)
49+(ffi/defcfn raw-encoder-destroy "opus_encoder_destroy" [:pointer] :void)
50+(ffi/defcfn raw-encode "opus_encode"
51+ [:pointer :pointer :int :pointer :int32] :int32)
52+;; Variadic: the CTL request decides the tail. Declared with the one tail
53+;; shape this namespace uses — an int32 — rather than a bare :&, so it costs
54+;; no compile at the first call.
55+(ffi/defcfn raw-encoder-ctl "opus_encoder_ctl" [:pointer :int :& :int32] :int)
56+
57+(ffi/defcfn raw-decoder-create "opus_decoder_create" [:int32 :int :pointer] :pointer)
58+(ffi/defcfn raw-decoder-destroy "opus_decoder_destroy" [:pointer] :void)
59+(ffi/defcfn raw-decode "opus_decode"
60+ [:pointer :pointer :int32 :pointer :int :int] :int)
61+
62+(ffi/defcfn raw-strerror "opus_strerror" [:int] :string)
63+
64+;; --- errors ------------------------------------------------------------------
65+
66+(defn- check!
67+ "libopus answers a negative int for every failure, in every function that
68+ returns one. There is no errno and no out-parameter to consult except on the
69+ constructors, so this is the whole error protocol."
70+ [n what]
71+ (if (neg? n)
72+ (throw (ex-info (str "opus: " what ": " (raw-strerror n))
73+ {:code n :error (errors n :unknown) :op what}))
74+ n))
75+
76+;; --- encoding ----------------------------------------------------------------
77+
78+(defn encoder
79+ "An Opus encoder. `sample-rate` is one of 8000, 12000, 16000, 24000, 48000."
80+ ([] (encoder 48000 1 :voip))
81+ ([sample-rate channels application]
82+ (ffi/with-arena [a]
83+ (let [err (ffi/alloc a 4)
84+ enc (raw-encoder-create sample-rate channels
85+ (or (applications application)
86+ (throw (ex-info "unknown opus application"
87+ {:got application
88+ :known (keys applications)})))
89+ err)]
90+ (check! (ffi/read err :int32) "encoder_create")
91+ (when (ffi/null? enc)
92+ (throw (ex-info "opus: encoder_create answered NULL" {})))
93+ enc))))
94+
95+(defn set-bitrate!
96+ "Bits per second across all channels."
97+ [enc bps]
98+ (check! (raw-encoder-ctl enc set-bitrate-request bps) "set_bitrate")
99+ nil)
100+
101+(defn encode!
102+ "Encode one frame; answers the number of bytes written into `out`.
103+
104+ `pcm` is a pointer to interleaved int16 samples and `samples-per-channel`
105+ counts them PER CHANNEL — see the namespace docstring on why that
106+ distinction bites silently rather than loudly.
107+
108+ A return of 2 bytes or fewer is not an error: it is DTX, the encoder saying
109+ this frame is silence and need not be sent at all."
110+ [enc pcm samples-per-channel out out-capacity]
111+ (check! (raw-encode enc pcm samples-per-channel out out-capacity) "encode"))
112+
113+(defn dtx?
114+ "Did `encode!` decide the frame was not worth sending?"
115+ [written]
116+ (<= written 2))
117+
118+(defn free-encoder! [enc] (raw-encoder-destroy enc) nil)
119+
120+;; --- decoding ----------------------------------------------------------------
121+
122+(defn decoder
123+ ([] (decoder 48000 1))
124+ ([sample-rate channels]
125+ (ffi/with-arena [a]
126+ (let [err (ffi/alloc a 4)
127+ dec (raw-decoder-create sample-rate channels err)]
128+ (check! (ffi/read err :int32) "decoder_create")
129+ (when (ffi/null? dec)
130+ (throw (ex-info "opus: decoder_create answered NULL" {})))
131+ dec))))
132+
133+(defn decode!
134+ "Decode one packet into `pcm`; answers samples decoded PER CHANNEL.
135+
136+ `capacity-per-channel` is how much room `pcm` has, again per channel. Pass a
137+ nil packet to conceal a lost one — that is what Opus's PLC is, and it is why
138+ `data` is allowed to be NULL where most C APIs would refuse."
139+ [dec data len pcm capacity-per-channel]
140+ (check! (raw-decode dec (or data ffi/null) (if data len 0)
141+ pcm capacity-per-channel 0)
142+ "decode"))
143+
144+(defn free-decoder! [dec] (raw-decoder-destroy dec) nil)
new file mode 100644
@@ -0,0 +1,144 @@
1+(ns frq.codec.opus
2+ "Opus, bound to libopus.
3+
4+ This is what the media plane's audio half becomes on this side of the port.
5+ libmoq_ffi carries MoQ over QUIC and nothing else — moq-ffi's `audio`
6+ feature, which would have brought Opus with it, costs a 1062-crate build —
7+ so the codec is linked where it has always lived, in a C library with a flat
8+ API and a twenty-year-old ABI.
9+
10+ Two things about that API shape the binding.
11+
12+ **PCM is int16, interleaved, and never a jolt value.** `encode!` and
13+ `decode!` take and answer [pointer length] spans of foreign memory, the same
14+ as `frq.moq.media`'s frame path. A 20ms stereo frame at 48kHz is 1920
15+ samples; turning that into a jolt vector twice per frame, fifty times a
16+ second, is work with nothing to show for it, and `frq.av`'s rule already
17+ says audio and video move as pointers.
18+
19+ **A frame is not any length you like.** Opus encodes exactly 2.5, 5, 10, 20,
20+ 40 or 60ms, and `frame-size` is a count of samples PER CHANNEL, not bytes
21+ and not interleaved samples. Handing it the interleaved count is the classic
22+ mistake: at stereo it asks for twice the duration, which is a valid frame
23+ size, so nothing raises and the audio simply runs fast."
24+ (:require [jolt.ffi :as ffi]))
25+
26+;; --- constants ---------------------------------------------------------------
27+
28+(def ^:const ok 0)
29+
30+(def applications
31+ "What the encoder is being asked to optimise for. :voip is what a call
32+ wants — it favours speech intelligibility over musical fidelity."
33+ {:voip 2048 :audio 2049 :low-delay 2051})
34+
35+(def ^:private errors
36+ {0 :ok -1 :bad-arg -2 :buffer-too-small -3 :internal-error
37+ -4 :invalid-packet -5 :unimplemented -6 :invalid-state -7 :alloc-fail})
38+
39+(def ^:const set-bitrate-request 4002)
40+
41+;; Sample counts PER CHANNEL for one frame at 48kHz, by frame duration.
42+(def frame-samples-48k
43+ {2.5 120, 5 240, 10 480, 20 960, 40 1920, 60 2880})
44+
45+;; --- the entry points --------------------------------------------------------
46+
47+(ffi/defcfn raw-encoder-create "opus_encoder_create"
48+ [:int32 :int :int :pointer] :pointer)
49+(ffi/defcfn raw-encoder-destroy "opus_encoder_destroy" [:pointer] :void)
50+(ffi/defcfn raw-encode "opus_encode"
51+ [:pointer :pointer :int :pointer :int32] :int32)
52+;; Variadic: the CTL request decides the tail. Declared with the one tail
53+;; shape this namespace uses — an int32 — rather than a bare :&, so it costs
54+;; no compile at the first call.
55+(ffi/defcfn raw-encoder-ctl "opus_encoder_ctl" [:pointer :int :& :int32] :int)
56+
57+(ffi/defcfn raw-decoder-create "opus_decoder_create" [:int32 :int :pointer] :pointer)
58+(ffi/defcfn raw-decoder-destroy "opus_decoder_destroy" [:pointer] :void)
59+(ffi/defcfn raw-decode "opus_decode"
60+ [:pointer :pointer :int32 :pointer :int :int] :int)
61+
62+(ffi/defcfn raw-strerror "opus_strerror" [:int] :string)
63+
64+;; --- errors ------------------------------------------------------------------
65+
66+(defn- check!
67+ "libopus answers a negative int for every failure, in every function that
68+ returns one. There is no errno and no out-parameter to consult except on the
69+ constructors, so this is the whole error protocol."
70+ [n what]
71+ (if (neg? n)
72+ (throw (ex-info (str "opus: " what ": " (raw-strerror n))
73+ {:code n :error (errors n :unknown) :op what}))
74+ n))
75+
76+;; --- encoding ----------------------------------------------------------------
77+
78+(defn encoder
79+ "An Opus encoder. `sample-rate` is one of 8000, 12000, 16000, 24000, 48000."
80+ ([] (encoder 48000 1 :voip))
81+ ([sample-rate channels application]
82+ (ffi/with-arena [a]
83+ (let [err (ffi/alloc a 4)
84+ enc (raw-encoder-create sample-rate channels
85+ (or (applications application)
86+ (throw (ex-info "unknown opus application"
87+ {:got application
88+ :known (keys applications)})))
89+ err)]
90+ (check! (ffi/read err :int32) "encoder_create")
91+ (when (ffi/null? enc)
92+ (throw (ex-info "opus: encoder_create answered NULL" {})))
93+ enc))))
94+
95+(defn set-bitrate!
96+ "Bits per second across all channels."
97+ [enc bps]
98+ (check! (raw-encoder-ctl enc set-bitrate-request bps) "set_bitrate")
99+ nil)
100+
101+(defn encode!
102+ "Encode one frame; answers the number of bytes written into `out`.
103+
104+ `pcm` is a pointer to interleaved int16 samples and `samples-per-channel`
105+ counts them PER CHANNEL — see the namespace docstring on why that
106+ distinction bites silently rather than loudly.
107+
108+ A return of 2 bytes or fewer is not an error: it is DTX, the encoder saying
109+ this frame is silence and need not be sent at all."
110+ [enc pcm samples-per-channel out out-capacity]
111+ (check! (raw-encode enc pcm samples-per-channel out out-capacity) "encode"))
112+
113+(defn dtx?
114+ "Did `encode!` decide the frame was not worth sending?"
115+ [written]
116+ (<= written 2))
117+
118+(defn free-encoder! [enc] (raw-encoder-destroy enc) nil)
119+
120+;; --- decoding ----------------------------------------------------------------
121+
122+(defn decoder
123+ ([] (decoder 48000 1))
124+ ([sample-rate channels]
125+ (ffi/with-arena [a]
126+ (let [err (ffi/alloc a 4)
127+ dec (raw-decoder-create sample-rate channels err)]
128+ (check! (ffi/read err :int32) "decoder_create")
129+ (when (ffi/null? dec)
130+ (throw (ex-info "opus: decoder_create answered NULL" {})))
131+ dec))))
132+
133+(defn decode!
134+ "Decode one packet into `pcm`; answers samples decoded PER CHANNEL.
135+
136+ `capacity-per-channel` is how much room `pcm` has, again per channel. Pass a
137+ nil packet to conceal a lost one — that is what Opus's PLC is, and it is why
138+ `data` is allowed to be NULL where most C APIs would refuse."
139+ [dec data len pcm capacity-per-channel]
140+ (check! (raw-decode dec (or data ffi/null) (if data len 0)
141+ pcm capacity-per-channel 0)
142+ "decode"))
143+
144+(defn free-decoder! [dec] (raw-decoder-destroy dec) nil)
modified src/frq/moq/smoke.clj +179 -1
@@ -36,6 +36,10 @@
3636 [frq.moq.raw :as raw]
3737 [frq.moq.client :as client]
3838 [frq.moq.media :as media]
39+ [frq.codec.opus :as opus]
40+ [frq.codec.h264 :as h264]
41+ [frq.capture.v4l2 :as v4l2]
42+ [frq.capture.alsa :as alsa]
3943 [jolt.ffi :as ffi]))
4044
4145 (defn- check-contract []
@@ -194,13 +198,187 @@
194198 (throw (ex-info "timestamp did not survive" {:frame frame})))
195199 true))))
196200
201+(defn- triangle-pcm
202+ "20ms of a triangle wave at 48kHz mono, as int16 in FOREIGN memory.
203+
204+ A triangle rather than a sine because it needs no transcendental and no
205+ Math namespace, and rather than silence because silence is the one input
206+ Opus is entitled to throw away: DTX would answer two bytes and the round
207+ trip would prove nothing.
208+
209+ Answers [pointer samples-per-channel]."
210+ [a]
211+ (let [n 960 ; 48000 * 0.020
212+ p (ffi/alloc a (* 2 n))]
213+ (dotimes [i n]
214+ ;; A 100-sample period ramp, +/- 8000 — loud enough that the decoder
215+ ;; cannot answer silence and have it look like success.
216+ (let [phase (mod i 100)
217+ v (if (< phase 50) (- (* phase 320) 8000) (- 8000 (* (- phase 50) 320)))]
218+ (ffi/write (+ p (* 2 i)) :int16 v)))
219+ [p n]))
220+
221+(defn- check-opus
222+ "Encode a frame with libopus and decode it back.
223+
224+ This is the first piece of the port that is a C library rather than
225+ somebody's binding to one, and the assertions are chosen to fail if it is
226+ only pretending to work. The encoded frame must be more than DTX's two
227+ bytes; the decode must answer exactly the frame size it was given; and the
228+ decoded audio must carry real amplitude, because a decoder that returned
229+ the right COUNT of zeroes would otherwise pass."
230+ []
231+ (ffi/with-arena [a]
232+ (let [enc (opus/encoder 48000 1 :voip)
233+ dec (opus/decoder 48000 1)]
234+ (try
235+ (opus/set-bitrate! enc 24000)
236+ (let [[pcm n] (triangle-pcm a)
237+ cap 4000
238+ out (ffi/alloc a cap)
239+ written (opus/encode! enc pcm n out cap)]
240+ (println " encoded" n "samples ->" written "bytes")
241+ (when (opus/dtx? written)
242+ (throw (ex-info "encoder answered DTX for a loud frame"
243+ {:written written})))
244+ (let [back (ffi/alloc a (* 2 n))
245+ decoded (opus/decode! dec out written back n)
246+ peak (reduce (fn [m i]
247+ (max m (abs (ffi/read (+ back (* 2 i)) :int16))))
248+ 0 (range decoded))]
249+ (println " decoded" decoded "samples, peak" peak)
250+ (when-not (= n decoded)
251+ (throw (ex-info "decoder answered a different frame size"
252+ {:sent n :got decoded})))
253+ (when (< peak 1000)
254+ (throw (ex-info "decoded audio is silent — a lossy codec is not this lossy"
255+ {:peak peak})))
256+ true))
257+ (finally
258+ (opus/free-encoder! enc)
259+ (opus/free-decoder! dec))))))
260+
261+(defn- check-h264
262+ "Encode an I420 frame to H.264 through the openh264 shim.
263+
264+ The assertion is the Annex B start code. A frame that came back with a
265+ plausible length and the wrong bytes would be a vtable walked incorrectly —
266+ the shim reading the wrong slot, or flattening the layers wrong — and a
267+ length check alone would not catch it. 00 00 00 01 at the front, and an
268+ IDR for the first frame, is openh264 having actually encoded something.
269+
270+ The picture is flat gray, which compresses to almost nothing; what is under
271+ test is the calling convention, not the encoder."
272+ []
273+ (ffi/with-arena [a]
274+ (let [w 64 h 64
275+ n (h264/i420-size w h)
276+ px (ffi/alloc a n)
277+ enc (h264/encoder {:width w :height h :fps 30 :bitrate 200000})]
278+ (dotimes [i n] (ffi/write (+ px i) :uint8 0x80))
279+ (try
280+ (let [got (h264/encode! enc px 0
281+ (fn [p len key?]
282+ ;; The span is the encoder's buffer and dies
283+ ;; at the next encode: read what is needed
284+ ;; here and let it go.
285+ {:len len
286+ :keyframe key?
287+ :first-4 (mapv #(ffi/read (+ p %) :uint8)
288+ (range (min 4 len)))}))]
289+ (println " encoded" n "bytes of I420 ->" (:len got)
290+ "bytes, keyframe" (:keyframe got)
291+ "starts" (pr-str (:first-4 got)))
292+ (when (zero? (:len got))
293+ (throw (ex-info "encoder skipped the first frame" {})))
294+ (when-not (= [0 0 0 1] (:first-4 got))
295+ (throw (ex-info "not Annex B — no start code" {:first-4 (:first-4 got)})))
296+ (when-not (:keyframe got)
297+ (throw (ex-info "first frame is not an IDR" {})))
298+ true)
299+ (finally (h264/close! enc))))))
300+
301+(defn- check-v4l2-layouts
302+ "Check every V4L2 struct against what a C compiler says.
303+
304+ There is no camera in this container, so the capture path itself cannot be
305+ exercised here. What CAN be checked is the part most likely to be wrong and
306+ least likely to announce it: every VIDIOC_ request number encodes the size
307+ of the struct it carries, so a layout one byte off does not misread a field
308+ — it produces a request the kernel has never heard of, and the driver
309+ answers ENOTTY for an ioctl that plainly exists.
310+
311+ The numbers on the right came from a C program compiled against this
312+ kernel's own headers (scratch/v4l2probe.c). They are the ground truth this
313+ namespace is transcribed from, so checking against them catches a typo
314+ rather than a misunderstanding — the misunderstanding needs a device."
315+ []
316+ (let [checks [["v4l2_capability size" (ffi/layout-size v4l2/capability) 104]
317+ [" capabilities@" (ffi/field-offset v4l2/capability [:capabilities]) 84]
318+ [" device-caps@" (ffi/field-offset v4l2/capability [:device-caps]) 88]
319+ ["v4l2_format size" (ffi/layout-size v4l2/format-pix) 208]
320+ [" width@" (ffi/field-offset v4l2/format-pix [:width]) 8]
321+ [" height@" (ffi/field-offset v4l2/format-pix [:height]) 12]
322+ [" pixelformat@" (ffi/field-offset v4l2/format-pix [:pixelformat]) 16]
323+ [" sizeimage@" (ffi/field-offset v4l2/format-pix [:sizeimage]) 28]
324+ ["v4l2_requestbuffers size" (ffi/layout-size v4l2/requestbuffers) 20]
325+ ["v4l2_buffer size" (ffi/layout-size v4l2/buffer) 88]
326+ [" bytesused@" (ffi/field-offset v4l2/buffer [:bytesused]) 8]
327+ [" timestamp@" (ffi/field-offset v4l2/buffer [:tv-sec]) 24]
328+ [" memory@" (ffi/field-offset v4l2/buffer [:memory]) 60]
329+ [" m.offset@" (ffi/field-offset v4l2/buffer [:offset]) 64]
330+ [" length@" (ffi/field-offset v4l2/buffer [:length]) 72]]
331+ bad (remove (fn [[_ got want]] (= got want)) checks)]
332+ (doseq [[what got want] checks]
333+ (println (str " " what " " got (when-not (= got want) (str " WANT " want)))))
334+ (when (seq bad)
335+ (throw (ex-info "V4L2 layouts do not match the kernel headers"
336+ {:mismatched (mapv (fn [[w g want]] {:what w :got g :want want}) bad)})))
337+ (println " (no camera here — the capture path itself is unexercised)")
338+ true))
339+
340+(defn- check-alsa
341+ "Open a PCM and push frames through it.
342+
343+ The device is ALSA's `null` PCM, which swallows everything and is always
344+ present — no hardware, no permissions, and nothing that depends on what
345+ this container happens to have plugged in. What that proves is the binding:
346+ the library loads, the handle out-parameter comes back, set_params accepts
347+ the format, and writei answers in FRAMES. What it cannot prove is that a
348+ real card behaves, which needs a real card.
349+
350+ The frame count is the assertion. 960 frames of mono S16 is 1920 bytes, and
351+ a writei that answered 1920 would be this code having confused the two —
352+ the mistake the namespace docstring warns about, and the one that looks
353+ like a slow device rather than a bug."
354+ []
355+ (ffi/with-arena [a]
356+ (let [frames 960
357+ buf (ffi/alloc a (* 2 frames))
358+ pcm (alsa/open-pcm "null" :playback {:rate 48000 :channels 1})]
359+ (try
360+ (dotimes [i frames] (ffi/write (+ buf (* 2 i)) :int16 0))
361+ (let [{:keys [frames written recovered]} (alsa/write! pcm buf frames)
362+ n (or frames written)]
363+ (println (str " null pcm: wrote " n " frames"
364+ (when recovered " (recovered from an overrun)")))
365+ (when-not (= 960 n)
366+ (throw (ex-info "writei answered something other than the frame count"
367+ {:asked 960 :got n})))
368+ true)
369+ (finally (alsa/close! pcm))))))
370+
197371 (defn -main [& _]
198372 (println "libmoq_ffi smoke test")
199373 (let [steps [["contract" check-contract]
200374 ["handle" check-handle]
201375 ["string" check-string]
202376 ["connect" check-connect]
203- ["media" check-media]]]
377+ ["media" check-media]
378+ ["opus" check-opus]
379+ ["h264" check-h264]
380+ ["v4l2" check-v4l2-layouts]
381+ ["alsa" check-alsa]]]
204382 (doseq [[name f] steps]
205383 (println (str name ":"))
206384 (f))
@@ -36,6 +36,10 @@
36 [frq.moq.raw :as raw]36 [frq.moq.raw :as raw]
37 [frq.moq.client :as client]37 [frq.moq.client :as client]
38 [frq.moq.media :as media]38 [frq.moq.media :as media]
39+ [frq.codec.opus :as opus]
40+ [frq.codec.h264 :as h264]
41+ [frq.capture.v4l2 :as v4l2]
42+ [frq.capture.alsa :as alsa]
39 [jolt.ffi :as ffi]))43 [jolt.ffi :as ffi]))
40 44
41 (defn- check-contract []45 (defn- check-contract []
@@ -194,13 +198,187 @@
194 (throw (ex-info "timestamp did not survive" {:frame frame})))198 (throw (ex-info "timestamp did not survive" {:frame frame})))
195 true))))199 true))))
196 200
201+(defn- triangle-pcm
202+ "20ms of a triangle wave at 48kHz mono, as int16 in FOREIGN memory.
203+
204+ A triangle rather than a sine because it needs no transcendental and no
205+ Math namespace, and rather than silence because silence is the one input
206+ Opus is entitled to throw away: DTX would answer two bytes and the round
207+ trip would prove nothing.
208+
209+ Answers [pointer samples-per-channel]."
210+ [a]
211+ (let [n 960 ; 48000 * 0.020
212+ p (ffi/alloc a (* 2 n))]
213+ (dotimes [i n]
214+ ;; A 100-sample period ramp, +/- 8000 — loud enough that the decoder
215+ ;; cannot answer silence and have it look like success.
216+ (let [phase (mod i 100)
217+ v (if (< phase 50) (- (* phase 320) 8000) (- 8000 (* (- phase 50) 320)))]
218+ (ffi/write (+ p (* 2 i)) :int16 v)))
219+ [p n]))
220+
221+(defn- check-opus
222+ "Encode a frame with libopus and decode it back.
223+
224+ This is the first piece of the port that is a C library rather than
225+ somebody's binding to one, and the assertions are chosen to fail if it is
226+ only pretending to work. The encoded frame must be more than DTX's two
227+ bytes; the decode must answer exactly the frame size it was given; and the
228+ decoded audio must carry real amplitude, because a decoder that returned
229+ the right COUNT of zeroes would otherwise pass."
230+ []
231+ (ffi/with-arena [a]
232+ (let [enc (opus/encoder 48000 1 :voip)
233+ dec (opus/decoder 48000 1)]
234+ (try
235+ (opus/set-bitrate! enc 24000)
236+ (let [[pcm n] (triangle-pcm a)
237+ cap 4000
238+ out (ffi/alloc a cap)
239+ written (opus/encode! enc pcm n out cap)]
240+ (println " encoded" n "samples ->" written "bytes")
241+ (when (opus/dtx? written)
242+ (throw (ex-info "encoder answered DTX for a loud frame"
243+ {:written written})))
244+ (let [back (ffi/alloc a (* 2 n))
245+ decoded (opus/decode! dec out written back n)
246+ peak (reduce (fn [m i]
247+ (max m (abs (ffi/read (+ back (* 2 i)) :int16))))
248+ 0 (range decoded))]
249+ (println " decoded" decoded "samples, peak" peak)
250+ (when-not (= n decoded)
251+ (throw (ex-info "decoder answered a different frame size"
252+ {:sent n :got decoded})))
253+ (when (< peak 1000)
254+ (throw (ex-info "decoded audio is silent — a lossy codec is not this lossy"
255+ {:peak peak})))
256+ true))
257+ (finally
258+ (opus/free-encoder! enc)
259+ (opus/free-decoder! dec))))))
260+
261+(defn- check-h264
262+ "Encode an I420 frame to H.264 through the openh264 shim.
263+
264+ The assertion is the Annex B start code. A frame that came back with a
265+ plausible length and the wrong bytes would be a vtable walked incorrectly —
266+ the shim reading the wrong slot, or flattening the layers wrong — and a
267+ length check alone would not catch it. 00 00 00 01 at the front, and an
268+ IDR for the first frame, is openh264 having actually encoded something.
269+
270+ The picture is flat gray, which compresses to almost nothing; what is under
271+ test is the calling convention, not the encoder."
272+ []
273+ (ffi/with-arena [a]
274+ (let [w 64 h 64
275+ n (h264/i420-size w h)
276+ px (ffi/alloc a n)
277+ enc (h264/encoder {:width w :height h :fps 30 :bitrate 200000})]
278+ (dotimes [i n] (ffi/write (+ px i) :uint8 0x80))
279+ (try
280+ (let [got (h264/encode! enc px 0
281+ (fn [p len key?]
282+ ;; The span is the encoder's buffer and dies
283+ ;; at the next encode: read what is needed
284+ ;; here and let it go.
285+ {:len len
286+ :keyframe key?
287+ :first-4 (mapv #(ffi/read (+ p %) :uint8)
288+ (range (min 4 len)))}))]
289+ (println " encoded" n "bytes of I420 ->" (:len got)
290+ "bytes, keyframe" (:keyframe got)
291+ "starts" (pr-str (:first-4 got)))
292+ (when (zero? (:len got))
293+ (throw (ex-info "encoder skipped the first frame" {})))
294+ (when-not (= [0 0 0 1] (:first-4 got))
295+ (throw (ex-info "not Annex B — no start code" {:first-4 (:first-4 got)})))
296+ (when-not (:keyframe got)
297+ (throw (ex-info "first frame is not an IDR" {})))
298+ true)
299+ (finally (h264/close! enc))))))
300+
301+(defn- check-v4l2-layouts
302+ "Check every V4L2 struct against what a C compiler says.
303+
304+ There is no camera in this container, so the capture path itself cannot be
305+ exercised here. What CAN be checked is the part most likely to be wrong and
306+ least likely to announce it: every VIDIOC_ request number encodes the size
307+ of the struct it carries, so a layout one byte off does not misread a field
308+ — it produces a request the kernel has never heard of, and the driver
309+ answers ENOTTY for an ioctl that plainly exists.
310+
311+ The numbers on the right came from a C program compiled against this
312+ kernel's own headers (scratch/v4l2probe.c). They are the ground truth this
313+ namespace is transcribed from, so checking against them catches a typo
314+ rather than a misunderstanding — the misunderstanding needs a device."
315+ []
316+ (let [checks [["v4l2_capability size" (ffi/layout-size v4l2/capability) 104]
317+ [" capabilities@" (ffi/field-offset v4l2/capability [:capabilities]) 84]
318+ [" device-caps@" (ffi/field-offset v4l2/capability [:device-caps]) 88]
319+ ["v4l2_format size" (ffi/layout-size v4l2/format-pix) 208]
320+ [" width@" (ffi/field-offset v4l2/format-pix [:width]) 8]
321+ [" height@" (ffi/field-offset v4l2/format-pix [:height]) 12]
322+ [" pixelformat@" (ffi/field-offset v4l2/format-pix [:pixelformat]) 16]
323+ [" sizeimage@" (ffi/field-offset v4l2/format-pix [:sizeimage]) 28]
324+ ["v4l2_requestbuffers size" (ffi/layout-size v4l2/requestbuffers) 20]
325+ ["v4l2_buffer size" (ffi/layout-size v4l2/buffer) 88]
326+ [" bytesused@" (ffi/field-offset v4l2/buffer [:bytesused]) 8]
327+ [" timestamp@" (ffi/field-offset v4l2/buffer [:tv-sec]) 24]
328+ [" memory@" (ffi/field-offset v4l2/buffer [:memory]) 60]
329+ [" m.offset@" (ffi/field-offset v4l2/buffer [:offset]) 64]
330+ [" length@" (ffi/field-offset v4l2/buffer [:length]) 72]]
331+ bad (remove (fn [[_ got want]] (= got want)) checks)]
332+ (doseq [[what got want] checks]
333+ (println (str " " what " " got (when-not (= got want) (str " WANT " want)))))
334+ (when (seq bad)
335+ (throw (ex-info "V4L2 layouts do not match the kernel headers"
336+ {:mismatched (mapv (fn [[w g want]] {:what w :got g :want want}) bad)})))
337+ (println " (no camera here — the capture path itself is unexercised)")
338+ true))
339+
340+(defn- check-alsa
341+ "Open a PCM and push frames through it.
342+
343+ The device is ALSA's `null` PCM, which swallows everything and is always
344+ present — no hardware, no permissions, and nothing that depends on what
345+ this container happens to have plugged in. What that proves is the binding:
346+ the library loads, the handle out-parameter comes back, set_params accepts
347+ the format, and writei answers in FRAMES. What it cannot prove is that a
348+ real card behaves, which needs a real card.
349+
350+ The frame count is the assertion. 960 frames of mono S16 is 1920 bytes, and
351+ a writei that answered 1920 would be this code having confused the two —
352+ the mistake the namespace docstring warns about, and the one that looks
353+ like a slow device rather than a bug."
354+ []
355+ (ffi/with-arena [a]
356+ (let [frames 960
357+ buf (ffi/alloc a (* 2 frames))
358+ pcm (alsa/open-pcm "null" :playback {:rate 48000 :channels 1})]
359+ (try
360+ (dotimes [i frames] (ffi/write (+ buf (* 2 i)) :int16 0))
361+ (let [{:keys [frames written recovered]} (alsa/write! pcm buf frames)
362+ n (or frames written)]
363+ (println (str " null pcm: wrote " n " frames"
364+ (when recovered " (recovered from an overrun)")))
365+ (when-not (= 960 n)
366+ (throw (ex-info "writei answered something other than the frame count"
367+ {:asked 960 :got n})))
368+ true)
369+ (finally (alsa/close! pcm))))))
370+
197 (defn -main [& _]371 (defn -main [& _]
198 (println "libmoq_ffi smoke test")372 (println "libmoq_ffi smoke test")
199 (let [steps [["contract" check-contract]373 (let [steps [["contract" check-contract]
200 ["handle" check-handle]374 ["handle" check-handle]
201 ["string" check-string]375 ["string" check-string]
202 ["connect" check-connect]376 ["connect" check-connect]
203- ["media" check-media]]]377+ ["media" check-media]
378+ ["opus" check-opus]
379+ ["h264" check-h264]
380+ ["v4l2" check-v4l2-layouts]
381+ ["alsa" check-alsa]]]
204 (doseq [[name f] steps]382 (doseq [[name f] steps]
205 (println (str name ":"))383 (println (str name ":"))
206 (f))384 (f))