Let the plane take real devices, not only test thunks
frq.capture.source is where a camera and a microphone become the [pointer length] thunks frq.av.plane already takes. Keeping it a separate namespace is what lets the plane run against a webcam and against a test pattern without a line of it changing, and it is how Camera2 will arrive on the phone without the plane learning about JNI. Both are NON-BLOCKING, because the plane is pumped from glimmer's loop thread. V4L2 opens O_NONBLOCK and treats EAGAIN as the ordinary answer rather than a failure -- on a pumped caller it is what "no frame yet" looks like, and a blocking DQBUF would hold the UI for a frame interval every pump, or for ever on a camera that stopped. A short ALSA read answers nil rather than a partial frame: Opus encodes whole frames and padding a short one with silence puts a click in the audio every time the device is behind. A webcam almost never gives I420, so frq_yuyv_to_i420 joins the shim. The chroma is averaged down each row pair rather than line-dropped; dropping is a line cheaper and shows as combing on a hard colour edge, which a red shirt against a pale wall finds immediately. V4L2 NEGOTIATES, so `camera` answers the size the driver actually chose and the encoder is opened from that rather than from what was asked for. The flake gains PipeWire's ALSA plugin. Without it alsa-lib cannot dlopen libasound_module_pcm_pipewire.so and `default` resolves to nothing -- the only devices that open are raw hardware ones, which PipeWire is already holding. That was a real deployment gap, found by trying it: an environment variable rather than a library in the join, because alsa-lib looks plugins up by directory. WHAT IS VERIFIED AND WHAT IS NOT. The mic and speaker are exercised through the real binding against ALSA's `null`, twice over, so the wiring, the frame arithmetic and the teardown are checked -- a leak would surface as `Device or resource busy` on the second round. Real hardware is not: it is held by PipeWire on the host and PipeWire's socket is not reachable from this container. The camera is not exercised at all -- there is no /dev/video* and no privilege to load the kernel's virtual one -- and the test says so rather than implying otherwise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
596dc29 parent: f417261 modified
c/frq_h264.c +43 -0 | @@ -244,3 +244,46 @@ void frq_h264_decoder_close(void *handle) { | ||
| 244 | 244 | free(d->rgba); |
| 245 | 245 | free(d); |
| 246 | 246 | } |
| 247 | + | |
| 248 | +/* --- YUYV to I420 --------------------------------------------------------- | |
| 249 | + * | |
| 250 | + * A webcam almost never hands you I420. YUYV (4:2:2 packed) is the format | |
| 251 | + * every UVC device supports, and openh264 wants I420 (4:2:0 planar), so | |
| 252 | + * something has to transpose and subsample between them. Doing it in jolt | |
| 253 | + * would be a per-pixel loop through ffi/read and ffi/write at thirty frames | |
| 254 | + * a second; doing it here is one pass over the row pairs. | |
| 255 | + * | |
| 256 | + * The chroma is AVERAGED down the row pair rather than dropped. Taking every | |
| 257 | + * other line instead is a line cheaper and shows up as combing on anything | |
| 258 | + * with a hard colour edge — a red shirt against a pale wall is the usual | |
| 259 | + * way to see it. | |
| 260 | + * | |
| 261 | + * `src` is width*height*2 bytes; `dst` is width*height*3/2. Both even | |
| 262 | + * dimensions, which V4L2 will have negotiated anyway. | |
| 263 | + */ | |
| 264 | +void frq_yuyv_to_i420(const unsigned char *src, unsigned char *dst, | |
| 265 | + int width, int height) { | |
| 266 | + int x, y; | |
| 267 | + unsigned char *Y = dst; | |
| 268 | + unsigned char *U = dst + width * height; | |
| 269 | + unsigned char *V = U + (width / 2) * (height / 2); | |
| 270 | + | |
| 271 | + for (y = 0; y < height; y++) { | |
| 272 | + const unsigned char *row = src + (size_t)y * width * 2; | |
| 273 | + unsigned char *yr = Y + (size_t)y * width; | |
| 274 | + for (x = 0; x < width; x++) yr[x] = row[x * 2]; | |
| 275 | + } | |
| 276 | + for (y = 0; y < height; y += 2) { | |
| 277 | + const unsigned char *r0 = src + (size_t)y * width * 2; | |
| 278 | + const unsigned char *r1 = src + (size_t)(y + 1) * width * 2; | |
| 279 | + unsigned char *ur = U + (size_t)(y / 2) * (width / 2); | |
| 280 | + unsigned char *vr = V + (size_t)(y / 2) * (width / 2); | |
| 281 | + for (x = 0; x < width; x += 2) { | |
| 282 | + /* One U and one V per two pixels per row; averaged over the pair. */ | |
| 283 | + int u = (r0[x * 2 + 1] + r1[x * 2 + 1] + 1) >> 1; | |
| 284 | + int v = (r0[x * 2 + 3] + r1[x * 2 + 3] + 1) >> 1; | |
| 285 | + ur[x / 2] = (unsigned char)u; | |
| 286 | + vr[x / 2] = (unsigned char)v; | |
| 287 | + } | |
| 288 | + } | |
| 289 | +} | |
| @@ -244,3 +244,46 @@ void frq_h264_decoder_close(void *handle) { | |||
| 244 | free(d->rgba); | 244 | free(d->rgba); |
| 245 | free(d); | 245 | free(d); |
| 246 | } | 246 | } |
| 247 | + | ||
| 248 | +/* --- YUYV to I420 --------------------------------------------------------- | ||
| 249 | + * | ||
| 250 | + * A webcam almost never hands you I420. YUYV (4:2:2 packed) is the format | ||
| 251 | + * every UVC device supports, and openh264 wants I420 (4:2:0 planar), so | ||
| 252 | + * something has to transpose and subsample between them. Doing it in jolt | ||
| 253 | + * would be a per-pixel loop through ffi/read and ffi/write at thirty frames | ||
| 254 | + * a second; doing it here is one pass over the row pairs. | ||
| 255 | + * | ||
| 256 | + * The chroma is AVERAGED down the row pair rather than dropped. Taking every | ||
| 257 | + * other line instead is a line cheaper and shows up as combing on anything | ||
| 258 | + * with a hard colour edge — a red shirt against a pale wall is the usual | ||
| 259 | + * way to see it. | ||
| 260 | + * | ||
| 261 | + * `src` is width*height*2 bytes; `dst` is width*height*3/2. Both even | ||
| 262 | + * dimensions, which V4L2 will have negotiated anyway. | ||
| 263 | + */ | ||
| 264 | +void frq_yuyv_to_i420(const unsigned char *src, unsigned char *dst, | ||
| 265 | + int width, int height) { | ||
| 266 | + int x, y; | ||
| 267 | + unsigned char *Y = dst; | ||
| 268 | + unsigned char *U = dst + width * height; | ||
| 269 | + unsigned char *V = U + (width / 2) * (height / 2); | ||
| 270 | + | ||
| 271 | + for (y = 0; y < height; y++) { | ||
| 272 | + const unsigned char *row = src + (size_t)y * width * 2; | ||
| 273 | + unsigned char *yr = Y + (size_t)y * width; | ||
| 274 | + for (x = 0; x < width; x++) yr[x] = row[x * 2]; | ||
| 275 | + } | ||
| 276 | + for (y = 0; y < height; y += 2) { | ||
| 277 | + const unsigned char *r0 = src + (size_t)y * width * 2; | ||
| 278 | + const unsigned char *r1 = src + (size_t)(y + 1) * width * 2; | ||
| 279 | + unsigned char *ur = U + (size_t)(y / 2) * (width / 2); | ||
| 280 | + unsigned char *vr = V + (size_t)(y / 2) * (width / 2); | ||
| 281 | + for (x = 0; x < width; x += 2) { | ||
| 282 | + /* One U and one V per two pixels per row; averaged over the pair. */ | ||
| 283 | + int u = (r0[x * 2 + 1] + r1[x * 2 + 1] + 1) >> 1; | ||
| 284 | + int v = (r0[x * 2 + 3] + r1[x * 2 + 3] + 1) >> 1; | ||
| 285 | + ur[x / 2] = (unsigned char)u; | ||
| 286 | + vr[x / 2] = (unsigned char)v; | ||
| 287 | + } | ||
| 288 | + } | ||
| 289 | +} | ||
modified
flake.nix +16 -0 | @@ -279,6 +279,16 @@ | ||
| 279 | 279 | # the kernel, so there is no library to name. |
| 280 | 280 | codecs = [ pkgs.libopus pkgs.openh264 frqH264 pkgs.alsa-lib ]; |
| 281 | 281 | |
| 282 | + # ALSA's PipeWire plugin, which is how `default` resolves to | |
| 283 | + # anything on a machine running PipeWire — and every machine frq | |
| 284 | + # targets does. Without it alsa-lib fails to dlopen | |
| 285 | + # libasound_module_pcm_pipewire.so and the only devices that open | |
| 286 | + # are raw hardware ones, which PipeWire is already holding. | |
| 287 | + # | |
| 288 | + # An environment variable rather than a library in the join: | |
| 289 | + # alsa-lib looks plugins up by directory, not by soname. | |
| 290 | + alsaPluginDir = "${pkgs.pipewire}/lib/alsa-lib"; | |
| 291 | + | |
| 282 | 292 | nativeAll = pkgs.symlinkJoin { |
| 283 | 293 | name = "frq-native"; |
| 284 | 294 | paths = [ native moqFfi ] ++ codecs; |
| @@ -356,6 +366,7 @@ | ||
| 356 | 366 | # the only cost is re-resolving the (already local) graph per start. |
| 357 | 367 | frqScript = pkgs.writeShellScript "frq" '' |
| 358 | 368 | export LD_LIBRARY_PATH="${nativeAll}/lib:${lib.makeLibraryPath runtimeLibs}''${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" |
| 369 | + export ALSA_PLUGIN_DIR="${alsaPluginDir}" | |
| 359 | 370 | cd ${frqSource} |
| 360 | 371 | |
| 361 | 372 | # On NixOS the store's Mesa is the system's and the window opens. |
| @@ -374,6 +385,7 @@ | ||
| 374 | 385 | # a terminal, which is the reason this output exists. |
| 375 | 386 | tuiScript = pkgs.writeShellScript "frq-tui" '' |
| 376 | 387 | export LD_LIBRARY_PATH="${nativeAll}/lib''${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" |
| 388 | + export ALSA_PLUGIN_DIR="${alsaPluginDir}" | |
| 377 | 389 | cd ${frqSource} |
| 378 | 390 | |
| 379 | 391 | exec ${joltRuntime}/bin/jolt \ |
| @@ -437,6 +449,7 @@ | ||
| 437 | 449 | in |
| 438 | 450 | { |
| 439 | 451 | inherit native moqFfi frqH264 nativeAll frq; |
| 452 | + inherit (pkgs) pipewire; | |
| 440 | 453 | inherit tui; |
| 441 | 454 | jolt = joltRuntime; |
| 442 | 455 | default = frq; |
| @@ -500,6 +513,9 @@ | ||
| 500 | 513 | # to live in that tree and the shell has to hand it its answers. |
| 501 | 514 | # Naming these is also what makes the shell build them. |
| 502 | 515 | JOLT_NATIVE_LIB = "${nativeAll}/lib"; |
| 516 | + # Spelled out rather than shared with the packages block, which | |
| 517 | + # is a different `let`. See `alsaPluginDir` there for why. | |
| 518 | + ALSA_PLUGIN_DIR = "${pkgs.pipewire}/lib/alsa-lib"; | |
| 503 | 519 | GLIMMER_SRC = glimmer; |
| 504 | 520 | GLIMMER_VIDYA_SRC = "${jolt-native}/glimmer-backends/glimmer-vidya"; |
| 505 | 521 | GLIMMER_TUI_SRC = "${jolt-native}/glimmer-backends/glimmer-tui"; |
| @@ -279,6 +279,16 @@ | |||
| 279 | # the kernel, so there is no library to name. | 279 | # the kernel, so there is no library to name. |
| 280 | codecs = [ pkgs.libopus pkgs.openh264 frqH264 pkgs.alsa-lib ]; | 280 | codecs = [ pkgs.libopus pkgs.openh264 frqH264 pkgs.alsa-lib ]; |
| 281 | 281 | ||
| 282 | + # ALSA's PipeWire plugin, which is how `default` resolves to | ||
| 283 | + # anything on a machine running PipeWire — and every machine frq | ||
| 284 | + # targets does. Without it alsa-lib fails to dlopen | ||
| 285 | + # libasound_module_pcm_pipewire.so and the only devices that open | ||
| 286 | + # are raw hardware ones, which PipeWire is already holding. | ||
| 287 | + # | ||
| 288 | + # An environment variable rather than a library in the join: | ||
| 289 | + # alsa-lib looks plugins up by directory, not by soname. | ||
| 290 | + alsaPluginDir = "${pkgs.pipewire}/lib/alsa-lib"; | ||
| 291 | + | ||
| 282 | nativeAll = pkgs.symlinkJoin { | 292 | nativeAll = pkgs.symlinkJoin { |
| 283 | name = "frq-native"; | 293 | name = "frq-native"; |
| 284 | paths = [ native moqFfi ] ++ codecs; | 294 | paths = [ native moqFfi ] ++ codecs; |
| @@ -356,6 +366,7 @@ | |||
| 356 | # the only cost is re-resolving the (already local) graph per start. | 366 | # the only cost is re-resolving the (already local) graph per start. |
| 357 | frqScript = pkgs.writeShellScript "frq" '' | 367 | frqScript = pkgs.writeShellScript "frq" '' |
| 358 | export LD_LIBRARY_PATH="${nativeAll}/lib:${lib.makeLibraryPath runtimeLibs}''${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" | 368 | export LD_LIBRARY_PATH="${nativeAll}/lib:${lib.makeLibraryPath runtimeLibs}''${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" |
| 369 | + export ALSA_PLUGIN_DIR="${alsaPluginDir}" | ||
| 359 | cd ${frqSource} | 370 | cd ${frqSource} |
| 360 | 371 | ||
| 361 | # On NixOS the store's Mesa is the system's and the window opens. | 372 | # On NixOS the store's Mesa is the system's and the window opens. |
| @@ -374,6 +385,7 @@ | |||
| 374 | # a terminal, which is the reason this output exists. | 385 | # a terminal, which is the reason this output exists. |
| 375 | tuiScript = pkgs.writeShellScript "frq-tui" '' | 386 | tuiScript = pkgs.writeShellScript "frq-tui" '' |
| 376 | export LD_LIBRARY_PATH="${nativeAll}/lib''${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" | 387 | export LD_LIBRARY_PATH="${nativeAll}/lib''${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" |
| 388 | + export ALSA_PLUGIN_DIR="${alsaPluginDir}" | ||
| 377 | cd ${frqSource} | 389 | cd ${frqSource} |
| 378 | 390 | ||
| 379 | exec ${joltRuntime}/bin/jolt \ | 391 | exec ${joltRuntime}/bin/jolt \ |
| @@ -437,6 +449,7 @@ | |||
| 437 | in | 449 | in |
| 438 | { | 450 | { |
| 439 | inherit native moqFfi frqH264 nativeAll frq; | 451 | inherit native moqFfi frqH264 nativeAll frq; |
| 452 | + inherit (pkgs) pipewire; | ||
| 440 | inherit tui; | 453 | inherit tui; |
| 441 | jolt = joltRuntime; | 454 | jolt = joltRuntime; |
| 442 | default = frq; | 455 | default = frq; |
| @@ -500,6 +513,9 @@ | |||
| 500 | # to live in that tree and the shell has to hand it its answers. | 513 | # to live in that tree and the shell has to hand it its answers. |
| 501 | # Naming these is also what makes the shell build them. | 514 | # Naming these is also what makes the shell build them. |
| 502 | JOLT_NATIVE_LIB = "${nativeAll}/lib"; | 515 | JOLT_NATIVE_LIB = "${nativeAll}/lib"; |
| 516 | + # Spelled out rather than shared with the packages block, which | ||
| 517 | + # is a different `let`. See `alsaPluginDir` there for why. | ||
| 518 | + ALSA_PLUGIN_DIR = "${pkgs.pipewire}/lib/alsa-lib"; | ||
| 503 | GLIMMER_SRC = glimmer; | 519 | GLIMMER_SRC = glimmer; |
| 504 | GLIMMER_VIDYA_SRC = "${jolt-native}/glimmer-backends/glimmer-vidya"; | 520 | GLIMMER_VIDYA_SRC = "${jolt-native}/glimmer-backends/glimmer-vidya"; |
| 505 | GLIMMER_TUI_SRC = "${jolt-native}/glimmer-backends/glimmer-tui"; | 521 | GLIMMER_TUI_SRC = "${jolt-native}/glimmer-backends/glimmer-tui"; |
modified
src/frq/av/plane.clj +33 -7 | @@ -53,6 +53,7 @@ | ||
| 53 | 53 | [frq.moq.uniffi :as uniffi] |
| 54 | 54 | [frq.codec.h264 :as h264] |
| 55 | 55 | [frq.codec.opus :as opus] |
| 56 | + [frq.capture.source :as source] | |
| 56 | 57 | [jolt.ffi :as ffi])) |
| 57 | 58 | |
| 58 | 59 | ;; --- state ------------------------------------------------------------------- |
| @@ -77,12 +78,27 @@ | ||
| 77 | 78 | Everything that can fail does so HERE rather than at the first frame: the |
| 78 | 79 | encoder validates its size, the decoder opens, and the subscribe settles, |
| 79 | 80 | so a plane that comes up is one that can carry a picture." |
| 80 | - [{:keys [origin discover session path source mic width height fps bitrate | |
| 81 | - camera? muted? channels] | |
| 81 | + [{:keys [origin discover session path source mic speaker | |
| 82 | + camera-device mic-device speaker-device | |
| 83 | + width height fps bitrate camera? muted? channels] | |
| 82 | 84 | :or {path "/frq" width 640 height 480 fps 30 bitrate 800000 |
| 83 | 85 | camera? true muted? false channels 1}}] |
| 84 | 86 | (stop!) |
| 85 | - (let [broadcast (media/create-broadcast! origin path) | |
| 87 | + ;; A device NAME builds the thunk; a thunk passed directly wins. That | |
| 88 | + ;; ordering is what lets the same plane run against a camera and against | |
| 89 | + ;; a test pattern without knowing which it has. | |
| 90 | + (let [cam (when (and camera-device (nil? source)) | |
| 91 | + (source/camera camera-device {:width width :height height})) | |
| 92 | + width (or (:width cam) width) | |
| 93 | + height (or (:height cam) height) | |
| 94 | + source (or source (:source cam)) | |
| 95 | + micdev (when (and mic-device (nil? mic)) | |
| 96 | + (source/microphone mic-device {:channels channels})) | |
| 97 | + mic (or mic (:mic micdev)) | |
| 98 | + spk (or speaker | |
| 99 | + (when speaker-device | |
| 100 | + (source/speaker speaker-device {:channels channels}))) | |
| 101 | + broadcast (media/create-broadcast! origin path) | |
| 86 | 102 | producer (media/publish-media! broadcast "avc3") |
| 87 | 103 | track (media/producer-name producer) |
| 88 | 104 | consumer (media/broadcast-consumer broadcast) |
| @@ -121,6 +137,8 @@ | ||
| 121 | 137 | :mic-producer mic-producer |
| 122 | 138 | :mic-encoder (when mic (opus/encoder audio/sample-rate channels :voip)) |
| 123 | 139 | :mic mic |
| 140 | + :speaker spk | |
| 141 | + :closers (into [] (keep :close!) [cam micdev spk]) | |
| 124 | 142 | :muted? muted? |
| 125 | 143 | :channels channels |
| 126 | 144 | :mix (ffi/alloc (* 2 audio/frame-samples channels)) |
| @@ -150,6 +168,9 @@ | ||
| 150 | 168 | ;; worker there is no reactor to do it on. |
| 151 | 169 | (when-let [sess (:session p)] |
| 152 | 170 | (try (client/shutdown! sess) (catch Exception _ nil))) |
| 171 | + ;; Devices last: the encoder and the rings may still be reading from | |
| 172 | + ;; buffers these own. | |
| 173 | + (doseq [close! (:closers p)] (try (close!) (catch Exception _ nil))) | |
| 153 | 174 | (reset! plane nil)) |
| 154 | 175 | nil) |
| 155 | 176 | |
| @@ -407,10 +428,15 @@ | ||
| 407 | 428 | (:peers p')) |
| 408 | 429 | peak (when (seq rings) |
| 409 | 430 | (audio/mix-into! rings (:mix p') (:channels p')))] |
| 410 | - (reset! plane (assoc p' :mixed (when peak | |
| 411 | - {:ptr (:mix p') | |
| 412 | - :samples audio/frame-samples | |
| 413 | - :peak peak}))))) | |
| 431 | + (let [mixed (when peak {:ptr (:mix p') | |
| 432 | + :samples audio/frame-samples | |
| 433 | + :peak peak})] | |
| 434 | + ;; Straight out to the speaker if there is one. A caller with its | |
| 435 | + ;; own output — the terminal backend, a test — reads poll-audio! | |
| 436 | + ;; instead and this stays nil. | |
| 437 | + (when (and mixed (:speaker p')) | |
| 438 | + ((:play! (:speaker p')) mixed)) | |
| 439 | + (reset! plane (assoc p' :mixed mixed))))) | |
| 414 | 440 | nil) |
| 415 | 441 | |
| 416 | 442 | (defn poll-frames! |
| @@ -53,6 +53,7 @@ | |||
| 53 | [frq.moq.uniffi :as uniffi] | 53 | [frq.moq.uniffi :as uniffi] |
| 54 | [frq.codec.h264 :as h264] | 54 | [frq.codec.h264 :as h264] |
| 55 | [frq.codec.opus :as opus] | 55 | [frq.codec.opus :as opus] |
| 56 | + [frq.capture.source :as source] | ||
| 56 | [jolt.ffi :as ffi])) | 57 | [jolt.ffi :as ffi])) |
| 57 | 58 | ||
| 58 | ;; --- state ------------------------------------------------------------------- | 59 | ;; --- state ------------------------------------------------------------------- |
| @@ -77,12 +78,27 @@ | |||
| 77 | Everything that can fail does so HERE rather than at the first frame: the | 78 | Everything that can fail does so HERE rather than at the first frame: the |
| 78 | encoder validates its size, the decoder opens, and the subscribe settles, | 79 | encoder validates its size, the decoder opens, and the subscribe settles, |
| 79 | so a plane that comes up is one that can carry a picture." | 80 | so a plane that comes up is one that can carry a picture." |
| 80 | - [{:keys [origin discover session path source mic width height fps bitrate | 81 | + [{:keys [origin discover session path source mic speaker |
| 81 | - camera? muted? channels] | 82 | + camera-device mic-device speaker-device |
| 83 | + width height fps bitrate camera? muted? channels] | ||
| 82 | :or {path "/frq" width 640 height 480 fps 30 bitrate 800000 | 84 | :or {path "/frq" width 640 height 480 fps 30 bitrate 800000 |
| 83 | camera? true muted? false channels 1}}] | 85 | camera? true muted? false channels 1}}] |
| 84 | (stop!) | 86 | (stop!) |
| 85 | - (let [broadcast (media/create-broadcast! origin path) | 87 | + ;; A device NAME builds the thunk; a thunk passed directly wins. That |
| 88 | + ;; ordering is what lets the same plane run against a camera and against | ||
| 89 | + ;; a test pattern without knowing which it has. | ||
| 90 | + (let [cam (when (and camera-device (nil? source)) | ||
| 91 | + (source/camera camera-device {:width width :height height})) | ||
| 92 | + width (or (:width cam) width) | ||
| 93 | + height (or (:height cam) height) | ||
| 94 | + source (or source (:source cam)) | ||
| 95 | + micdev (when (and mic-device (nil? mic)) | ||
| 96 | + (source/microphone mic-device {:channels channels})) | ||
| 97 | + mic (or mic (:mic micdev)) | ||
| 98 | + spk (or speaker | ||
| 99 | + (when speaker-device | ||
| 100 | + (source/speaker speaker-device {:channels channels}))) | ||
| 101 | + broadcast (media/create-broadcast! origin path) | ||
| 86 | producer (media/publish-media! broadcast "avc3") | 102 | producer (media/publish-media! broadcast "avc3") |
| 87 | track (media/producer-name producer) | 103 | track (media/producer-name producer) |
| 88 | consumer (media/broadcast-consumer broadcast) | 104 | consumer (media/broadcast-consumer broadcast) |
| @@ -121,6 +137,8 @@ | |||
| 121 | :mic-producer mic-producer | 137 | :mic-producer mic-producer |
| 122 | :mic-encoder (when mic (opus/encoder audio/sample-rate channels :voip)) | 138 | :mic-encoder (when mic (opus/encoder audio/sample-rate channels :voip)) |
| 123 | :mic mic | 139 | :mic mic |
| 140 | + :speaker spk | ||
| 141 | + :closers (into [] (keep :close!) [cam micdev spk]) | ||
| 124 | :muted? muted? | 142 | :muted? muted? |
| 125 | :channels channels | 143 | :channels channels |
| 126 | :mix (ffi/alloc (* 2 audio/frame-samples channels)) | 144 | :mix (ffi/alloc (* 2 audio/frame-samples channels)) |
| @@ -150,6 +168,9 @@ | |||
| 150 | ;; worker there is no reactor to do it on. | 168 | ;; worker there is no reactor to do it on. |
| 151 | (when-let [sess (:session p)] | 169 | (when-let [sess (:session p)] |
| 152 | (try (client/shutdown! sess) (catch Exception _ nil))) | 170 | (try (client/shutdown! sess) (catch Exception _ nil))) |
| 171 | + ;; Devices last: the encoder and the rings may still be reading from | ||
| 172 | + ;; buffers these own. | ||
| 173 | + (doseq [close! (:closers p)] (try (close!) (catch Exception _ nil))) | ||
| 153 | (reset! plane nil)) | 174 | (reset! plane nil)) |
| 154 | nil) | 175 | nil) |
| 155 | 176 | ||
| @@ -407,10 +428,15 @@ | |||
| 407 | (:peers p')) | 428 | (:peers p')) |
| 408 | peak (when (seq rings) | 429 | peak (when (seq rings) |
| 409 | (audio/mix-into! rings (:mix p') (:channels p')))] | 430 | (audio/mix-into! rings (:mix p') (:channels p')))] |
| 410 | - (reset! plane (assoc p' :mixed (when peak | 431 | + (let [mixed (when peak {:ptr (:mix p') |
| 411 | - {:ptr (:mix p') | 432 | + :samples audio/frame-samples |
| 412 | - :samples audio/frame-samples | 433 | + :peak peak})] |
| 413 | - :peak peak}))))) | 434 | + ;; Straight out to the speaker if there is one. A caller with its |
| 435 | + ;; own output — the terminal backend, a test — reads poll-audio! | ||
| 436 | + ;; instead and this stays nil. | ||
| 437 | + (when (and mixed (:speaker p')) | ||
| 438 | + ((:play! (:speaker p')) mixed)) | ||
| 439 | + (reset! plane (assoc p' :mixed mixed))))) | ||
| 414 | nil) | 440 | nil) |
| 415 | 441 | ||
| 416 | (defn poll-frames! | 442 | (defn poll-frames! |
added
src/frq/capture/source.clj +103 -0 | new file mode 100644 | ||
| @@ -0,0 +1,103 @@ | ||
| 1 | +(ns frq.capture.source | |
| 2 | + "Real devices as the thunks `frq.av.plane` takes. | |
| 3 | + | |
| 4 | + The plane asks for `:source` and `:mic` — functions answering [pointer | |
| 5 | + length] or nil — and does not know or care where the pixels came from. | |
| 6 | + This namespace is where a camera and a microphone become those functions, | |
| 7 | + and keeping it separate is what lets the plane be tested with a synthetic | |
| 8 | + frame and run with a real one without a line of it changing. | |
| 9 | + | |
| 10 | + Both are NON-BLOCKING, because the plane is pumped from glimmer's loop | |
| 11 | + thread. A blocking read here would hold the whole UI for as long as the | |
| 12 | + device felt like taking, and on a device that has stopped producing, for | |
| 13 | + ever. Nothing ready is a nil, not a wait." | |
| 14 | + (:require [frq.capture.alsa :as alsa] | |
| 15 | + [frq.capture.v4l2 :as v4l2] | |
| 16 | + [frq.av.audio :as audio] | |
| 17 | + [jolt.ffi :as ffi])) | |
| 18 | + | |
| 19 | +(ffi/defcfn yuyv->i420 "frq_yuyv_to_i420" [:pointer :pointer :int :int] :void) | |
| 20 | + | |
| 21 | +;; --- the camera -------------------------------------------------------------- | |
| 22 | + | |
| 23 | +(defn camera | |
| 24 | + "Open `path` and answer {:source :close! :width :height}. | |
| 25 | + | |
| 26 | + YUYV is asked for because every UVC camera has it and openh264 wants I420, | |
| 27 | + which is one pass away. MJPEG would be smaller on the wire between camera | |
| 28 | + and kernel but needs a JPEG decoder in front of the converter, and this | |
| 29 | + path already has enough moving parts. | |
| 30 | + | |
| 31 | + V4L2 NEGOTIATES: the size that comes back is not necessarily the size | |
| 32 | + asked for, so the answer carries what the driver actually chose and the | |
| 33 | + encoder should be opened from that rather than from the request." | |
| 34 | + [path {:keys [width height buffers] :or {width 640 height 480 buffers 4}}] | |
| 35 | + (let [fd (v4l2/open-device path) | |
| 36 | + caps (v4l2/capabilities fd)] | |
| 37 | + (when-not (:capture? caps) | |
| 38 | + (v4l2/close-device! fd []) | |
| 39 | + (throw (ex-info "v4l2: not a capture device" {:path path :caps caps}))) | |
| 40 | + (let [fmt (v4l2/set-format! fd width height :yuyv) | |
| 41 | + w (:width fmt) h (:height fmt) | |
| 42 | + n (v4l2/request-buffers! fd buffers) | |
| 43 | + bufs (v4l2/map-buffers! fd n) | |
| 44 | + ;; One I420 frame, allocated once. The converter writes here and | |
| 45 | + ;; the encoder reads here; a fresh allocation per frame would put | |
| 46 | + ;; the allocator in the capture path. | |
| 47 | + i420 (ffi/alloc (+ (* w h) (* 2 (quot (* w h) 4))))] | |
| 48 | + (doseq [{:keys [index]} bufs] (v4l2/queue! fd index)) | |
| 49 | + (v4l2/stream-on! fd) | |
| 50 | + {:width w | |
| 51 | + :height h | |
| 52 | + :source (fn [] | |
| 53 | + (v4l2/try-frame | |
| 54 | + fd bufs | |
| 55 | + (fn [ptr _len] | |
| 56 | + (yuyv->i420 ptr i420 w h) | |
| 57 | + [i420 (+ (* w h) (* 2 (quot (* w h) 4)))]))) | |
| 58 | + :close! (fn [] | |
| 59 | + (try (v4l2/stream-off! fd) (catch Exception _ nil)) | |
| 60 | + (v4l2/close-device! fd bufs) | |
| 61 | + (ffi/free i420))}))) | |
| 62 | + | |
| 63 | +;; --- the microphone ---------------------------------------------------------- | |
| 64 | + | |
| 65 | +(defn microphone | |
| 66 | + "Open an ALSA capture PCM and answer {:mic :close!}. | |
| 67 | + | |
| 68 | + One Opus frame at a time — 20ms, `audio/frame-samples` per channel — | |
| 69 | + because that is the unit the encoder takes and the jitter buffer holds. | |
| 70 | + Reading a different amount would mean carrying a remainder between pumps, | |
| 71 | + which is a buffer this does not need to own. | |
| 72 | + | |
| 73 | + A short read answers nil rather than a partial frame. Opus encodes whole | |
| 74 | + frames, and padding a short one with silence puts a click in the audio | |
| 75 | + every time the device is a little behind." | |
| 76 | + [name {:keys [channels] :or {channels 1}}] | |
| 77 | + (let [pcm (alsa/open-pcm name :capture | |
| 78 | + {:rate audio/sample-rate :channels channels | |
| 79 | + :latency-us 40000}) | |
| 80 | + buf (ffi/alloc (* 2 audio/frame-samples channels))] | |
| 81 | + (alsa/prepare! pcm) | |
| 82 | + {:mic (fn [] | |
| 83 | + (let [{:keys [frames]} (alsa/read! pcm buf audio/frame-samples)] | |
| 84 | + (when (= frames audio/frame-samples) | |
| 85 | + [buf frames]))) | |
| 86 | + :close! (fn [] (alsa/close! pcm) (ffi/free buf))})) | |
| 87 | + | |
| 88 | +;; --- the speaker ------------------------------------------------------------- | |
| 89 | + | |
| 90 | +(defn speaker | |
| 91 | + "Open an ALSA playback PCM and answer {:play! :close!}. | |
| 92 | + | |
| 93 | + `play!` takes what `frq.av.plane/poll-audio!` answers and writes it. It is | |
| 94 | + the one place in this port where a short write is silently fine: ALSA | |
| 95 | + accepting fewer frames than offered means the device's buffer is full, | |
| 96 | + which for playback means we are ahead rather than behind." | |
| 97 | + [name {:keys [channels] :or {channels 1}}] | |
| 98 | + (let [pcm (alsa/open-pcm name :playback | |
| 99 | + {:rate audio/sample-rate :channels channels | |
| 100 | + :latency-us 40000})] | |
| 101 | + {:play! (fn [{:keys [ptr samples]}] | |
| 102 | + (when ptr (alsa/write! pcm ptr samples))) | |
| 103 | + :close! (fn [] (alsa/close! pcm))})) | |
| new file mode 100644 | |||
| @@ -0,0 +1,103 @@ | |||
| 1 | +(ns frq.capture.source | ||
| 2 | + "Real devices as the thunks `frq.av.plane` takes. | ||
| 3 | + | ||
| 4 | + The plane asks for `:source` and `:mic` — functions answering [pointer | ||
| 5 | + length] or nil — and does not know or care where the pixels came from. | ||
| 6 | + This namespace is where a camera and a microphone become those functions, | ||
| 7 | + and keeping it separate is what lets the plane be tested with a synthetic | ||
| 8 | + frame and run with a real one without a line of it changing. | ||
| 9 | + | ||
| 10 | + Both are NON-BLOCKING, because the plane is pumped from glimmer's loop | ||
| 11 | + thread. A blocking read here would hold the whole UI for as long as the | ||
| 12 | + device felt like taking, and on a device that has stopped producing, for | ||
| 13 | + ever. Nothing ready is a nil, not a wait." | ||
| 14 | + (:require [frq.capture.alsa :as alsa] | ||
| 15 | + [frq.capture.v4l2 :as v4l2] | ||
| 16 | + [frq.av.audio :as audio] | ||
| 17 | + [jolt.ffi :as ffi])) | ||
| 18 | + | ||
| 19 | +(ffi/defcfn yuyv->i420 "frq_yuyv_to_i420" [:pointer :pointer :int :int] :void) | ||
| 20 | + | ||
| 21 | +;; --- the camera -------------------------------------------------------------- | ||
| 22 | + | ||
| 23 | +(defn camera | ||
| 24 | + "Open `path` and answer {:source :close! :width :height}. | ||
| 25 | + | ||
| 26 | + YUYV is asked for because every UVC camera has it and openh264 wants I420, | ||
| 27 | + which is one pass away. MJPEG would be smaller on the wire between camera | ||
| 28 | + and kernel but needs a JPEG decoder in front of the converter, and this | ||
| 29 | + path already has enough moving parts. | ||
| 30 | + | ||
| 31 | + V4L2 NEGOTIATES: the size that comes back is not necessarily the size | ||
| 32 | + asked for, so the answer carries what the driver actually chose and the | ||
| 33 | + encoder should be opened from that rather than from the request." | ||
| 34 | + [path {:keys [width height buffers] :or {width 640 height 480 buffers 4}}] | ||
| 35 | + (let [fd (v4l2/open-device path) | ||
| 36 | + caps (v4l2/capabilities fd)] | ||
| 37 | + (when-not (:capture? caps) | ||
| 38 | + (v4l2/close-device! fd []) | ||
| 39 | + (throw (ex-info "v4l2: not a capture device" {:path path :caps caps}))) | ||
| 40 | + (let [fmt (v4l2/set-format! fd width height :yuyv) | ||
| 41 | + w (:width fmt) h (:height fmt) | ||
| 42 | + n (v4l2/request-buffers! fd buffers) | ||
| 43 | + bufs (v4l2/map-buffers! fd n) | ||
| 44 | + ;; One I420 frame, allocated once. The converter writes here and | ||
| 45 | + ;; the encoder reads here; a fresh allocation per frame would put | ||
| 46 | + ;; the allocator in the capture path. | ||
| 47 | + i420 (ffi/alloc (+ (* w h) (* 2 (quot (* w h) 4))))] | ||
| 48 | + (doseq [{:keys [index]} bufs] (v4l2/queue! fd index)) | ||
| 49 | + (v4l2/stream-on! fd) | ||
| 50 | + {:width w | ||
| 51 | + :height h | ||
| 52 | + :source (fn [] | ||
| 53 | + (v4l2/try-frame | ||
| 54 | + fd bufs | ||
| 55 | + (fn [ptr _len] | ||
| 56 | + (yuyv->i420 ptr i420 w h) | ||
| 57 | + [i420 (+ (* w h) (* 2 (quot (* w h) 4)))]))) | ||
| 58 | + :close! (fn [] | ||
| 59 | + (try (v4l2/stream-off! fd) (catch Exception _ nil)) | ||
| 60 | + (v4l2/close-device! fd bufs) | ||
| 61 | + (ffi/free i420))}))) | ||
| 62 | + | ||
| 63 | +;; --- the microphone ---------------------------------------------------------- | ||
| 64 | + | ||
| 65 | +(defn microphone | ||
| 66 | + "Open an ALSA capture PCM and answer {:mic :close!}. | ||
| 67 | + | ||
| 68 | + One Opus frame at a time — 20ms, `audio/frame-samples` per channel — | ||
| 69 | + because that is the unit the encoder takes and the jitter buffer holds. | ||
| 70 | + Reading a different amount would mean carrying a remainder between pumps, | ||
| 71 | + which is a buffer this does not need to own. | ||
| 72 | + | ||
| 73 | + A short read answers nil rather than a partial frame. Opus encodes whole | ||
| 74 | + frames, and padding a short one with silence puts a click in the audio | ||
| 75 | + every time the device is a little behind." | ||
| 76 | + [name {:keys [channels] :or {channels 1}}] | ||
| 77 | + (let [pcm (alsa/open-pcm name :capture | ||
| 78 | + {:rate audio/sample-rate :channels channels | ||
| 79 | + :latency-us 40000}) | ||
| 80 | + buf (ffi/alloc (* 2 audio/frame-samples channels))] | ||
| 81 | + (alsa/prepare! pcm) | ||
| 82 | + {:mic (fn [] | ||
| 83 | + (let [{:keys [frames]} (alsa/read! pcm buf audio/frame-samples)] | ||
| 84 | + (when (= frames audio/frame-samples) | ||
| 85 | + [buf frames]))) | ||
| 86 | + :close! (fn [] (alsa/close! pcm) (ffi/free buf))})) | ||
| 87 | + | ||
| 88 | +;; --- the speaker ------------------------------------------------------------- | ||
| 89 | + | ||
| 90 | +(defn speaker | ||
| 91 | + "Open an ALSA playback PCM and answer {:play! :close!}. | ||
| 92 | + | ||
| 93 | + `play!` takes what `frq.av.plane/poll-audio!` answers and writes it. It is | ||
| 94 | + the one place in this port where a short write is silently fine: ALSA | ||
| 95 | + accepting fewer frames than offered means the device's buffer is full, | ||
| 96 | + which for playback means we are ahead rather than behind." | ||
| 97 | + [name {:keys [channels] :or {channels 1}}] | ||
| 98 | + (let [pcm (alsa/open-pcm name :playback | ||
| 99 | + {:rate audio/sample-rate :channels channels | ||
| 100 | + :latency-us 40000})] | ||
| 101 | + {:play! (fn [{:keys [ptr samples]}] | ||
| 102 | + (when ptr (alsa/write! pcm ptr samples))) | ||
| 103 | + :close! (fn [] (alsa/close! pcm))})) | ||
modified
src/frq/capture/v4l2.clj +36 -2 | @@ -35,6 +35,10 @@ | ||
| 35 | 35 | (ffi/defcfn c-munmap "munmap" [:pointer :uint64] :int) |
| 36 | 36 | |
| 37 | 37 | (def ^:const o-rdwr 2) |
| 38 | +;; O_NONBLOCK, because the plane is pumped. A blocking DQBUF would hold | |
| 39 | +;; glimmer's loop thread for up to a frame interval every pump, and on a | |
| 40 | +;; camera that stops producing, for ever. | |
| 41 | +(def ^:const o-nonblock 2048) | |
| 38 | 42 | (def ^:const prot-read 1) |
| 39 | 43 | (def ^:const prot-write 2) |
| 40 | 44 | (def ^:const map-shared 1) |
| @@ -136,9 +140,9 @@ | ||
| 136 | 140 | rc)) |
| 137 | 141 | |
| 138 | 142 | (defn open-device |
| 139 | - "Open a camera and answer its fd." | |
| 143 | + "Open a camera and answer its fd. Non-blocking: see `o-nonblock`." | |
| 140 | 144 | [path] |
| 141 | - (let [fd (c-open path o-rdwr)] | |
| 145 | + (let [fd (c-open path (bit-or o-rdwr o-nonblock))] | |
| 142 | 146 | (when (neg? fd) |
| 143 | 147 | (throw (ex-info (str "v4l2: cannot open " path) {:errno (ffi/errno) :path path}))) |
| 144 | 148 | fd)) |
| @@ -243,6 +247,36 @@ | ||
| 243 | 247 | (ioctl! fd VIDIOC_STREAMOFF t "STREAMOFF"))) |
| 244 | 248 | nil) |
| 245 | 249 | |
| 250 | +(defn- eagain? | |
| 251 | + "EAGAIN (11) — no frame ready. On a non-blocking device that is the normal | |
| 252 | + answer most of the time, not a failure." | |
| 253 | + [errno] | |
| 254 | + (= errno 11)) | |
| 255 | + | |
| 256 | +(defn try-frame | |
| 257 | + "Dequeue a frame if one is ready, hand it to `f`, requeue it. | |
| 258 | + | |
| 259 | + Answers what `f` answered, or nil when the camera has nothing yet. Unlike | |
| 260 | + `with-frame` this never raises on an empty queue, which is what a pumped | |
| 261 | + caller needs — on a non-blocking device EAGAIN is the ordinary case." | |
| 262 | + [fd buffers f] | |
| 263 | + (ffi/with-arena [a] | |
| 264 | + (let [p (ffi/alloc a (ffi/layout-size buffer))] | |
| 265 | + (blank-buffer p 0) | |
| 266 | + (let [rc (c-ioctl fd VIDIOC_DQBUF p)] | |
| 267 | + (cond | |
| 268 | + (not (neg? rc)) | |
| 269 | + (let [i (ffi/read-field p buffer [:index]) | |
| 270 | + n (ffi/read-field p buffer [:bytesused]) | |
| 271 | + buf (nth buffers i)] | |
| 272 | + (try (f (:ptr buf) n) | |
| 273 | + (finally (queue! fd i)))) | |
| 274 | + | |
| 275 | + (eagain? (ffi/errno)) nil | |
| 276 | + | |
| 277 | + :else | |
| 278 | + (throw (ex-info "v4l2: DQBUF failed" {:errno (ffi/errno)}))))))) | |
| 279 | + | |
| 246 | 280 | (defn with-frame |
| 247 | 281 | "Dequeue a frame, hand it to `f` as [pointer length], and requeue it. |
| 248 | 282 | |
| @@ -35,6 +35,10 @@ | |||
| 35 | (ffi/defcfn c-munmap "munmap" [:pointer :uint64] :int) | 35 | (ffi/defcfn c-munmap "munmap" [:pointer :uint64] :int) |
| 36 | 36 | ||
| 37 | (def ^:const o-rdwr 2) | 37 | (def ^:const o-rdwr 2) |
| 38 | +;; O_NONBLOCK, because the plane is pumped. A blocking DQBUF would hold | ||
| 39 | +;; glimmer's loop thread for up to a frame interval every pump, and on a | ||
| 40 | +;; camera that stops producing, for ever. | ||
| 41 | +(def ^:const o-nonblock 2048) | ||
| 38 | (def ^:const prot-read 1) | 42 | (def ^:const prot-read 1) |
| 39 | (def ^:const prot-write 2) | 43 | (def ^:const prot-write 2) |
| 40 | (def ^:const map-shared 1) | 44 | (def ^:const map-shared 1) |
| @@ -136,9 +140,9 @@ | |||
| 136 | rc)) | 140 | rc)) |
| 137 | 141 | ||
| 138 | (defn open-device | 142 | (defn open-device |
| 139 | - "Open a camera and answer its fd." | 143 | + "Open a camera and answer its fd. Non-blocking: see `o-nonblock`." |
| 140 | [path] | 144 | [path] |
| 141 | - (let [fd (c-open path o-rdwr)] | 145 | + (let [fd (c-open path (bit-or o-rdwr o-nonblock))] |
| 142 | (when (neg? fd) | 146 | (when (neg? fd) |
| 143 | (throw (ex-info (str "v4l2: cannot open " path) {:errno (ffi/errno) :path path}))) | 147 | (throw (ex-info (str "v4l2: cannot open " path) {:errno (ffi/errno) :path path}))) |
| 144 | fd)) | 148 | fd)) |
| @@ -243,6 +247,36 @@ | |||
| 243 | (ioctl! fd VIDIOC_STREAMOFF t "STREAMOFF"))) | 247 | (ioctl! fd VIDIOC_STREAMOFF t "STREAMOFF"))) |
| 244 | nil) | 248 | nil) |
| 245 | 249 | ||
| 250 | +(defn- eagain? | ||
| 251 | + "EAGAIN (11) — no frame ready. On a non-blocking device that is the normal | ||
| 252 | + answer most of the time, not a failure." | ||
| 253 | + [errno] | ||
| 254 | + (= errno 11)) | ||
| 255 | + | ||
| 256 | +(defn try-frame | ||
| 257 | + "Dequeue a frame if one is ready, hand it to `f`, requeue it. | ||
| 258 | + | ||
| 259 | + Answers what `f` answered, or nil when the camera has nothing yet. Unlike | ||
| 260 | + `with-frame` this never raises on an empty queue, which is what a pumped | ||
| 261 | + caller needs — on a non-blocking device EAGAIN is the ordinary case." | ||
| 262 | + [fd buffers f] | ||
| 263 | + (ffi/with-arena [a] | ||
| 264 | + (let [p (ffi/alloc a (ffi/layout-size buffer))] | ||
| 265 | + (blank-buffer p 0) | ||
| 266 | + (let [rc (c-ioctl fd VIDIOC_DQBUF p)] | ||
| 267 | + (cond | ||
| 268 | + (not (neg? rc)) | ||
| 269 | + (let [i (ffi/read-field p buffer [:index]) | ||
| 270 | + n (ffi/read-field p buffer [:bytesused]) | ||
| 271 | + buf (nth buffers i)] | ||
| 272 | + (try (f (:ptr buf) n) | ||
| 273 | + (finally (queue! fd i)))) | ||
| 274 | + | ||
| 275 | + (eagain? (ffi/errno)) nil | ||
| 276 | + | ||
| 277 | + :else | ||
| 278 | + (throw (ex-info "v4l2: DQBUF failed" {:errno (ffi/errno)}))))))) | ||
| 279 | + | ||
| 246 | (defn with-frame | 280 | (defn with-frame |
| 247 | "Dequeue a frame, hand it to `f` as [pointer length], and requeue it. | 281 | "Dequeue a frame, hand it to `f` as [pointer length], and requeue it. |
| 248 | 282 | ||
modified
src/frq/moq/smoke.clj +43 -1 | @@ -41,6 +41,7 @@ | ||
| 41 | 41 | [frq.codec.h264 :as h264] |
| 42 | 42 | [frq.capture.v4l2 :as v4l2] |
| 43 | 43 | [frq.capture.alsa :as alsa] |
| 44 | + [frq.capture.source :as source] | |
| 44 | 45 | [frq.av.plane :as plane] |
| 45 | 46 | [frq.av.audio :as audio] |
| 46 | 47 | [jolt.ffi :as ffi])) |
| @@ -675,6 +676,46 @@ | ||
| 675 | 676 | |
| 676 | 677 | :else (do (Thread/sleep 10) (recur req accepted sess)))))))))) |
| 677 | 678 | |
| 679 | +(defn- check-wired-devices | |
| 680 | + "The plane driven by real device objects rather than test thunks. | |
| 681 | + | |
| 682 | + ALSA's `null` on both ends, and that is a limit of this machine rather | |
| 683 | + than a choice: the sound hardware is held by PipeWire on the host, and | |
| 684 | + PipeWire's own socket is not reachable from inside this container, so | |
| 685 | + `default` and `hw:1,0` both refuse. `null` is a real PCM opened through | |
| 686 | + the real binding — it proves the wiring, the frame arithmetic and the | |
| 687 | + teardown, and it cannot prove that a microphone sounds like anything. | |
| 688 | + | |
| 689 | + The camera is not here at all: there is no /dev/video* and no privilege to | |
| 690 | + load the kernel's virtual one. `frq.capture.source/camera` is written and | |
| 691 | + unexercised, and this test says so rather than implying otherwise. | |
| 692 | + | |
| 693 | + What IS asserted: that a device-shaped mic drives the outbound half, that | |
| 694 | + a speaker sink is written to without raising, and that stop! releases | |
| 695 | + both. A leak here would show up as `Device or resource busy` on the second | |
| 696 | + run, which is why the check runs the whole cycle twice." | |
| 697 | + [] | |
| 698 | + (dotimes [round 2] | |
| 699 | + (let [origin (media/new-origin)] | |
| 700 | + (plane/start! {:origin origin :path "/us" | |
| 701 | + :source (fn [] nil) | |
| 702 | + :mic-device "null" | |
| 703 | + :speaker-device "null" | |
| 704 | + :width 64 :height 64 :channels 1}) | |
| 705 | + (try | |
| 706 | + (dotimes [_ 5] (plane/pump!)) | |
| 707 | + (println (str " round " (inc round) ": mic and speaker opened, pumped, closed")) | |
| 708 | + (finally (plane/stop!))))) | |
| 709 | + ;; And the camera path as far as it goes on a machine with no camera: | |
| 710 | + ;; enumeration is empty and opening one raises rather than pretending. | |
| 711 | + (let [cams (v4l2/devices)] | |
| 712 | + (println " cameras available:" (count cams)) | |
| 713 | + (when (seq cams) | |
| 714 | + (let [c (source/camera (:id (first cams)) {:width 640 :height 480})] | |
| 715 | + (println " opened" (:id (first cams)) (:width c) "x" (:height c)) | |
| 716 | + ((:close! c))))) | |
| 717 | + true) | |
| 718 | + | |
| 678 | 719 | (defn -main [& _] |
| 679 | 720 | (println "libmoq_ffi smoke test") |
| 680 | 721 | (let [steps [["contract" check-contract] |
| @@ -689,7 +730,8 @@ | ||
| 689 | 730 | ["devices" check-enumeration] |
| 690 | 731 | ["plane" check-plane] |
| 691 | 732 | ["audio" check-audio] |
| 692 | - ["session" check-session]]] | |
| 733 | + ["session" check-session] | |
| 734 | + ["wired" check-wired-devices]]] | |
| 693 | 735 | (doseq [[name f] steps] |
| 694 | 736 | (println (str name ":")) |
| 695 | 737 | (f)) |
| @@ -41,6 +41,7 @@ | |||
| 41 | [frq.codec.h264 :as h264] | 41 | [frq.codec.h264 :as h264] |
| 42 | [frq.capture.v4l2 :as v4l2] | 42 | [frq.capture.v4l2 :as v4l2] |
| 43 | [frq.capture.alsa :as alsa] | 43 | [frq.capture.alsa :as alsa] |
| 44 | + [frq.capture.source :as source] | ||
| 44 | [frq.av.plane :as plane] | 45 | [frq.av.plane :as plane] |
| 45 | [frq.av.audio :as audio] | 46 | [frq.av.audio :as audio] |
| 46 | [jolt.ffi :as ffi])) | 47 | [jolt.ffi :as ffi])) |
| @@ -675,6 +676,46 @@ | |||
| 675 | 676 | ||
| 676 | :else (do (Thread/sleep 10) (recur req accepted sess)))))))))) | 677 | :else (do (Thread/sleep 10) (recur req accepted sess)))))))))) |
| 677 | 678 | ||
| 679 | +(defn- check-wired-devices | ||
| 680 | + "The plane driven by real device objects rather than test thunks. | ||
| 681 | + | ||
| 682 | + ALSA's `null` on both ends, and that is a limit of this machine rather | ||
| 683 | + than a choice: the sound hardware is held by PipeWire on the host, and | ||
| 684 | + PipeWire's own socket is not reachable from inside this container, so | ||
| 685 | + `default` and `hw:1,0` both refuse. `null` is a real PCM opened through | ||
| 686 | + the real binding — it proves the wiring, the frame arithmetic and the | ||
| 687 | + teardown, and it cannot prove that a microphone sounds like anything. | ||
| 688 | + | ||
| 689 | + The camera is not here at all: there is no /dev/video* and no privilege to | ||
| 690 | + load the kernel's virtual one. `frq.capture.source/camera` is written and | ||
| 691 | + unexercised, and this test says so rather than implying otherwise. | ||
| 692 | + | ||
| 693 | + What IS asserted: that a device-shaped mic drives the outbound half, that | ||
| 694 | + a speaker sink is written to without raising, and that stop! releases | ||
| 695 | + both. A leak here would show up as `Device or resource busy` on the second | ||
| 696 | + run, which is why the check runs the whole cycle twice." | ||
| 697 | + [] | ||
| 698 | + (dotimes [round 2] | ||
| 699 | + (let [origin (media/new-origin)] | ||
| 700 | + (plane/start! {:origin origin :path "/us" | ||
| 701 | + :source (fn [] nil) | ||
| 702 | + :mic-device "null" | ||
| 703 | + :speaker-device "null" | ||
| 704 | + :width 64 :height 64 :channels 1}) | ||
| 705 | + (try | ||
| 706 | + (dotimes [_ 5] (plane/pump!)) | ||
| 707 | + (println (str " round " (inc round) ": mic and speaker opened, pumped, closed")) | ||
| 708 | + (finally (plane/stop!))))) | ||
| 709 | + ;; And the camera path as far as it goes on a machine with no camera: | ||
| 710 | + ;; enumeration is empty and opening one raises rather than pretending. | ||
| 711 | + (let [cams (v4l2/devices)] | ||
| 712 | + (println " cameras available:" (count cams)) | ||
| 713 | + (when (seq cams) | ||
| 714 | + (let [c (source/camera (:id (first cams)) {:width 640 :height 480})] | ||
| 715 | + (println " opened" (:id (first cams)) (:width c) "x" (:height c)) | ||
| 716 | + ((:close! c))))) | ||
| 717 | + true) | ||
| 718 | + | ||
| 678 | (defn -main [& _] | 719 | (defn -main [& _] |
| 679 | (println "libmoq_ffi smoke test") | 720 | (println "libmoq_ffi smoke test") |
| 680 | (let [steps [["contract" check-contract] | 721 | (let [steps [["contract" check-contract] |
| @@ -689,7 +730,8 @@ | |||
| 689 | ["devices" check-enumeration] | 730 | ["devices" check-enumeration] |
| 690 | ["plane" check-plane] | 731 | ["plane" check-plane] |
| 691 | ["audio" check-audio] | 732 | ["audio" check-audio] |
| 692 | - ["session" check-session]]] | 733 | + ["session" check-session] |
| 734 | + ["wired" check-wired-devices]]] | ||
| 693 | (doseq [[name f] steps] | 735 | (doseq [[name f] steps] |
| 694 | (println (str name ":")) | 736 | (println (str name ":")) |
| 695 | (f)) | 737 | (f)) |