Decode H.264 to RGBA, and list the devices av.clj offers
The leaves av.clj needs that were still missing. Not av.clj itself: see below. The decoder is the same vtable problem as the encoder and the same answer -- ISVCDecoder is `const ISVCDecoderVtbl*`, so the shim walks it. It answers RGBA rather than I420 because vidya/frame-rgba! is where this is going: the conversion happens once in C instead of dragging three planes and their strides into jolt to be rearranged. Those strides are also why it belongs there. A decoder pads its rows, stride is not width, and copying width bytes from a stride-wide plane is what shears a picture diagonally. Round-tripped end to end: gray I420 in, 47 bytes of H.264, back out as 64x64 RGBA reading [130 130 130 255]. The assertion is the pixel, not the size -- a decode answering the right dimensions full of the wrong bytes would pass a length check. 130 rather than 128 is BT.601 doing its arithmetic. Writing that test found the thing a subscriber hits: the second frame out of an encoder is a P-frame, and a decoder handed one first answers dsNoParamSets because it has no SPS or PPS to decode against. Anyone joining a call mid-stream is in exactly that position, which is what force-keyframe! is for. Enumeration is what av.clj's cameras/microphones/speakers become. V4L2 probes /dev/videoN and filters on device_caps rather than on the node existing: one camera usually presents several nodes and only one of them streams, so listing the rest is how a device menu fills with entries that fail when picked. ALSA walks snd_device_name_hint, whose every field is a char* the caller frees -- so they are read as pointers and released, where :string would hand back a jolt string and lose the address. `null` is dropped: it accepts everything and gives silence that looks like a broken microphone. WHAT IS NOT DONE, and it is the part that matters: av.clj is untouched, and its 27 joltmoq_* calls are still there. libjoltmoq is not a wrapper around these leaves, it is a media plane -- joltmoq_start connects, publishes mic and camera, subscribes to every peer, decodes their video and mixes their audio, behind one call. What is still missing above these bindings is the pipeline: per-peer subscribe and catalog handling, audio mixing across peers with a jitter buffer and resampling, capture and encode paced off the loop thread, and an Android half where neither V4L2 nor ALSA exists. That is the three thousand lines deps.edn warned about, and it is what the port needs next -- not a swap of call sites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
982c702 parent: 87e5be9 modified
c/frq_h264.c +114 -0 | @@ -130,3 +130,117 @@ void frq_h264_close(void *handle) { | ||
| 130 | 130 | free(h->out); |
| 131 | 131 | free(h); |
| 132 | 132 | } |
| 133 | + | |
| 134 | +/* --- decoding ------------------------------------------------------------- | |
| 135 | + * | |
| 136 | + * Same vtable problem, same answer. ISVCDecoder is `const ISVCDecoderVtbl*`, | |
| 137 | + * so DecodeFrameNoDelay is a function pointer and jolt cannot reach it. | |
| 138 | + * | |
| 139 | + * What comes out is I420 in the decoder's OWN buffers, three planes with | |
| 140 | + * their own strides — which are not the width. A decoder pads its rows, so | |
| 141 | + * copying `width` bytes per row from a `stride`-wide plane is the mistake | |
| 142 | + * that produces a picture sheared diagonally, and it is why the strides are | |
| 143 | + * carried through to the converter below rather than assumed away. | |
| 144 | + */ | |
| 145 | +#include <wels/codec_def.h> | |
| 146 | + | |
| 147 | +typedef struct { | |
| 148 | + ISVCDecoder *dec; | |
| 149 | + unsigned char *rgba; /* converted output, grown as needed */ | |
| 150 | + size_t rgba_cap; | |
| 151 | +} frq_h264_dec; | |
| 152 | + | |
| 153 | +int frq_h264_decoder_open(void **handle) { | |
| 154 | + ISVCDecoder *dec = NULL; | |
| 155 | + frq_h264_dec *d; | |
| 156 | + SDecodingParam p; | |
| 157 | + int rc; | |
| 158 | + | |
| 159 | + *handle = NULL; | |
| 160 | + rc = WelsCreateDecoder(&dec); | |
| 161 | + if (rc != 0 || dec == NULL) return rc ? rc : -1; | |
| 162 | + | |
| 163 | + memset(&p, 0, sizeof(p)); | |
| 164 | + p.eEcActiveIdc = ERROR_CON_SLICE_COPY; | |
| 165 | + p.sVideoProperty.eVideoBsType = VIDEO_BITSTREAM_AVC; | |
| 166 | + | |
| 167 | + rc = (int)(*dec)->Initialize(dec, &p); | |
| 168 | + if (rc != 0) { WelsDestroyDecoder(dec); return rc; } | |
| 169 | + | |
| 170 | + d = (frq_h264_dec *)calloc(1, sizeof(frq_h264_dec)); | |
| 171 | + if (d == NULL) { (*dec)->Uninitialize(dec); WelsDestroyDecoder(dec); return -1; } | |
| 172 | + d->dec = dec; | |
| 173 | + *handle = d; | |
| 174 | + return 0; | |
| 175 | +} | |
| 176 | + | |
| 177 | +/* Decode one Annex B frame and convert it to RGBA. | |
| 178 | + * | |
| 179 | + * RGBA rather than I420 because that is what the far end of this is: | |
| 180 | + * vidya/frame-rgba! takes a tightly packed RGBA buffer, and converting here | |
| 181 | + * means the pixels are touched once, in C, instead of crossing into jolt to | |
| 182 | + * be rearranged. On success answers 0; *out is NULL and *w/*h are 0 when the | |
| 183 | + * decoder has no picture yet, which is normal for the first packets. */ | |
| 184 | +int frq_h264_decode_rgba(void *handle, const unsigned char *annexb, int len, | |
| 185 | + const unsigned char **out, int *w, int *h) { | |
| 186 | + frq_h264_dec *d = (frq_h264_dec *)handle; | |
| 187 | + unsigned char *planes[3] = {NULL, NULL, NULL}; | |
| 188 | + SBufferInfo info; | |
| 189 | + DECODING_STATE st; | |
| 190 | + int width, height, y, x, sy, su, sv; | |
| 191 | + size_t need; | |
| 192 | + | |
| 193 | + *out = NULL; *w = 0; *h = 0; | |
| 194 | + memset(&info, 0, sizeof(info)); | |
| 195 | + | |
| 196 | + st = (*d->dec)->DecodeFrameNoDelay(d->dec, annexb, len, planes, &info); | |
| 197 | + if (st != dsErrorFree) return (int)st; | |
| 198 | + if (info.iBufferStatus != 1) return 0; /* no picture this time */ | |
| 199 | + | |
| 200 | + width = info.UsrData.sSystemBuffer.iWidth; | |
| 201 | + height = info.UsrData.sSystemBuffer.iHeight; | |
| 202 | + sy = info.UsrData.sSystemBuffer.iStride[0]; | |
| 203 | + su = info.UsrData.sSystemBuffer.iStride[1]; | |
| 204 | + sv = su; | |
| 205 | + if (width <= 0 || height <= 0) return 0; | |
| 206 | + | |
| 207 | + need = (size_t)width * (size_t)height * 4u; | |
| 208 | + if (need > d->rgba_cap) { | |
| 209 | + unsigned char *grown = (unsigned char *)realloc(d->rgba, need); | |
| 210 | + if (grown == NULL) return -1; | |
| 211 | + d->rgba = grown; d->rgba_cap = need; | |
| 212 | + } | |
| 213 | + | |
| 214 | + /* BT.601 limited range, integer. Not a quality decision worth agonising | |
| 215 | + * over here: it is what a webcam stream is tagged as, and the alternative | |
| 216 | + * is dragging a colour-management dependency in for a video call. */ | |
| 217 | + for (y = 0; y < height; y++) { | |
| 218 | + const unsigned char *Y = planes[0] + (size_t)y * sy; | |
| 219 | + const unsigned char *U = planes[1] + (size_t)(y / 2) * su; | |
| 220 | + const unsigned char *V = planes[2] + (size_t)(y / 2) * sv; | |
| 221 | + unsigned char *dst = d->rgba + (size_t)y * width * 4; | |
| 222 | + for (x = 0; x < width; x++) { | |
| 223 | + int c = (int)Y[x] - 16; | |
| 224 | + int u = (int)U[x / 2] - 128; | |
| 225 | + int v = (int)V[x / 2] - 128; | |
| 226 | + int r = (298 * c + 409 * v + 128) >> 8; | |
| 227 | + int g = (298 * c - 100 * u - 208 * v + 128) >> 8; | |
| 228 | + int b = (298 * c + 516 * u + 128) >> 8; | |
| 229 | + dst[x * 4 + 0] = (unsigned char)(r < 0 ? 0 : r > 255 ? 255 : r); | |
| 230 | + dst[x * 4 + 1] = (unsigned char)(g < 0 ? 0 : g > 255 ? 255 : g); | |
| 231 | + dst[x * 4 + 2] = (unsigned char)(b < 0 ? 0 : b > 255 ? 255 : b); | |
| 232 | + dst[x * 4 + 3] = 255; | |
| 233 | + } | |
| 234 | + } | |
| 235 | + | |
| 236 | + *out = d->rgba; *w = width; *h = height; | |
| 237 | + return 0; | |
| 238 | +} | |
| 239 | + | |
| 240 | +void frq_h264_decoder_close(void *handle) { | |
| 241 | + frq_h264_dec *d = (frq_h264_dec *)handle; | |
| 242 | + if (d == NULL) return; | |
| 243 | + if (d->dec) { (*d->dec)->Uninitialize(d->dec); WelsDestroyDecoder(d->dec); } | |
| 244 | + free(d->rgba); | |
| 245 | + free(d); | |
| 246 | +} | |
| @@ -130,3 +130,117 @@ void frq_h264_close(void *handle) { | |||
| 130 | free(h->out); | 130 | free(h->out); |
| 131 | free(h); | 131 | free(h); |
| 132 | } | 132 | } |
| 133 | + | ||
| 134 | +/* --- decoding ------------------------------------------------------------- | ||
| 135 | + * | ||
| 136 | + * Same vtable problem, same answer. ISVCDecoder is `const ISVCDecoderVtbl*`, | ||
| 137 | + * so DecodeFrameNoDelay is a function pointer and jolt cannot reach it. | ||
| 138 | + * | ||
| 139 | + * What comes out is I420 in the decoder's OWN buffers, three planes with | ||
| 140 | + * their own strides — which are not the width. A decoder pads its rows, so | ||
| 141 | + * copying `width` bytes per row from a `stride`-wide plane is the mistake | ||
| 142 | + * that produces a picture sheared diagonally, and it is why the strides are | ||
| 143 | + * carried through to the converter below rather than assumed away. | ||
| 144 | + */ | ||
| 145 | +#include <wels/codec_def.h> | ||
| 146 | + | ||
| 147 | +typedef struct { | ||
| 148 | + ISVCDecoder *dec; | ||
| 149 | + unsigned char *rgba; /* converted output, grown as needed */ | ||
| 150 | + size_t rgba_cap; | ||
| 151 | +} frq_h264_dec; | ||
| 152 | + | ||
| 153 | +int frq_h264_decoder_open(void **handle) { | ||
| 154 | + ISVCDecoder *dec = NULL; | ||
| 155 | + frq_h264_dec *d; | ||
| 156 | + SDecodingParam p; | ||
| 157 | + int rc; | ||
| 158 | + | ||
| 159 | + *handle = NULL; | ||
| 160 | + rc = WelsCreateDecoder(&dec); | ||
| 161 | + if (rc != 0 || dec == NULL) return rc ? rc : -1; | ||
| 162 | + | ||
| 163 | + memset(&p, 0, sizeof(p)); | ||
| 164 | + p.eEcActiveIdc = ERROR_CON_SLICE_COPY; | ||
| 165 | + p.sVideoProperty.eVideoBsType = VIDEO_BITSTREAM_AVC; | ||
| 166 | + | ||
| 167 | + rc = (int)(*dec)->Initialize(dec, &p); | ||
| 168 | + if (rc != 0) { WelsDestroyDecoder(dec); return rc; } | ||
| 169 | + | ||
| 170 | + d = (frq_h264_dec *)calloc(1, sizeof(frq_h264_dec)); | ||
| 171 | + if (d == NULL) { (*dec)->Uninitialize(dec); WelsDestroyDecoder(dec); return -1; } | ||
| 172 | + d->dec = dec; | ||
| 173 | + *handle = d; | ||
| 174 | + return 0; | ||
| 175 | +} | ||
| 176 | + | ||
| 177 | +/* Decode one Annex B frame and convert it to RGBA. | ||
| 178 | + * | ||
| 179 | + * RGBA rather than I420 because that is what the far end of this is: | ||
| 180 | + * vidya/frame-rgba! takes a tightly packed RGBA buffer, and converting here | ||
| 181 | + * means the pixels are touched once, in C, instead of crossing into jolt to | ||
| 182 | + * be rearranged. On success answers 0; *out is NULL and *w/*h are 0 when the | ||
| 183 | + * decoder has no picture yet, which is normal for the first packets. */ | ||
| 184 | +int frq_h264_decode_rgba(void *handle, const unsigned char *annexb, int len, | ||
| 185 | + const unsigned char **out, int *w, int *h) { | ||
| 186 | + frq_h264_dec *d = (frq_h264_dec *)handle; | ||
| 187 | + unsigned char *planes[3] = {NULL, NULL, NULL}; | ||
| 188 | + SBufferInfo info; | ||
| 189 | + DECODING_STATE st; | ||
| 190 | + int width, height, y, x, sy, su, sv; | ||
| 191 | + size_t need; | ||
| 192 | + | ||
| 193 | + *out = NULL; *w = 0; *h = 0; | ||
| 194 | + memset(&info, 0, sizeof(info)); | ||
| 195 | + | ||
| 196 | + st = (*d->dec)->DecodeFrameNoDelay(d->dec, annexb, len, planes, &info); | ||
| 197 | + if (st != dsErrorFree) return (int)st; | ||
| 198 | + if (info.iBufferStatus != 1) return 0; /* no picture this time */ | ||
| 199 | + | ||
| 200 | + width = info.UsrData.sSystemBuffer.iWidth; | ||
| 201 | + height = info.UsrData.sSystemBuffer.iHeight; | ||
| 202 | + sy = info.UsrData.sSystemBuffer.iStride[0]; | ||
| 203 | + su = info.UsrData.sSystemBuffer.iStride[1]; | ||
| 204 | + sv = su; | ||
| 205 | + if (width <= 0 || height <= 0) return 0; | ||
| 206 | + | ||
| 207 | + need = (size_t)width * (size_t)height * 4u; | ||
| 208 | + if (need > d->rgba_cap) { | ||
| 209 | + unsigned char *grown = (unsigned char *)realloc(d->rgba, need); | ||
| 210 | + if (grown == NULL) return -1; | ||
| 211 | + d->rgba = grown; d->rgba_cap = need; | ||
| 212 | + } | ||
| 213 | + | ||
| 214 | + /* BT.601 limited range, integer. Not a quality decision worth agonising | ||
| 215 | + * over here: it is what a webcam stream is tagged as, and the alternative | ||
| 216 | + * is dragging a colour-management dependency in for a video call. */ | ||
| 217 | + for (y = 0; y < height; y++) { | ||
| 218 | + const unsigned char *Y = planes[0] + (size_t)y * sy; | ||
| 219 | + const unsigned char *U = planes[1] + (size_t)(y / 2) * su; | ||
| 220 | + const unsigned char *V = planes[2] + (size_t)(y / 2) * sv; | ||
| 221 | + unsigned char *dst = d->rgba + (size_t)y * width * 4; | ||
| 222 | + for (x = 0; x < width; x++) { | ||
| 223 | + int c = (int)Y[x] - 16; | ||
| 224 | + int u = (int)U[x / 2] - 128; | ||
| 225 | + int v = (int)V[x / 2] - 128; | ||
| 226 | + int r = (298 * c + 409 * v + 128) >> 8; | ||
| 227 | + int g = (298 * c - 100 * u - 208 * v + 128) >> 8; | ||
| 228 | + int b = (298 * c + 516 * u + 128) >> 8; | ||
| 229 | + dst[x * 4 + 0] = (unsigned char)(r < 0 ? 0 : r > 255 ? 255 : r); | ||
| 230 | + dst[x * 4 + 1] = (unsigned char)(g < 0 ? 0 : g > 255 ? 255 : g); | ||
| 231 | + dst[x * 4 + 2] = (unsigned char)(b < 0 ? 0 : b > 255 ? 255 : b); | ||
| 232 | + dst[x * 4 + 3] = 255; | ||
| 233 | + } | ||
| 234 | + } | ||
| 235 | + | ||
| 236 | + *out = d->rgba; *w = width; *h = height; | ||
| 237 | + return 0; | ||
| 238 | +} | ||
| 239 | + | ||
| 240 | +void frq_h264_decoder_close(void *handle) { | ||
| 241 | + frq_h264_dec *d = (frq_h264_dec *)handle; | ||
| 242 | + if (d == NULL) return; | ||
| 243 | + if (d->dec) { (*d->dec)->Uninitialize(d->dec); WelsDestroyDecoder(d->dec); } | ||
| 244 | + free(d->rgba); | ||
| 245 | + free(d); | ||
| 246 | +} | ||
modified
src/frq/capture/alsa.clj +60 -1 | @@ -19,7 +19,8 @@ | ||
| 19 | 19 | is not a failure — `snd_pcm_recover` puts the stream back and the next read |
| 20 | 20 | continues. `read!` does that itself and reports the loss rather than |
| 21 | 21 | raising, because a dropped buffer is a thing a call survives." |
| 22 | - (:require [jolt.ffi :as ffi])) | |
| 22 | + (:require [clojure.string :as str] | |
| 23 | + [jolt.ffi :as ffi])) | |
| 23 | 24 | |
| 24 | 25 | (def ^:const format-s16-le 2) |
| 25 | 26 | (def ^:const access-rw-interleaved 3) |
| @@ -83,6 +84,64 @@ | ||
| 83 | 84 | :recovered true}) |
| 84 | 85 | {:frames n}))) |
| 85 | 86 | |
| 87 | +;; --- enumeration ------------------------------------------------------------- | |
| 88 | +;; snd_device_name_hint answers a NULL-terminated array of opaque hints, and | |
| 89 | +;; each field of a hint is a char* the CALLER frees. Both facts shape this: | |
| 90 | +;; the array is walked a pointer at a time, and every string is read through | |
| 91 | +;; ptr->string and then released, because declaring it :string would hand | |
| 92 | +;; back a jolt string and lose the address that has to be freed. | |
| 93 | + | |
| 94 | +(ffi/defcfn raw-name-hint "snd_device_name_hint" [:int :string :pointer] :int) | |
| 95 | +(ffi/defcfn raw-get-hint "snd_device_name_get_hint" [:pointer :string] :pointer) | |
| 96 | +(ffi/defcfn raw-free-hint "snd_device_name_free_hint" [:pointer] :int) | |
| 97 | + | |
| 98 | +(defn- hint-field [hint id] | |
| 99 | + (let [p (raw-get-hint hint id)] | |
| 100 | + (when-not (ffi/null? p) | |
| 101 | + (let [s (ffi/ptr->string p)] | |
| 102 | + (ffi/free p) | |
| 103 | + s)))) | |
| 104 | + | |
| 105 | +(defn devices | |
| 106 | + "PCMs ALSA is willing to name, as {:id :name :default?}. | |
| 107 | + | |
| 108 | + `direction` is :capture or :playback, and it filters on the hint's IOID: | |
| 109 | + a device with no IOID does both, which is most of them, so absence means | |
| 110 | + yes rather than no. | |
| 111 | + | |
| 112 | + The `null` PCM is dropped. It is always present, it swallows everything, | |
| 113 | + and a person picking it from a list of microphones would get silence that | |
| 114 | + looks exactly like a broken device." | |
| 115 | + [direction] | |
| 116 | + (let [want (case direction :capture "Input" :playback "Output")] | |
| 117 | + (ffi/with-arena [a] | |
| 118 | + (let [out (ffi/alloc a 8)] | |
| 119 | + (when (neg? (raw-name-hint -1 "pcm" out)) | |
| 120 | + (throw (ex-info "alsa: could not list devices" {}))) | |
| 121 | + (let [arr (ffi/read out :pointer)] | |
| 122 | + (if (ffi/null? arr) | |
| 123 | + [] | |
| 124 | + (try | |
| 125 | + (loop [i 0 acc []] | |
| 126 | + (let [hint (ffi/read (+ arr (* 8 i)) :pointer)] | |
| 127 | + (if (ffi/null? hint) | |
| 128 | + acc | |
| 129 | + (let [name (hint-field hint "NAME") | |
| 130 | + desc (hint-field hint "DESC") | |
| 131 | + ioid (hint-field hint "IOID")] | |
| 132 | + (recur (inc i) | |
| 133 | + (if (and name | |
| 134 | + (not= "null" name) | |
| 135 | + (or (nil? ioid) (= ioid want))) | |
| 136 | + (conj acc {:id name | |
| 137 | + ;; DESC is multi-line: a friendly | |
| 138 | + ;; name, then the card detail. | |
| 139 | + :name (or (some-> desc str/split-lines first) | |
| 140 | + name) | |
| 141 | + :default? (= "default" name)}) | |
| 142 | + acc)))))) | |
| 143 | + (finally (raw-free-hint arr))))))))) | |
| 144 | + | |
| 86 | 145 | (defn prepare! [pcm] (check! (raw-prepare pcm) "prepare") nil) |
| 87 | 146 | (defn drain! [pcm] (check! (raw-drain pcm) "drain") nil) |
| 88 | 147 | (defn close! [pcm] (raw-close pcm) nil) |
| @@ -19,7 +19,8 @@ | |||
| 19 | is not a failure — `snd_pcm_recover` puts the stream back and the next read | 19 | is not a failure — `snd_pcm_recover` puts the stream back and the next read |
| 20 | continues. `read!` does that itself and reports the loss rather than | 20 | continues. `read!` does that itself and reports the loss rather than |
| 21 | raising, because a dropped buffer is a thing a call survives." | 21 | raising, because a dropped buffer is a thing a call survives." |
| 22 | - (:require [jolt.ffi :as ffi])) | 22 | + (:require [clojure.string :as str] |
| 23 | + [jolt.ffi :as ffi])) | ||
| 23 | 24 | ||
| 24 | (def ^:const format-s16-le 2) | 25 | (def ^:const format-s16-le 2) |
| 25 | (def ^:const access-rw-interleaved 3) | 26 | (def ^:const access-rw-interleaved 3) |
| @@ -83,6 +84,64 @@ | |||
| 83 | :recovered true}) | 84 | :recovered true}) |
| 84 | {:frames n}))) | 85 | {:frames n}))) |
| 85 | 86 | ||
| 87 | +;; --- enumeration ------------------------------------------------------------- | ||
| 88 | +;; snd_device_name_hint answers a NULL-terminated array of opaque hints, and | ||
| 89 | +;; each field of a hint is a char* the CALLER frees. Both facts shape this: | ||
| 90 | +;; the array is walked a pointer at a time, and every string is read through | ||
| 91 | +;; ptr->string and then released, because declaring it :string would hand | ||
| 92 | +;; back a jolt string and lose the address that has to be freed. | ||
| 93 | + | ||
| 94 | +(ffi/defcfn raw-name-hint "snd_device_name_hint" [:int :string :pointer] :int) | ||
| 95 | +(ffi/defcfn raw-get-hint "snd_device_name_get_hint" [:pointer :string] :pointer) | ||
| 96 | +(ffi/defcfn raw-free-hint "snd_device_name_free_hint" [:pointer] :int) | ||
| 97 | + | ||
| 98 | +(defn- hint-field [hint id] | ||
| 99 | + (let [p (raw-get-hint hint id)] | ||
| 100 | + (when-not (ffi/null? p) | ||
| 101 | + (let [s (ffi/ptr->string p)] | ||
| 102 | + (ffi/free p) | ||
| 103 | + s)))) | ||
| 104 | + | ||
| 105 | +(defn devices | ||
| 106 | + "PCMs ALSA is willing to name, as {:id :name :default?}. | ||
| 107 | + | ||
| 108 | + `direction` is :capture or :playback, and it filters on the hint's IOID: | ||
| 109 | + a device with no IOID does both, which is most of them, so absence means | ||
| 110 | + yes rather than no. | ||
| 111 | + | ||
| 112 | + The `null` PCM is dropped. It is always present, it swallows everything, | ||
| 113 | + and a person picking it from a list of microphones would get silence that | ||
| 114 | + looks exactly like a broken device." | ||
| 115 | + [direction] | ||
| 116 | + (let [want (case direction :capture "Input" :playback "Output")] | ||
| 117 | + (ffi/with-arena [a] | ||
| 118 | + (let [out (ffi/alloc a 8)] | ||
| 119 | + (when (neg? (raw-name-hint -1 "pcm" out)) | ||
| 120 | + (throw (ex-info "alsa: could not list devices" {}))) | ||
| 121 | + (let [arr (ffi/read out :pointer)] | ||
| 122 | + (if (ffi/null? arr) | ||
| 123 | + [] | ||
| 124 | + (try | ||
| 125 | + (loop [i 0 acc []] | ||
| 126 | + (let [hint (ffi/read (+ arr (* 8 i)) :pointer)] | ||
| 127 | + (if (ffi/null? hint) | ||
| 128 | + acc | ||
| 129 | + (let [name (hint-field hint "NAME") | ||
| 130 | + desc (hint-field hint "DESC") | ||
| 131 | + ioid (hint-field hint "IOID")] | ||
| 132 | + (recur (inc i) | ||
| 133 | + (if (and name | ||
| 134 | + (not= "null" name) | ||
| 135 | + (or (nil? ioid) (= ioid want))) | ||
| 136 | + (conj acc {:id name | ||
| 137 | + ;; DESC is multi-line: a friendly | ||
| 138 | + ;; name, then the card detail. | ||
| 139 | + :name (or (some-> desc str/split-lines first) | ||
| 140 | + name) | ||
| 141 | + :default? (= "default" name)}) | ||
| 142 | + acc)))))) | ||
| 143 | + (finally (raw-free-hint arr))))))))) | ||
| 144 | + | ||
| 86 | (defn prepare! [pcm] (check! (raw-prepare pcm) "prepare") nil) | 145 | (defn prepare! [pcm] (check! (raw-prepare pcm) "prepare") nil) |
| 87 | (defn drain! [pcm] (check! (raw-drain pcm) "drain") nil) | 146 | (defn drain! [pcm] (check! (raw-drain pcm) "drain") nil) |
| 88 | (defn close! [pcm] (raw-close pcm) nil) | 147 | (defn close! [pcm] (raw-close pcm) nil) |
modified
src/frq/capture/v4l2.clj +42 -2 | @@ -18,11 +18,15 @@ | ||
| 18 | 18 | is valid until the buffer goes back, and nothing here copies it. That is |
| 19 | 19 | `frq.av`'s rule arriving from the other end — capture buffer to encoder as a |
| 20 | 20 | pointer, the way the decoder's buffer already reaches a texture." |
| 21 | - (:require [jolt.ffi :as ffi])) | |
| 21 | + (:require [clojure.string :as str] | |
| 22 | + [jolt.ffi :as ffi])) | |
| 22 | 23 | |
| 23 | 24 | ;; --- libc -------------------------------------------------------------------- |
| 24 | 25 | |
| 25 | -(ffi/defcfn c-open "open" [:string :int :& :int] :int) | |
| 26 | +;; Not variadic: open(2)'s third argument exists only for O_CREAT, which a | |
| 27 | +;; device node never wants. Declaring the tail would oblige every call to | |
| 28 | +;; pass a mode that the kernel then ignores. | |
| 29 | +(ffi/defcfn c-open "open" [:string :int] :int) | |
| 26 | 30 | (ffi/defcfn c-close "close" [:int] :int) |
| 27 | 31 | ;; ioctl's third argument is whatever the request says it is; for every |
| 28 | 32 | ;; request here it is a pointer, and the declared tail costs no compile. |
| @@ -258,6 +262,42 @@ | ||
| 258 | 262 | (f (:ptr buf) n) |
| 259 | 263 | (finally (queue! fd i))))))) |
| 260 | 264 | |
| 265 | +(defn devices | |
| 266 | + "Every /dev/videoN that can actually capture, as {:id :name :default?}. | |
| 267 | + | |
| 268 | + Probed rather than listed: reading the directory would mean binding | |
| 269 | + opendir and readdir for a range the kernel keeps small anyway, and a node | |
| 270 | + that cannot be opened is one this process could not have used regardless. | |
| 271 | + | |
| 272 | + The filter matters more than it looks. A single camera usually presents | |
| 273 | + SEVERAL /dev/video nodes — the capture node beside metadata and control | |
| 274 | + nodes — and only one of them streams. Offering the others is how a device | |
| 275 | + list ends up with entries that fail the moment they are picked, so | |
| 276 | + QUERYCAP's device_caps decides rather than the node existing." | |
| 277 | + [] | |
| 278 | + (->> (range 64) | |
| 279 | + (keep (fn [i] | |
| 280 | + (let [path (str "/dev/video" i) | |
| 281 | + fd (c-open path o-rdwr)] | |
| 282 | + (when-not (neg? fd) | |
| 283 | + (try | |
| 284 | + (let [caps (capabilities fd)] | |
| 285 | + (when (and (:capture? caps) (:streaming? caps)) | |
| 286 | + (ffi/with-arena [a] | |
| 287 | + (let [p (ffi/alloc a (ffi/layout-size capability))] | |
| 288 | + (ioctl! fd VIDIOC_QUERYCAP p "QUERYCAP") | |
| 289 | + (let [card (ffi/read-bytes | |
| 290 | + (+ p (ffi/field-offset capability [:card])) 32) | |
| 291 | + ;; The kernel pads `card` with NULs to 32 | |
| 292 | + ;; bytes; a jolt string of the whole field | |
| 293 | + ;; would carry them. | |
| 294 | + name (first (str/split card #"\x00"))] | |
| 295 | + {:id path | |
| 296 | + :name (if (seq name) name path) | |
| 297 | + :default? (zero? i)}))))) | |
| 298 | + (finally (c-close fd))))))) | |
| 299 | + vec)) | |
| 300 | + | |
| 261 | 301 | (defn close-device! [fd buffers] |
| 262 | 302 | (doseq [{:keys [ptr len]} buffers] (c-munmap ptr len)) |
| 263 | 303 | (c-close fd) |
| @@ -18,11 +18,15 @@ | |||
| 18 | is valid until the buffer goes back, and nothing here copies it. That is | 18 | is valid until the buffer goes back, and nothing here copies it. That is |
| 19 | `frq.av`'s rule arriving from the other end — capture buffer to encoder as a | 19 | `frq.av`'s rule arriving from the other end — capture buffer to encoder as a |
| 20 | pointer, the way the decoder's buffer already reaches a texture." | 20 | pointer, the way the decoder's buffer already reaches a texture." |
| 21 | - (:require [jolt.ffi :as ffi])) | 21 | + (:require [clojure.string :as str] |
| 22 | + [jolt.ffi :as ffi])) | ||
| 22 | 23 | ||
| 23 | ;; --- libc -------------------------------------------------------------------- | 24 | ;; --- libc -------------------------------------------------------------------- |
| 24 | 25 | ||
| 25 | -(ffi/defcfn c-open "open" [:string :int :& :int] :int) | 26 | +;; Not variadic: open(2)'s third argument exists only for O_CREAT, which a |
| 27 | +;; device node never wants. Declaring the tail would oblige every call to | ||
| 28 | +;; pass a mode that the kernel then ignores. | ||
| 29 | +(ffi/defcfn c-open "open" [:string :int] :int) | ||
| 26 | (ffi/defcfn c-close "close" [:int] :int) | 30 | (ffi/defcfn c-close "close" [:int] :int) |
| 27 | ;; ioctl's third argument is whatever the request says it is; for every | 31 | ;; ioctl's third argument is whatever the request says it is; for every |
| 28 | ;; request here it is a pointer, and the declared tail costs no compile. | 32 | ;; request here it is a pointer, and the declared tail costs no compile. |
| @@ -258,6 +262,42 @@ | |||
| 258 | (f (:ptr buf) n) | 262 | (f (:ptr buf) n) |
| 259 | (finally (queue! fd i))))))) | 263 | (finally (queue! fd i))))))) |
| 260 | 264 | ||
| 265 | +(defn devices | ||
| 266 | + "Every /dev/videoN that can actually capture, as {:id :name :default?}. | ||
| 267 | + | ||
| 268 | + Probed rather than listed: reading the directory would mean binding | ||
| 269 | + opendir and readdir for a range the kernel keeps small anyway, and a node | ||
| 270 | + that cannot be opened is one this process could not have used regardless. | ||
| 271 | + | ||
| 272 | + The filter matters more than it looks. A single camera usually presents | ||
| 273 | + SEVERAL /dev/video nodes — the capture node beside metadata and control | ||
| 274 | + nodes — and only one of them streams. Offering the others is how a device | ||
| 275 | + list ends up with entries that fail the moment they are picked, so | ||
| 276 | + QUERYCAP's device_caps decides rather than the node existing." | ||
| 277 | + [] | ||
| 278 | + (->> (range 64) | ||
| 279 | + (keep (fn [i] | ||
| 280 | + (let [path (str "/dev/video" i) | ||
| 281 | + fd (c-open path o-rdwr)] | ||
| 282 | + (when-not (neg? fd) | ||
| 283 | + (try | ||
| 284 | + (let [caps (capabilities fd)] | ||
| 285 | + (when (and (:capture? caps) (:streaming? caps)) | ||
| 286 | + (ffi/with-arena [a] | ||
| 287 | + (let [p (ffi/alloc a (ffi/layout-size capability))] | ||
| 288 | + (ioctl! fd VIDIOC_QUERYCAP p "QUERYCAP") | ||
| 289 | + (let [card (ffi/read-bytes | ||
| 290 | + (+ p (ffi/field-offset capability [:card])) 32) | ||
| 291 | + ;; The kernel pads `card` with NULs to 32 | ||
| 292 | + ;; bytes; a jolt string of the whole field | ||
| 293 | + ;; would carry them. | ||
| 294 | + name (first (str/split card #"\x00"))] | ||
| 295 | + {:id path | ||
| 296 | + :name (if (seq name) name path) | ||
| 297 | + :default? (zero? i)}))))) | ||
| 298 | + (finally (c-close fd))))))) | ||
| 299 | + vec)) | ||
| 300 | + | ||
| 261 | (defn close-device! [fd buffers] | 301 | (defn close-device! [fd buffers] |
| 262 | (doseq [{:keys [ptr len]} buffers] (c-munmap ptr len)) | 302 | (doseq [{:keys [ptr len]} buffers] (c-munmap ptr len)) |
| 263 | (c-close fd) | 303 | (c-close fd) |
modified
src/frq/codec/h264.clj +47 -0 | @@ -31,6 +31,11 @@ | ||
| 31 | 31 | (ffi/defcfn raw-force-keyframe "frq_h264_force_keyframe" [:pointer] :int) |
| 32 | 32 | (ffi/defcfn raw-close "frq_h264_close" [:pointer] :void) |
| 33 | 33 | |
| 34 | +(ffi/defcfn raw-decoder-open "frq_h264_decoder_open" [:pointer] :int) | |
| 35 | +(ffi/defcfn raw-decode-rgba "frq_h264_decode_rgba" | |
| 36 | + [:pointer :pointer :int :pointer :pointer :pointer] :int) | |
| 37 | +(ffi/defcfn raw-decoder-close "frq_h264_decoder_close" [:pointer] :void) | |
| 38 | + | |
| 34 | 39 | (defn i420-size |
| 35 | 40 | "Bytes in one I420 frame: a luma plane, then two at quarter resolution." |
| 36 | 41 | [width height] |
| @@ -82,3 +87,45 @@ | ||
| 82 | 87 | nil) |
| 83 | 88 | |
| 84 | 89 | (defn close! [enc] (raw-close enc) nil) |
| 90 | + | |
| 91 | +;; --- decoding ---------------------------------------------------------------- | |
| 92 | + | |
| 93 | +(defn decoder | |
| 94 | + "Open a decoder. It answers RGBA, not I420 — see `decode!`." | |
| 95 | + [] | |
| 96 | + (ffi/with-arena [a] | |
| 97 | + (let [out (ffi/alloc a 8) | |
| 98 | + rc (raw-decoder-open out)] | |
| 99 | + (when-not (zero? rc) | |
| 100 | + (throw (ex-info "openh264: could not open a decoder" {:code rc}))) | |
| 101 | + (ffi/read out :pointer)))) | |
| 102 | + | |
| 103 | +(defn decode! | |
| 104 | + "Decode one Annex B frame and hand the picture to `use-frame`. | |
| 105 | + | |
| 106 | + `use-frame` is called with [pointer width height] and its value answered. | |
| 107 | + The pointer is tightly packed RGBA in the decoder's own buffer, valid until | |
| 108 | + the next decode on the same decoder. | |
| 109 | + | |
| 110 | + RGBA rather than I420 on purpose: `vidya/frame-rgba!` is where this is | |
| 111 | + going, so the conversion happens once in C rather than dragging three | |
| 112 | + planes and their strides across into jolt to be rearranged. Those strides | |
| 113 | + are also why it happens there — a decoder pads its rows, and `stride` is | |
| 114 | + not `width`. | |
| 115 | + | |
| 116 | + A decoder with no picture yet — normal for the first packets of a stream — | |
| 117 | + calls `use-frame` with a NULL pointer and zero dimensions rather than | |
| 118 | + raising." | |
| 119 | + [dec annexb len use-frame] | |
| 120 | + (ffi/with-arena [a] | |
| 121 | + (let [out (ffi/alloc a 8) | |
| 122 | + w (ffi/alloc a 4) | |
| 123 | + h (ffi/alloc a 4) | |
| 124 | + rc (raw-decode-rgba dec annexb len out w h)] | |
| 125 | + (when-not (zero? rc) | |
| 126 | + (throw (ex-info "openh264: decode failed" {:code rc}))) | |
| 127 | + (use-frame (ffi/read out :pointer) | |
| 128 | + (ffi/read w :int32) | |
| 129 | + (ffi/read h :int32))))) | |
| 130 | + | |
| 131 | +(defn close-decoder! [dec] (raw-decoder-close dec) nil) | |
| @@ -31,6 +31,11 @@ | |||
| 31 | (ffi/defcfn raw-force-keyframe "frq_h264_force_keyframe" [:pointer] :int) | 31 | (ffi/defcfn raw-force-keyframe "frq_h264_force_keyframe" [:pointer] :int) |
| 32 | (ffi/defcfn raw-close "frq_h264_close" [:pointer] :void) | 32 | (ffi/defcfn raw-close "frq_h264_close" [:pointer] :void) |
| 33 | 33 | ||
| 34 | +(ffi/defcfn raw-decoder-open "frq_h264_decoder_open" [:pointer] :int) | ||
| 35 | +(ffi/defcfn raw-decode-rgba "frq_h264_decode_rgba" | ||
| 36 | + [:pointer :pointer :int :pointer :pointer :pointer] :int) | ||
| 37 | +(ffi/defcfn raw-decoder-close "frq_h264_decoder_close" [:pointer] :void) | ||
| 38 | + | ||
| 34 | (defn i420-size | 39 | (defn i420-size |
| 35 | "Bytes in one I420 frame: a luma plane, then two at quarter resolution." | 40 | "Bytes in one I420 frame: a luma plane, then two at quarter resolution." |
| 36 | [width height] | 41 | [width height] |
| @@ -82,3 +87,45 @@ | |||
| 82 | nil) | 87 | nil) |
| 83 | 88 | ||
| 84 | (defn close! [enc] (raw-close enc) nil) | 89 | (defn close! [enc] (raw-close enc) nil) |
| 90 | + | ||
| 91 | +;; --- decoding ---------------------------------------------------------------- | ||
| 92 | + | ||
| 93 | +(defn decoder | ||
| 94 | + "Open a decoder. It answers RGBA, not I420 — see `decode!`." | ||
| 95 | + [] | ||
| 96 | + (ffi/with-arena [a] | ||
| 97 | + (let [out (ffi/alloc a 8) | ||
| 98 | + rc (raw-decoder-open out)] | ||
| 99 | + (when-not (zero? rc) | ||
| 100 | + (throw (ex-info "openh264: could not open a decoder" {:code rc}))) | ||
| 101 | + (ffi/read out :pointer)))) | ||
| 102 | + | ||
| 103 | +(defn decode! | ||
| 104 | + "Decode one Annex B frame and hand the picture to `use-frame`. | ||
| 105 | + | ||
| 106 | + `use-frame` is called with [pointer width height] and its value answered. | ||
| 107 | + The pointer is tightly packed RGBA in the decoder's own buffer, valid until | ||
| 108 | + the next decode on the same decoder. | ||
| 109 | + | ||
| 110 | + RGBA rather than I420 on purpose: `vidya/frame-rgba!` is where this is | ||
| 111 | + going, so the conversion happens once in C rather than dragging three | ||
| 112 | + planes and their strides across into jolt to be rearranged. Those strides | ||
| 113 | + are also why it happens there — a decoder pads its rows, and `stride` is | ||
| 114 | + not `width`. | ||
| 115 | + | ||
| 116 | + A decoder with no picture yet — normal for the first packets of a stream — | ||
| 117 | + calls `use-frame` with a NULL pointer and zero dimensions rather than | ||
| 118 | + raising." | ||
| 119 | + [dec annexb len use-frame] | ||
| 120 | + (ffi/with-arena [a] | ||
| 121 | + (let [out (ffi/alloc a 8) | ||
| 122 | + w (ffi/alloc a 4) | ||
| 123 | + h (ffi/alloc a 4) | ||
| 124 | + rc (raw-decode-rgba dec annexb len out w h)] | ||
| 125 | + (when-not (zero? rc) | ||
| 126 | + (throw (ex-info "openh264: decode failed" {:code rc}))) | ||
| 127 | + (use-frame (ffi/read out :pointer) | ||
| 128 | + (ffi/read w :int32) | ||
| 129 | + (ffi/read h :int32))))) | ||
| 130 | + | ||
| 131 | +(defn close-decoder! [dec] (raw-decoder-close dec) nil) | ||
modified
src/frq/moq/smoke.clj +62 -1 | @@ -295,6 +295,42 @@ | ||
| 295 | 295 | (throw (ex-info "not Annex B — no start code" {:first-4 (:first-4 got)}))) |
| 296 | 296 | (when-not (:keyframe got) |
| 297 | 297 | (throw (ex-info "first frame is not an IDR" {}))) |
| 298 | + | |
| 299 | + ;; And back again. A decode that answered the right SIZE full of | |
| 300 | + ;; the wrong pixels would pass a length check, so the assertion is | |
| 301 | + ;; the picture: flat gray in, flat gray out, opaque. | |
| 302 | + (let [dec (h264/decoder)] | |
| 303 | + (try | |
| 304 | + ;; The frame above was the IDR; this one would be a P-frame | |
| 305 | + ;; referencing it, and a decoder handed that first answers | |
| 306 | + ;; dsNoParamSets (16) — no SPS or PPS to decode against. A | |
| 307 | + ;; subscriber joining mid-call is in exactly that position, | |
| 308 | + ;; which is what force-keyframe! is for. | |
| 309 | + (h264/force-keyframe! enc) | |
| 310 | + (h264/encode! enc px 33333 | |
| 311 | + (fn [p len _] | |
| 312 | + (h264/decode! dec p len | |
| 313 | + (fn [rgba dw dh] | |
| 314 | + (println " decoded ->" dw "x" dh "RGBA") | |
| 315 | + (when (or (zero? dw) (ffi/null? rgba)) | |
| 316 | + (throw (ex-info "decoder produced no picture" {}))) | |
| 317 | + (when-not (and (= w dw) (= h dh)) | |
| 318 | + (throw (ex-info "decoded size does not match" | |
| 319 | + {:want [w h] :got [dw dh]}))) | |
| 320 | + (let [px0 (mapv #(ffi/read (+ rgba %) :uint8) (range 4)) | |
| 321 | + mid (* 4 (+ (* (quot dh 2) dw) (quot dw 2))) | |
| 322 | + pxm (mapv #(ffi/read (+ rgba mid %) :uint8) (range 4))] | |
| 323 | + (println " first pixel" (pr-str px0) "centre" (pr-str pxm)) | |
| 324 | + (when-not (= 255 (nth px0 3)) | |
| 325 | + (throw (ex-info "alpha is not opaque" {:pixel px0}))) | |
| 326 | + ;; 0x80 luma with neutral chroma is mid gray; the | |
| 327 | + ;; BT.601 maths lands near 125, not exactly 128. | |
| 328 | + (doseq [c (take 3 pxm)] | |
| 329 | + (when-not (< 100 c 150) | |
| 330 | + (throw (ex-info "centre pixel is not gray" | |
| 331 | + {:pixel pxm}))))) | |
| 332 | + true)))) | |
| 333 | + (finally (h264/close-decoder! dec)))) | |
| 298 | 334 | true) |
| 299 | 335 | (finally (h264/close! enc)))))) |
| 300 | 336 | |
| @@ -368,6 +404,30 @@ | ||
| 368 | 404 | true) |
| 369 | 405 | (finally (alsa/close! pcm)))))) |
| 370 | 406 | |
| 407 | +(defn- check-enumeration | |
| 408 | + "List the devices, which is what av.clj's cameras/microphones/speakers are. | |
| 409 | + | |
| 410 | + There is no assertion on the CONTENTS: this container has no camera, and | |
| 411 | + which PCMs ALSA offers is a property of the machine rather than of this | |
| 412 | + code. What is asserted is that enumeration returns without leaking or | |
| 413 | + faulting and that every entry is shaped the way av.clj's `parse-devices` | |
| 414 | + produced them — an :id to pass back, a :name to show, a :default? flag — | |
| 415 | + because that shape is the contract the UI already reads." | |
| 416 | + [] | |
| 417 | + (let [shaped? (fn [d] (and (string? (:id d)) (string? (:name d)) | |
| 418 | + (contains? d :default?))) | |
| 419 | + cams (v4l2/devices) | |
| 420 | + mics (alsa/devices :capture) | |
| 421 | + outs (alsa/devices :playback)] | |
| 422 | + (println " cameras:" (count cams) (pr-str (mapv :name (take 2 cams)))) | |
| 423 | + (println " capture:" (count mics) (pr-str (mapv :name (take 2 mics)))) | |
| 424 | + (println " playback:" (count outs) (pr-str (mapv :name (take 2 outs)))) | |
| 425 | + (doseq [[what ds] [["camera" cams] ["capture" mics] ["playback" outs]]] | |
| 426 | + (when-let [bad (first (remove shaped? ds))] | |
| 427 | + (throw (ex-info (str what " device is not shaped like av.clj expects") | |
| 428 | + {:device bad})))) | |
| 429 | + true)) | |
| 430 | + | |
| 371 | 431 | (defn -main [& _] |
| 372 | 432 | (println "libmoq_ffi smoke test") |
| 373 | 433 | (let [steps [["contract" check-contract] |
| @@ -378,7 +438,8 @@ | ||
| 378 | 438 | ["opus" check-opus] |
| 379 | 439 | ["h264" check-h264] |
| 380 | 440 | ["v4l2" check-v4l2-layouts] |
| 381 | - ["alsa" check-alsa]]] | |
| 441 | + ["alsa" check-alsa] | |
| 442 | + ["devices" check-enumeration]]] | |
| 382 | 443 | (doseq [[name f] steps] |
| 383 | 444 | (println (str name ":")) |
| 384 | 445 | (f)) |
| @@ -295,6 +295,42 @@ | |||
| 295 | (throw (ex-info "not Annex B — no start code" {:first-4 (:first-4 got)}))) | 295 | (throw (ex-info "not Annex B — no start code" {:first-4 (:first-4 got)}))) |
| 296 | (when-not (:keyframe got) | 296 | (when-not (:keyframe got) |
| 297 | (throw (ex-info "first frame is not an IDR" {}))) | 297 | (throw (ex-info "first frame is not an IDR" {}))) |
| 298 | + | ||
| 299 | + ;; And back again. A decode that answered the right SIZE full of | ||
| 300 | + ;; the wrong pixels would pass a length check, so the assertion is | ||
| 301 | + ;; the picture: flat gray in, flat gray out, opaque. | ||
| 302 | + (let [dec (h264/decoder)] | ||
| 303 | + (try | ||
| 304 | + ;; The frame above was the IDR; this one would be a P-frame | ||
| 305 | + ;; referencing it, and a decoder handed that first answers | ||
| 306 | + ;; dsNoParamSets (16) — no SPS or PPS to decode against. A | ||
| 307 | + ;; subscriber joining mid-call is in exactly that position, | ||
| 308 | + ;; which is what force-keyframe! is for. | ||
| 309 | + (h264/force-keyframe! enc) | ||
| 310 | + (h264/encode! enc px 33333 | ||
| 311 | + (fn [p len _] | ||
| 312 | + (h264/decode! dec p len | ||
| 313 | + (fn [rgba dw dh] | ||
| 314 | + (println " decoded ->" dw "x" dh "RGBA") | ||
| 315 | + (when (or (zero? dw) (ffi/null? rgba)) | ||
| 316 | + (throw (ex-info "decoder produced no picture" {}))) | ||
| 317 | + (when-not (and (= w dw) (= h dh)) | ||
| 318 | + (throw (ex-info "decoded size does not match" | ||
| 319 | + {:want [w h] :got [dw dh]}))) | ||
| 320 | + (let [px0 (mapv #(ffi/read (+ rgba %) :uint8) (range 4)) | ||
| 321 | + mid (* 4 (+ (* (quot dh 2) dw) (quot dw 2))) | ||
| 322 | + pxm (mapv #(ffi/read (+ rgba mid %) :uint8) (range 4))] | ||
| 323 | + (println " first pixel" (pr-str px0) "centre" (pr-str pxm)) | ||
| 324 | + (when-not (= 255 (nth px0 3)) | ||
| 325 | + (throw (ex-info "alpha is not opaque" {:pixel px0}))) | ||
| 326 | + ;; 0x80 luma with neutral chroma is mid gray; the | ||
| 327 | + ;; BT.601 maths lands near 125, not exactly 128. | ||
| 328 | + (doseq [c (take 3 pxm)] | ||
| 329 | + (when-not (< 100 c 150) | ||
| 330 | + (throw (ex-info "centre pixel is not gray" | ||
| 331 | + {:pixel pxm}))))) | ||
| 332 | + true)))) | ||
| 333 | + (finally (h264/close-decoder! dec)))) | ||
| 298 | true) | 334 | true) |
| 299 | (finally (h264/close! enc)))))) | 335 | (finally (h264/close! enc)))))) |
| 300 | 336 | ||
| @@ -368,6 +404,30 @@ | |||
| 368 | true) | 404 | true) |
| 369 | (finally (alsa/close! pcm)))))) | 405 | (finally (alsa/close! pcm)))))) |
| 370 | 406 | ||
| 407 | +(defn- check-enumeration | ||
| 408 | + "List the devices, which is what av.clj's cameras/microphones/speakers are. | ||
| 409 | + | ||
| 410 | + There is no assertion on the CONTENTS: this container has no camera, and | ||
| 411 | + which PCMs ALSA offers is a property of the machine rather than of this | ||
| 412 | + code. What is asserted is that enumeration returns without leaking or | ||
| 413 | + faulting and that every entry is shaped the way av.clj's `parse-devices` | ||
| 414 | + produced them — an :id to pass back, a :name to show, a :default? flag — | ||
| 415 | + because that shape is the contract the UI already reads." | ||
| 416 | + [] | ||
| 417 | + (let [shaped? (fn [d] (and (string? (:id d)) (string? (:name d)) | ||
| 418 | + (contains? d :default?))) | ||
| 419 | + cams (v4l2/devices) | ||
| 420 | + mics (alsa/devices :capture) | ||
| 421 | + outs (alsa/devices :playback)] | ||
| 422 | + (println " cameras:" (count cams) (pr-str (mapv :name (take 2 cams)))) | ||
| 423 | + (println " capture:" (count mics) (pr-str (mapv :name (take 2 mics)))) | ||
| 424 | + (println " playback:" (count outs) (pr-str (mapv :name (take 2 outs)))) | ||
| 425 | + (doseq [[what ds] [["camera" cams] ["capture" mics] ["playback" outs]]] | ||
| 426 | + (when-let [bad (first (remove shaped? ds))] | ||
| 427 | + (throw (ex-info (str what " device is not shaped like av.clj expects") | ||
| 428 | + {:device bad})))) | ||
| 429 | + true)) | ||
| 430 | + | ||
| 371 | (defn -main [& _] | 431 | (defn -main [& _] |
| 372 | (println "libmoq_ffi smoke test") | 432 | (println "libmoq_ffi smoke test") |
| 373 | (let [steps [["contract" check-contract] | 433 | (let [steps [["contract" check-contract] |
| @@ -378,7 +438,8 @@ | |||
| 378 | ["opus" check-opus] | 438 | ["opus" check-opus] |
| 379 | ["h264" check-h264] | 439 | ["h264" check-h264] |
| 380 | ["v4l2" check-v4l2-layouts] | 440 | ["v4l2" check-v4l2-layouts] |
| 381 | - ["alsa" check-alsa]]] | 441 | + ["alsa" check-alsa] |
| 442 | + ["devices" check-enumeration]]] | ||
| 382 | (doseq [[name f] steps] | 443 | (doseq [[name f] steps] |
| 383 | (println (str name ":")) | 444 | (println (str name ":")) |
| 384 | (f)) | 445 | (f)) |