rickub
Sign in Get started
nandi/frqpublic⑂ Fork 0
⑂ 7a293a9
Branches
claude/add-remote-repository-47d12e main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/frq.git
git clone ssh://git@rickub.com/nandi/frq.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Encode a frame to H.264, and never let it become a jolt value

publish_video opens an ENCODER rather than a track -- an unsupported codec,
resolution or backend fails there instead of on the first frame -- and this
is the half the fetched object could not do at all: without moq-ffi's video
feature there is no publish_video to call. So frq.moq.smoke's video check is
also what says the built object is the one being loaded. Sixty-four squares of
flat gray in, fifty-one bytes of H.264 out, keyframe, Annex B start code at
the front, which is openh264 having actually run rather than a lowering bug in
a plausible disguise.

The frame path keeps frq.av's contract instead of bending it. write-video!
takes [pointer length] and copies foreign memory straight into the buffer;
lift-media-frame hands the payload back as a BORROWED span, valid only inside
the call that reads it. Neither direction turns a frame into a jolt value,
which is the rule av.clj states in as many words -- from the decoder's buffer
to the texture as a pointer, never copied on this side. The string payload
that frq.moq.media had until now was a debugging affordance and said so; this
is what replaces it for a real frame. The codec grew :bytes for exactly that,
beside :u32 for the encoder's dimensions.

One more container trap, found the way the others were. A video track
publishes as LEGACY, and subscribing to one with LOC answers `mux: loc:
malformed loc properties` -- a message about properties, from a mismatch about
containers. It is written down as video-container, with the note that reading
the catalog is the honest way to learn it rather than knowing it.

Audio is bound and untested: publish_audio and moqaudioproducer_write are the
mirror of this and want an OpusHead up front, which video does not because it
resolves in band.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nandi committed 2026-09-10T00:52:15-07:00 Browse files
7a293a9 parent: eab184b
modified src/frq/moq/media.clj +101 -0
@@ -239,3 +239,104 @@
239239 :keyframe (uniffi/r-bool! c)}))))]
240240 (uniffi/with-out-status #(raw/rustbuffer-free rb-ptr %))
241241 v))
242+
243+;; --- video -------------------------------------------------------------------
244+;; This is the half the fetched object could not do. `publish_video` opens an
245+;; ENCODER — an unsupported codec, resolution or backend fails there rather
246+;; than on the first frame — and `write` takes RAW frames, which the track
247+;; carries as H.264.
248+;;
249+;; Note the container: a video track publishes as LEGACY, and subscribing to
250+;; one with LOC answers `mux: loc: malformed loc properties` rather than
251+;; anything about containers. The catalog is the honest way to learn it;
252+;; `video-container` is what this namespace knows until it reads one.
253+
254+(def pixel-formats {:i420 1 :rgba 2})
255+(def video-codecs {:h264 1 :h265 2})
256+(def encoder-kinds {:auto 1 :hardware 2 :software 3})
257+
258+(def video-container
259+ "The container `publish_video` publishes into."
260+ :legacy)
261+
262+(defn- enum-ops [table k what]
263+ [[:i32 (or (table k)
264+ (throw (ex-info (str "unknown " what) {:got k :known (keys table)})))]])
265+
266+(defn publish-video!
267+ "Open an encoder on this broadcast and answer its MoqVideoProducer.
268+
269+ `input` describes the raw frames going in — pixel format, width, height and
270+ framerate. `output` describes the track coming out: codec, an optional track
271+ name (nil derives one from the codec), an optional bitrate and GOP, and
272+ which encoder to use. :software is openh264, which moq-video vendors and
273+ links statically, so it is the one that works everywhere."
274+ [broadcast {:keys [format width height framerate]}
275+ {:keys [codec kind bitrate gop track]
276+ :or {codec :h264 kind :software}}]
277+ (let [h (uniffi/with-out-status #(raw/clone-moqbroadcastproducer broadcast %))]
278+ (ffi/with-arena [a]
279+ (let [cell #(ffi/alloc a (ffi/layout-size uniffi/rust-buffer))
280+ in (uniffi/lower-buffer (cell)
281+ (concat (enum-ops pixel-formats format "MoqVideoPixelFormat")
282+ [[:u32 width] [:u32 height] [:u32 framerate]]))
283+ out (uniffi/lower-buffer (cell)
284+ (concat (enum-ops video-codecs codec "MoqVideoCodec")
285+ (if track [[:u8 1] [:string track]] [[:u8 0]])
286+ (if bitrate [[:u8 1] [:u64 bitrate]] [[:u8 0]])
287+ (if gop [[:u8 1] [:u32 gop]] [[:u8 0]])
288+ (enum-ops encoder-kinds kind "MoqVideoEncoderKind")))]
289+ (uniffi/with-out-status
290+ #(raw/method-moqbroadcastproducer-publish-video h in out %))))))
291+
292+(defn video-name
293+ "The track name the encoder chose."
294+ [producer]
295+ (let [h (uniffi/with-out-status #(raw/clone-moqvideoproducer producer %))]
296+ (ffi/with-arena [a]
297+ (let [out (ffi/alloc a (ffi/layout-size uniffi/rust-buffer))]
298+ (uniffi/with-out-status #(raw/method-moqvideoproducer-name out h %))
299+ (uniffi/lift-string out)))))
300+
301+(defn write-video!
302+ "Encode one raw frame. `pixels` is [pointer length] — foreign memory, copied
303+ straight into the buffer without ever becoming a jolt value."
304+ [producer timestamp-us pixels]
305+ (let [h (uniffi/with-out-status #(raw/clone-moqvideoproducer producer %))]
306+ (lowered [[:u64 timestamp-us] [:bytes pixels]]
307+ (fn [buf]
308+ (uniffi/with-out-status
309+ #(raw/method-moqvideoproducer-write h buf %)))))
310+ nil)
311+
312+(defn set-video-bitrate!
313+ [producer bits-per-second]
314+ (let [h (uniffi/with-out-status #(raw/clone-moqvideoproducer producer %))]
315+ (uniffi/with-out-status
316+ #(raw/method-moqvideoproducer-set-bitrate h bits-per-second %)))
317+ nil)
318+
319+(defn lift-media-frame
320+ "Read an Optional<MoqMediaFrame>, handing the payload to `use-payload` as a
321+ BORROWED [pointer length] span.
322+
323+ The span is valid only inside this call: the buffer is freed on the way out.
324+ That is the shape on purpose — it is what lets an encoded frame go straight
325+ on to a texture, the way frq.av already moves one, instead of being copied
326+ into a jolt value first."
327+ [rb-ptr use-payload]
328+ (let [len (ffi/read-field rb-ptr uniffi/rust-buffer [:len])
329+ data (ffi/read-field rb-ptr uniffi/rust-buffer [:data])
330+ v (when (and (pos? len) (not (ffi/null? data)))
331+ (let [c (uniffi/reader data len)]
332+ (uniffi/r-optional!
333+ c (fn [c]
334+ (let [span (uniffi/r-bytes-span! c)
335+ ts (uniffi/r-u64! c)
336+ kf (uniffi/r-bool! c)]
337+ {:payload (use-payload (:ptr span) (:len span))
338+ :bytes (:len span)
339+ :timestamp-us ts
340+ :keyframe kf})))))]
341+ (uniffi/with-out-status #(raw/rustbuffer-free rb-ptr %))
342+ v))
@@ -239,3 +239,104 @@
239 :keyframe (uniffi/r-bool! c)}))))]239 :keyframe (uniffi/r-bool! c)}))))]
240 (uniffi/with-out-status #(raw/rustbuffer-free rb-ptr %))240 (uniffi/with-out-status #(raw/rustbuffer-free rb-ptr %))
241 v))241 v))
242+
243+;; --- video -------------------------------------------------------------------
244+;; This is the half the fetched object could not do. `publish_video` opens an
245+;; ENCODER — an unsupported codec, resolution or backend fails there rather
246+;; than on the first frame — and `write` takes RAW frames, which the track
247+;; carries as H.264.
248+;;
249+;; Note the container: a video track publishes as LEGACY, and subscribing to
250+;; one with LOC answers `mux: loc: malformed loc properties` rather than
251+;; anything about containers. The catalog is the honest way to learn it;
252+;; `video-container` is what this namespace knows until it reads one.
253+
254+(def pixel-formats {:i420 1 :rgba 2})
255+(def video-codecs {:h264 1 :h265 2})
256+(def encoder-kinds {:auto 1 :hardware 2 :software 3})
257+
258+(def video-container
259+ "The container `publish_video` publishes into."
260+ :legacy)
261+
262+(defn- enum-ops [table k what]
263+ [[:i32 (or (table k)
264+ (throw (ex-info (str "unknown " what) {:got k :known (keys table)})))]])
265+
266+(defn publish-video!
267+ "Open an encoder on this broadcast and answer its MoqVideoProducer.
268+
269+ `input` describes the raw frames going in — pixel format, width, height and
270+ framerate. `output` describes the track coming out: codec, an optional track
271+ name (nil derives one from the codec), an optional bitrate and GOP, and
272+ which encoder to use. :software is openh264, which moq-video vendors and
273+ links statically, so it is the one that works everywhere."
274+ [broadcast {:keys [format width height framerate]}
275+ {:keys [codec kind bitrate gop track]
276+ :or {codec :h264 kind :software}}]
277+ (let [h (uniffi/with-out-status #(raw/clone-moqbroadcastproducer broadcast %))]
278+ (ffi/with-arena [a]
279+ (let [cell #(ffi/alloc a (ffi/layout-size uniffi/rust-buffer))
280+ in (uniffi/lower-buffer (cell)
281+ (concat (enum-ops pixel-formats format "MoqVideoPixelFormat")
282+ [[:u32 width] [:u32 height] [:u32 framerate]]))
283+ out (uniffi/lower-buffer (cell)
284+ (concat (enum-ops video-codecs codec "MoqVideoCodec")
285+ (if track [[:u8 1] [:string track]] [[:u8 0]])
286+ (if bitrate [[:u8 1] [:u64 bitrate]] [[:u8 0]])
287+ (if gop [[:u8 1] [:u32 gop]] [[:u8 0]])
288+ (enum-ops encoder-kinds kind "MoqVideoEncoderKind")))]
289+ (uniffi/with-out-status
290+ #(raw/method-moqbroadcastproducer-publish-video h in out %))))))
291+
292+(defn video-name
293+ "The track name the encoder chose."
294+ [producer]
295+ (let [h (uniffi/with-out-status #(raw/clone-moqvideoproducer producer %))]
296+ (ffi/with-arena [a]
297+ (let [out (ffi/alloc a (ffi/layout-size uniffi/rust-buffer))]
298+ (uniffi/with-out-status #(raw/method-moqvideoproducer-name out h %))
299+ (uniffi/lift-string out)))))
300+
301+(defn write-video!
302+ "Encode one raw frame. `pixels` is [pointer length] — foreign memory, copied
303+ straight into the buffer without ever becoming a jolt value."
304+ [producer timestamp-us pixels]
305+ (let [h (uniffi/with-out-status #(raw/clone-moqvideoproducer producer %))]
306+ (lowered [[:u64 timestamp-us] [:bytes pixels]]
307+ (fn [buf]
308+ (uniffi/with-out-status
309+ #(raw/method-moqvideoproducer-write h buf %)))))
310+ nil)
311+
312+(defn set-video-bitrate!
313+ [producer bits-per-second]
314+ (let [h (uniffi/with-out-status #(raw/clone-moqvideoproducer producer %))]
315+ (uniffi/with-out-status
316+ #(raw/method-moqvideoproducer-set-bitrate h bits-per-second %)))
317+ nil)
318+
319+(defn lift-media-frame
320+ "Read an Optional<MoqMediaFrame>, handing the payload to `use-payload` as a
321+ BORROWED [pointer length] span.
322+
323+ The span is valid only inside this call: the buffer is freed on the way out.
324+ That is the shape on purpose — it is what lets an encoded frame go straight
325+ on to a texture, the way frq.av already moves one, instead of being copied
326+ into a jolt value first."
327+ [rb-ptr use-payload]
328+ (let [len (ffi/read-field rb-ptr uniffi/rust-buffer [:len])
329+ data (ffi/read-field rb-ptr uniffi/rust-buffer [:data])
330+ v (when (and (pos? len) (not (ffi/null? data)))
331+ (let [c (uniffi/reader data len)]
332+ (uniffi/r-optional!
333+ c (fn [c]
334+ (let [span (uniffi/r-bytes-span! c)
335+ ts (uniffi/r-u64! c)
336+ kf (uniffi/r-bool! c)]
337+ {:payload (use-payload (:ptr span) (:len span))
338+ :bytes (:len span)
339+ :timestamp-us ts
340+ :keyframe kf})))))]
341+ (uniffi/with-out-status #(raw/rustbuffer-free rb-ptr %))
342+ v))
modified src/frq/moq/smoke.clj +66 -1
@@ -194,13 +194,78 @@
194194 (throw (ex-info "timestamp did not survive" {:frame frame})))
195195 true))))
196196
197+(defn- gray-i420
198+ "A flat mid-gray I420 frame in FOREIGN memory; answers [pointer length].
199+
200+ I420 is one luma byte per pixel followed by two quarter-resolution chroma
201+ planes, so w*h*3/2 in total. 0x80 across all three is neutral gray — an
202+ image with nothing in it, which is the point: what is under test is the
203+ encoder accepting a well-formed frame, not what it makes of the picture.
204+
205+ It is built with ffi/write rather than as a jolt value that gets copied in,
206+ because that is the shape a real capture buffer arrives in."
207+ [a w h]
208+ (let [n (+ (* w h) (* 2 (quot (* w h) 4)))
209+ p (ffi/alloc a n)]
210+ (dotimes [i n] (ffi/write (+ p i) :uint8 0x80))
211+ [p n]))
212+
213+(defn- check-video
214+ "Encode a raw frame to H.264 and read the encoded frame back out.
215+
216+ This is the half the fetched object could not do at all: publish_video opens
217+ an encoder, and without the video feature there is no publish_video to
218+ call. So this is also the test that says the built object is the built one.
219+
220+ The assertion is the Annex B start code at the front of the payload. A frame
221+ that came back with the right length and the wrong bytes would be a lowering
222+ bug wearing a plausible disguise; 00 00 00 01 followed by an SPS NAL is
223+ openh264 having actually run."
224+ []
225+ (ffi/with-arena [a]
226+ (let [origin (media/new-origin)
227+ broadcast (media/create-broadcast! origin "/v")
228+ producer (media/publish-video! broadcast
229+ {:format :i420 :width 64 :height 64
230+ :framerate 30}
231+ {:codec :h264 :kind :software})
232+ track (media/video-name producer)
233+ consumer (media/broadcast-consumer broadcast)]
234+ (println " encoder open, video track:" (pr-str track))
235+ (let [mc (settle! (media/subscribe-media! consumer track media/video-container)
236+ "subscribe_media" 10000)
237+ [px n] (gray-i420 a 64 64)]
238+ (println " subscribed, wrote" n "bytes of I420")
239+ (dotimes [i 3]
240+ (media/write-video! producer (* i 33333) [px n]))
241+ (let [frame (settle! (media/next-frame! mc) "next" 15000
242+ #(media/lift-media-frame
243+ %
244+ ;; The payload never becomes a jolt value:
245+ ;; four bytes are read out of the borrowed
246+ ;; span and the span is done with.
247+ (fn [p _len]
248+ (mapv (fn [i] (ffi/read (+ p i) :uint8))
249+ (range 4)))))]
250+ (println " encoded frame:" (pr-str (dissoc frame :payload))
251+ "starts" (pr-str (:payload frame)))
252+ (when-not frame
253+ (throw (ex-info "no encoded frame arrived" {})))
254+ (when-not (= [0 0 0 1] (:payload frame))
255+ (throw (ex-info "payload is not Annex B — no start code"
256+ {:first-4 (:payload frame)})))
257+ (when-not (:keyframe frame)
258+ (throw (ex-info "first encoded frame is not a keyframe" {:frame frame})))
259+ true)))))
260+
197261 (defn -main [& _]
198262 (println "libmoq_ffi smoke test")
199263 (let [steps [["contract" check-contract]
200264 ["handle" check-handle]
201265 ["string" check-string]
202266 ["connect" check-connect]
203- ["media" check-media]]]
267+ ["media" check-media]
268+ ["video" check-video]]]
204269 (doseq [[name f] steps]
205270 (println (str name ":"))
206271 (f))
@@ -194,13 +194,78 @@
194 (throw (ex-info "timestamp did not survive" {:frame frame})))194 (throw (ex-info "timestamp did not survive" {:frame frame})))
195 true))))195 true))))
196 196
197+(defn- gray-i420
198+ "A flat mid-gray I420 frame in FOREIGN memory; answers [pointer length].
199+
200+ I420 is one luma byte per pixel followed by two quarter-resolution chroma
201+ planes, so w*h*3/2 in total. 0x80 across all three is neutral gray — an
202+ image with nothing in it, which is the point: what is under test is the
203+ encoder accepting a well-formed frame, not what it makes of the picture.
204+
205+ It is built with ffi/write rather than as a jolt value that gets copied in,
206+ because that is the shape a real capture buffer arrives in."
207+ [a w h]
208+ (let [n (+ (* w h) (* 2 (quot (* w h) 4)))
209+ p (ffi/alloc a n)]
210+ (dotimes [i n] (ffi/write (+ p i) :uint8 0x80))
211+ [p n]))
212+
213+(defn- check-video
214+ "Encode a raw frame to H.264 and read the encoded frame back out.
215+
216+ This is the half the fetched object could not do at all: publish_video opens
217+ an encoder, and without the video feature there is no publish_video to
218+ call. So this is also the test that says the built object is the built one.
219+
220+ The assertion is the Annex B start code at the front of the payload. A frame
221+ that came back with the right length and the wrong bytes would be a lowering
222+ bug wearing a plausible disguise; 00 00 00 01 followed by an SPS NAL is
223+ openh264 having actually run."
224+ []
225+ (ffi/with-arena [a]
226+ (let [origin (media/new-origin)
227+ broadcast (media/create-broadcast! origin "/v")
228+ producer (media/publish-video! broadcast
229+ {:format :i420 :width 64 :height 64
230+ :framerate 30}
231+ {:codec :h264 :kind :software})
232+ track (media/video-name producer)
233+ consumer (media/broadcast-consumer broadcast)]
234+ (println " encoder open, video track:" (pr-str track))
235+ (let [mc (settle! (media/subscribe-media! consumer track media/video-container)
236+ "subscribe_media" 10000)
237+ [px n] (gray-i420 a 64 64)]
238+ (println " subscribed, wrote" n "bytes of I420")
239+ (dotimes [i 3]
240+ (media/write-video! producer (* i 33333) [px n]))
241+ (let [frame (settle! (media/next-frame! mc) "next" 15000
242+ #(media/lift-media-frame
243+ %
244+ ;; The payload never becomes a jolt value:
245+ ;; four bytes are read out of the borrowed
246+ ;; span and the span is done with.
247+ (fn [p _len]
248+ (mapv (fn [i] (ffi/read (+ p i) :uint8))
249+ (range 4)))))]
250+ (println " encoded frame:" (pr-str (dissoc frame :payload))
251+ "starts" (pr-str (:payload frame)))
252+ (when-not frame
253+ (throw (ex-info "no encoded frame arrived" {})))
254+ (when-not (= [0 0 0 1] (:payload frame))
255+ (throw (ex-info "payload is not Annex B — no start code"
256+ {:first-4 (:payload frame)})))
257+ (when-not (:keyframe frame)
258+ (throw (ex-info "first encoded frame is not a keyframe" {:frame frame})))
259+ true)))))
260+
197 (defn -main [& _]261 (defn -main [& _]
198 (println "libmoq_ffi smoke test")262 (println "libmoq_ffi smoke test")
199 (let [steps [["contract" check-contract]263 (let [steps [["contract" check-contract]
200 ["handle" check-handle]264 ["handle" check-handle]
201 ["string" check-string]265 ["string" check-string]
202 ["connect" check-connect]266 ["connect" check-connect]
203- ["media" check-media]]]267+ ["media" check-media]
268+ ["video" check-video]]]
204 (doseq [[name f] steps]269 (doseq [[name f] steps]
205 (println (str name ":"))270 (println (str name ":"))
206 (f))271 (f))
modified src/frq/moq/uniffi.clj +30 -9
@@ -256,6 +256,19 @@
256256 (let [n (r-i32! c)]
257257 (if (zero? n) "" (ffi/read-bytes (take! c n) n))))
258258
259+(defn r-bytes-span!
260+ "An i32 byte length, then that many RAW bytes — answered as a BORROWED span,
261+ {:ptr :len}, not copied.
262+
263+ `r-string!` is the wrong tool for a payload: it decodes UTF-8, and H.264 is
264+ not text. It is also the wrong SHAPE. frq.av's rule is that a video frame
265+ goes from the decoder's buffer to the texture as a pointer and never becomes
266+ a jolt value, so what belongs here is the address, valid exactly as long as
267+ the RustBuffer it points into."
268+ [c]
269+ (let [n (r-i32! c)]
270+ {:ptr (take! c n) :len n}))
271+
259272 (defn r-optional!
260273 "A flag byte, then `f` when it is set."
261274 [c f]
@@ -270,6 +283,9 @@
270283
271284 [[:i32 3] [:u8 0]] ; MoqContainer::LOC, then Optional::None
272285
286+ A :bytes op takes [pointer length] and copies those bytes as they are, which
287+ is how a raw frame crosses without becoming a jolt string on the way.
288+
273289 Strings are materialised FIRST, in the arena, because their wire length is a
274290 UTF-8 byte count and jolt will only tell us one by encoding the string — so
275291 the total size is not known until every string has been. `string->ptr`
@@ -281,17 +297,21 @@
281297 (ffi/with-arena [a]
282298 (let [;; [op value encoded-pointer byte-count]
283299 prepared (mapv (fn [[op v]]
284- (if (= op :string)
285- (let [p (ffi/string->ptr a v)]
286- [op v p (max 0 (dec (ffi/size p)))])
300+ (case op
301+ :string (let [p (ffi/string->ptr a v)]
302+ [op v p (max 0 (dec (ffi/size p)))])
303+ ;; [pointer length], copied straight out of
304+ ;; foreign memory — never through a jolt value.
305+ :bytes [op v (first v) (second v)]
287306 [op v nil nil]))
288307 ops)
289308 n (reduce (fn [n [op _ _ k]]
290309 (+ n (case op
291310 (:u8 :bool) 1
292- :i32 4
311+ (:i32 :u32) 4
293312 :u64 8
294- :string (+ 4 k))))
313+ ;; Both carry an i32 length and then k bytes.
314+ (:string :bytes) (+ 4 k))))
295315 0 prepared)
296316 buf (ffi/alloc a (max n 1))
297317 fbs (ffi/alloc a (ffi/layout-size foreign-bytes))
@@ -310,11 +330,12 @@
310330 (if (= op :bool) (if v 1 0) v))
311331 (inc off))
312332
313- :i32 (do (put-int! off 4 v) (+ off 4))
333+ (:i32 :u32) (do (put-int! off 4 v) (+ off 4))
314334 :u64 (do (put-int! off 8 v) (+ off 8))
315- :string (do (put-int! off 4 k)
316- (when (pos? k) (ffi/copy p (+ buf off 4) k))
317- (+ off 4 k)))
335+ (:string :bytes)
336+ (do (put-int! off 4 k)
337+ (when (pos? k) (ffi/copy p (+ buf off 4) k))
338+ (+ off 4 k)))
318339 (next todo)))))
319340 (ffi/write fbs foreign-bytes {:len n :data buf})
320341 (with-out-status #(raw/rustbuffer-from-bytes dest fbs %))
@@ -256,6 +256,19 @@
256 (let [n (r-i32! c)]256 (let [n (r-i32! c)]
257 (if (zero? n) "" (ffi/read-bytes (take! c n) n))))257 (if (zero? n) "" (ffi/read-bytes (take! c n) n))))
258 258
259+(defn r-bytes-span!
260+ "An i32 byte length, then that many RAW bytes — answered as a BORROWED span,
261+ {:ptr :len}, not copied.
262+
263+ `r-string!` is the wrong tool for a payload: it decodes UTF-8, and H.264 is
264+ not text. It is also the wrong SHAPE. frq.av's rule is that a video frame
265+ goes from the decoder's buffer to the texture as a pointer and never becomes
266+ a jolt value, so what belongs here is the address, valid exactly as long as
267+ the RustBuffer it points into."
268+ [c]
269+ (let [n (r-i32! c)]
270+ {:ptr (take! c n) :len n}))
271+
259 (defn r-optional!272 (defn r-optional!
260 "A flag byte, then `f` when it is set."273 "A flag byte, then `f` when it is set."
261 [c f]274 [c f]
@@ -270,6 +283,9 @@
270 283
271 [[:i32 3] [:u8 0]] ; MoqContainer::LOC, then Optional::None284 [[:i32 3] [:u8 0]] ; MoqContainer::LOC, then Optional::None
272 285
286+ A :bytes op takes [pointer length] and copies those bytes as they are, which
287+ is how a raw frame crosses without becoming a jolt string on the way.
288+
273 Strings are materialised FIRST, in the arena, because their wire length is a289 Strings are materialised FIRST, in the arena, because their wire length is a
274 UTF-8 byte count and jolt will only tell us one by encoding the string — so290 UTF-8 byte count and jolt will only tell us one by encoding the string — so
275 the total size is not known until every string has been. `string->ptr`291 the total size is not known until every string has been. `string->ptr`
@@ -281,17 +297,21 @@
281 (ffi/with-arena [a]297 (ffi/with-arena [a]
282 (let [;; [op value encoded-pointer byte-count]298 (let [;; [op value encoded-pointer byte-count]
283 prepared (mapv (fn [[op v]]299 prepared (mapv (fn [[op v]]
284- (if (= op :string)300+ (case op
285- (let [p (ffi/string->ptr a v)]301+ :string (let [p (ffi/string->ptr a v)]
286- [op v p (max 0 (dec (ffi/size p)))])302+ [op v p (max 0 (dec (ffi/size p)))])
303+ ;; [pointer length], copied straight out of
304+ ;; foreign memory — never through a jolt value.
305+ :bytes [op v (first v) (second v)]
287 [op v nil nil]))306 [op v nil nil]))
288 ops)307 ops)
289 n (reduce (fn [n [op _ _ k]]308 n (reduce (fn [n [op _ _ k]]
290 (+ n (case op309 (+ n (case op
291 (:u8 :bool) 1310 (:u8 :bool) 1
292- :i32 4311+ (:i32 :u32) 4
293 :u64 8312 :u64 8
294- :string (+ 4 k))))313+ ;; Both carry an i32 length and then k bytes.
314+ (:string :bytes) (+ 4 k))))
295 0 prepared)315 0 prepared)
296 buf (ffi/alloc a (max n 1))316 buf (ffi/alloc a (max n 1))
297 fbs (ffi/alloc a (ffi/layout-size foreign-bytes))317 fbs (ffi/alloc a (ffi/layout-size foreign-bytes))
@@ -310,11 +330,12 @@
310 (if (= op :bool) (if v 1 0) v))330 (if (= op :bool) (if v 1 0) v))
311 (inc off))331 (inc off))
312 332
313- :i32 (do (put-int! off 4 v) (+ off 4))333+ (:i32 :u32) (do (put-int! off 4 v) (+ off 4))
314 :u64 (do (put-int! off 8 v) (+ off 8))334 :u64 (do (put-int! off 8 v) (+ off 8))
315- :string (do (put-int! off 4 k)335+ (:string :bytes)
316- (when (pos? k) (ffi/copy p (+ buf off 4) k))336+ (do (put-int! off 4 k)
317- (+ off 4 k)))337+ (when (pos? k) (ffi/copy p (+ buf off 4) k))
338+ (+ off 4 k)))
318 (next todo)))))339 (next todo)))))
319 (ffi/write fbs foreign-bytes {:len n :data buf})340 (ffi/write fbs foreign-bytes {:len n :data buf})
320 (with-out-status #(raw/rustbuffer-from-bytes dest fbs %))341 (with-out-status #(raw/rustbuffer-from-bytes dest fbs %))