Calls
Signaling is IRC, so it is written here: av-start, av-join and av-leave go out as TAGMSGs and the server's av-state broadcasts are what actually move the state. A press is optimistic — the controls appear immediately and the server settles what really happened. Losing a race to open a call is answered by joining the one that won, since the person asked to be in a call in that room and there is one. Audio and video are not IRC and are not written here. They ride MoQ through freeq's SFU, and that is libjoltmoq — Opus, H.264, capture and transport, lifted out of sleek into its own repo rather than written a second time in jolt. src/frq/av.jolt is the whole of what this client says to it. Two rules come from that side. Nothing calls back, so status and video are drained by a timer on the loop thread; and a frame is borrowed, so the decoder's buffer goes to Vidya as a pointer and is painted by an :image with a :feed. The pixels never become a jolt value and are never copied here, which is the only way thirty frames a second is affordable. Mute and deafen are separate buttons because they are separate things: a deafened microphone still carries your voice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
f31ad3d parent: 3c98eb0 modified
README.md +46 -1 | @@ -18,6 +18,7 @@ src/frq/store.jolt the saved sign-in, mode 600 in the config directory | ||
| 18 | 18 | src/frq/avatars.jolt profile pictures, by DID or handle |
| 19 | 19 | src/frq/media.jolt image links: spot them, fetch them once, cache on disk |
| 20 | 20 | src/frq/upload.jolt a pasted picture to freeq's media endpoint, as multipart |
| 21 | +src/frq/av.jolt calls: the signaling, and a handle on the media plane | |
| 21 | 22 | src/frq/clock.jolt the reader's own zone, twelve-hour times, day headings |
| 22 | 23 | src/frq/emoji.jolt the picker's catalog: every drawable emoji and its name |
| 23 | 24 | src/frq/irc.jolt IRC over TLS or TCP: parser, reader thread, SASL, PRIVMSG |
| @@ -39,6 +40,14 @@ just lib | ||
| 39 | 40 | just run |
| 40 | 41 | ``` |
| 41 | 42 | |
| 43 | +Calls need one more object, `libjoltmoq`, from | |
| 44 | +[jolt-native](https://gitlab.com/nandithebull/jolt-native). Everything else | |
| 45 | +works without it: | |
| 46 | + | |
| 47 | +```bash | |
| 48 | +just av | |
| 49 | +``` | |
| 50 | + | |
| 42 | 51 | `just run` is `jolt -M:frq` with `LD_LIBRARY_PATH` pointed at the built |
| 43 | 52 | library. It connects to `irc.freeq.at:6697` over TLS and joins `#test`. Untick |
| 44 | 53 | TLS on the connect screen (or point it at `127.0.0.1`) for a local server's |
| @@ -132,6 +141,37 @@ surface — that surface does not work on Android either, while the syscalls do. | ||
| 132 | 141 | * Conversations listed most recently opened first |
| 133 | 142 | * Bluesky avatars beside the sender, resolved from the DID freeq tags each |
| 134 | 143 | message with |
| 144 | +* Calls: a Call button opens one in a channel, a banner offers Join where | |
| 145 | + somebody already has, and in one there is mute, deafen, video and leave. Mute | |
| 146 | + and deafen are separate — a deafened microphone still carries your voice. | |
| 147 | + Whoever turns a camera on appears as a tile; the self-view is labelled You | |
| 148 | + and sits last, where it cannot push a face you are talking to off the row | |
| 149 | + | |
| 150 | +## Calls | |
| 151 | + | |
| 152 | +Signaling is IRC and lives here: `+freeq.at/av-start`, `av-join` and `av-leave` | |
| 153 | +go out as TAGMSGs and the server broadcasts `+freeq.at/av-state` back, which is | |
| 154 | +what actually moves this client's state — a press is optimistic, and the server | |
| 155 | +settles it. Losing a race to open a call (`start-collision`) is answered by | |
| 156 | +joining the call that won rather than by reporting an error, since the person | |
| 157 | +asked to be in a call in that room and there is one. | |
| 158 | + | |
| 159 | +Media is not IRC and is not here. Audio and video ride MoQ — Media over QUIC — | |
| 160 | +through freeq's SFU, and that is `libjoltmoq`: Opus, H.264, capture and | |
| 161 | +transport, lifted out of sleek rather than written a second time in jolt. | |
| 162 | +`src/frq/av.jolt` is the whole of what frq says to it, and two of its rules | |
| 163 | +shape this side: | |
| 164 | + | |
| 165 | +* **Nothing calls back.** Status and video are polled, drained by a timer that | |
| 166 | + glimmer runs on the loop thread — the only thread allowed to touch a node. | |
| 167 | +* **A frame is borrowed.** The decoder's own buffer is handed to Vidya as a | |
| 168 | + pointer and painted by an `:image` with a `:feed`. The pixels never become a | |
| 169 | + jolt value and are never copied on this side, which is the only way thirty | |
| 170 | + frames a second is affordable here. | |
| 171 | + | |
| 172 | +The SFU is dialled once the server has minted a token, not when we ask to join: | |
| 173 | +a remote SFU refuses a connection without one, and the MoQ client then retries | |
| 174 | +in a loop that looks exactly like a hang. | |
| 135 | 175 | |
| 136 | 176 | ## Limits |
| 137 | 177 | |
| @@ -145,12 +185,17 @@ surface — that surface does not work on Android either, while the syscalls do. | ||
| 145 | 185 | format, and a fetch needs TLS, so the phone shows links. The link is left in |
| 146 | 186 | place either way. |
| 147 | 187 | * **Nothing evicts the media cache.** |
| 188 | +* **Calls are desktop-only.** `libjoltmoq` is not built for Android here, and | |
| 189 | + the camera and microphone paths that are would still need the runtime | |
| 190 | + permissions the APK does not ask for. | |
| 191 | +* **One call at a time**, which is the media plane's rule and the microphone's. | |
| 192 | +* **No call is offered in a DM** — freeq's AV signaling is a channel's. | |
| 148 | 193 | * **Pasting a picture needs a sign-in and a desktop.** The upload is filed |
| 149 | 194 | under the DID of a live session, so a guest cannot make one; and it is read |
| 150 | 195 | off the clipboard through the ABI's `vidya_clipboard_image_png`, which |
| 151 | 196 | arboard backs on desktop and nothing backs on Android. It also shares |
| 152 | 197 | nothing to your PDS and posts nothing to Bluesky — those fields are opt-in |
| 153 | 198 | and this client does not send them. |
| 154 | -* **No scrollback trimming, reactions, threads, or calls.** | |
| 199 | +* **No scrollback trimming or threads.** | |
| 155 | 200 | * A sent line waits up to 200ms for the reader thread to flush it. |
| 156 | 201 | * Message lists are keyed vboxes; glimmer-vidya has no `:listbox` yet. |
| @@ -18,6 +18,7 @@ src/frq/store.jolt the saved sign-in, mode 600 in the config directory | |||
| 18 | src/frq/avatars.jolt profile pictures, by DID or handle | 18 | src/frq/avatars.jolt profile pictures, by DID or handle |
| 19 | src/frq/media.jolt image links: spot them, fetch them once, cache on disk | 19 | src/frq/media.jolt image links: spot them, fetch them once, cache on disk |
| 20 | src/frq/upload.jolt a pasted picture to freeq's media endpoint, as multipart | 20 | src/frq/upload.jolt a pasted picture to freeq's media endpoint, as multipart |
| 21 | +src/frq/av.jolt calls: the signaling, and a handle on the media plane | ||
| 21 | src/frq/clock.jolt the reader's own zone, twelve-hour times, day headings | 22 | src/frq/clock.jolt the reader's own zone, twelve-hour times, day headings |
| 22 | src/frq/emoji.jolt the picker's catalog: every drawable emoji and its name | 23 | src/frq/emoji.jolt the picker's catalog: every drawable emoji and its name |
| 23 | src/frq/irc.jolt IRC over TLS or TCP: parser, reader thread, SASL, PRIVMSG | 24 | src/frq/irc.jolt IRC over TLS or TCP: parser, reader thread, SASL, PRIVMSG |
| @@ -39,6 +40,14 @@ just lib | |||
| 39 | just run | 40 | just run |
| 40 | ``` | 41 | ``` |
| 41 | 42 | ||
| 43 | +Calls need one more object, `libjoltmoq`, from | ||
| 44 | +[jolt-native](https://gitlab.com/nandithebull/jolt-native). Everything else | ||
| 45 | +works without it: | ||
| 46 | + | ||
| 47 | +```bash | ||
| 48 | +just av | ||
| 49 | +``` | ||
| 50 | + | ||
| 42 | `just run` is `jolt -M:frq` with `LD_LIBRARY_PATH` pointed at the built | 51 | `just run` is `jolt -M:frq` with `LD_LIBRARY_PATH` pointed at the built |
| 43 | library. It connects to `irc.freeq.at:6697` over TLS and joins `#test`. Untick | 52 | library. It connects to `irc.freeq.at:6697` over TLS and joins `#test`. Untick |
| 44 | TLS on the connect screen (or point it at `127.0.0.1`) for a local server's | 53 | TLS on the connect screen (or point it at `127.0.0.1`) for a local server's |
| @@ -132,6 +141,37 @@ surface — that surface does not work on Android either, while the syscalls do. | |||
| 132 | * Conversations listed most recently opened first | 141 | * Conversations listed most recently opened first |
| 133 | * Bluesky avatars beside the sender, resolved from the DID freeq tags each | 142 | * Bluesky avatars beside the sender, resolved from the DID freeq tags each |
| 134 | message with | 143 | message with |
| 144 | +* Calls: a Call button opens one in a channel, a banner offers Join where | ||
| 145 | + somebody already has, and in one there is mute, deafen, video and leave. Mute | ||
| 146 | + and deafen are separate — a deafened microphone still carries your voice. | ||
| 147 | + Whoever turns a camera on appears as a tile; the self-view is labelled You | ||
| 148 | + and sits last, where it cannot push a face you are talking to off the row | ||
| 149 | + | ||
| 150 | +## Calls | ||
| 151 | + | ||
| 152 | +Signaling is IRC and lives here: `+freeq.at/av-start`, `av-join` and `av-leave` | ||
| 153 | +go out as TAGMSGs and the server broadcasts `+freeq.at/av-state` back, which is | ||
| 154 | +what actually moves this client's state — a press is optimistic, and the server | ||
| 155 | +settles it. Losing a race to open a call (`start-collision`) is answered by | ||
| 156 | +joining the call that won rather than by reporting an error, since the person | ||
| 157 | +asked to be in a call in that room and there is one. | ||
| 158 | + | ||
| 159 | +Media is not IRC and is not here. Audio and video ride MoQ — Media over QUIC — | ||
| 160 | +through freeq's SFU, and that is `libjoltmoq`: Opus, H.264, capture and | ||
| 161 | +transport, lifted out of sleek rather than written a second time in jolt. | ||
| 162 | +`src/frq/av.jolt` is the whole of what frq says to it, and two of its rules | ||
| 163 | +shape this side: | ||
| 164 | + | ||
| 165 | +* **Nothing calls back.** Status and video are polled, drained by a timer that | ||
| 166 | + glimmer runs on the loop thread — the only thread allowed to touch a node. | ||
| 167 | +* **A frame is borrowed.** The decoder's own buffer is handed to Vidya as a | ||
| 168 | + pointer and painted by an `:image` with a `:feed`. The pixels never become a | ||
| 169 | + jolt value and are never copied on this side, which is the only way thirty | ||
| 170 | + frames a second is affordable here. | ||
| 171 | + | ||
| 172 | +The SFU is dialled once the server has minted a token, not when we ask to join: | ||
| 173 | +a remote SFU refuses a connection without one, and the MoQ client then retries | ||
| 174 | +in a loop that looks exactly like a hang. | ||
| 135 | 175 | ||
| 136 | ## Limits | 176 | ## Limits |
| 137 | 177 | ||
| @@ -145,12 +185,17 @@ surface — that surface does not work on Android either, while the syscalls do. | |||
| 145 | format, and a fetch needs TLS, so the phone shows links. The link is left in | 185 | format, and a fetch needs TLS, so the phone shows links. The link is left in |
| 146 | place either way. | 186 | place either way. |
| 147 | * **Nothing evicts the media cache.** | 187 | * **Nothing evicts the media cache.** |
| 188 | +* **Calls are desktop-only.** `libjoltmoq` is not built for Android here, and | ||
| 189 | + the camera and microphone paths that are would still need the runtime | ||
| 190 | + permissions the APK does not ask for. | ||
| 191 | +* **One call at a time**, which is the media plane's rule and the microphone's. | ||
| 192 | +* **No call is offered in a DM** — freeq's AV signaling is a channel's. | ||
| 148 | * **Pasting a picture needs a sign-in and a desktop.** The upload is filed | 193 | * **Pasting a picture needs a sign-in and a desktop.** The upload is filed |
| 149 | under the DID of a live session, so a guest cannot make one; and it is read | 194 | under the DID of a live session, so a guest cannot make one; and it is read |
| 150 | off the clipboard through the ABI's `vidya_clipboard_image_png`, which | 195 | off the clipboard through the ABI's `vidya_clipboard_image_png`, which |
| 151 | arboard backs on desktop and nothing backs on Android. It also shares | 196 | arboard backs on desktop and nothing backs on Android. It also shares |
| 152 | nothing to your PDS and posts nothing to Bluesky — those fields are opt-in | 197 | nothing to your PDS and posts nothing to Bluesky — those fields are opt-in |
| 153 | and this client does not send them. | 198 | and this client does not send them. |
| 154 | -* **No scrollback trimming, reactions, threads, or calls.** | 199 | +* **No scrollback trimming or threads.** |
| 155 | * A sent line waits up to 200ms for the reader thread to flush it. | 200 | * A sent line waits up to 200ms for the reader thread to flush it. |
| 156 | * Message lists are keyed vboxes; glimmer-vidya has no `:listbox` yet. | 201 | * Message lists are keyed vboxes; glimmer-vidya has no `:listbox` yet. |
modified
deps.edn +10 -1 | @@ -14,9 +14,18 @@ | ||
| 14 | 14 | ;; ABI the backend binds): |
| 15 | 15 | ;; cd ../vidya && just ffi |
| 16 | 16 | ;; LD_LIBRARY_PATH=../vidya/build jolt -M:frq |
| 17 | + ;; libjoltmoq is the AV media plane — MoQ over QUIC, Opus, H.264 and capture — | |
| 18 | + ;; from gitlab.com/nandithebull/jolt-native. Calls are the one thing this | |
| 19 | + ;; client cannot do in jolt: signaling is IRC and lives in src/frq/av.jolt, | |
| 20 | + ;; but the media half is three thousand lines of codec and transport that | |
| 21 | + ;; would only be written badly a second time here. | |
| 22 | + ;; cd ../jolt-native && just build | |
| 17 | 23 | :jolt/native [{:name "vidya" |
| 18 | 24 | :darwin ["libvidya.dylib"] |
| 19 | - :linux ["libvidya.so"]}] | |
| 25 | + :linux ["libvidya.so"]} | |
| 26 | + {:name "joltmoq" | |
| 27 | + :darwin ["libjoltmoq.dylib"] | |
| 28 | + :linux ["libjoltmoq.so"]}] | |
| 20 | 29 | |
| 21 | 30 | :aliases {:frq {:main-opts ["-m" "frq.app"]}} |
| 22 | 31 | |
| @@ -14,9 +14,18 @@ | |||
| 14 | ;; ABI the backend binds): | 14 | ;; ABI the backend binds): |
| 15 | ;; cd ../vidya && just ffi | 15 | ;; cd ../vidya && just ffi |
| 16 | ;; LD_LIBRARY_PATH=../vidya/build jolt -M:frq | 16 | ;; LD_LIBRARY_PATH=../vidya/build jolt -M:frq |
| 17 | + ;; libjoltmoq is the AV media plane — MoQ over QUIC, Opus, H.264 and capture — | ||
| 18 | + ;; from gitlab.com/nandithebull/jolt-native. Calls are the one thing this | ||
| 19 | + ;; client cannot do in jolt: signaling is IRC and lives in src/frq/av.jolt, | ||
| 20 | + ;; but the media half is three thousand lines of codec and transport that | ||
| 21 | + ;; would only be written badly a second time here. | ||
| 22 | + ;; cd ../jolt-native && just build | ||
| 17 | :jolt/native [{:name "vidya" | 23 | :jolt/native [{:name "vidya" |
| 18 | :darwin ["libvidya.dylib"] | 24 | :darwin ["libvidya.dylib"] |
| 19 | - :linux ["libvidya.so"]}] | 25 | + :linux ["libvidya.so"]} |
| 26 | + {:name "joltmoq" | ||
| 27 | + :darwin ["libjoltmoq.dylib"] | ||
| 28 | + :linux ["libjoltmoq.so"]}] | ||
| 20 | 29 | ||
| 21 | :aliases {:frq {:main-opts ["-m" "frq.app"]}} | 30 | :aliases {:frq {:main-opts ["-m" "frq.app"]}} |
| 22 | 31 | ||
modified
justfile +7 -1 | @@ -1,6 +1,7 @@ | ||
| 1 | 1 | set shell := ["bash", "-euo", "pipefail", "-c"] |
| 2 | 2 | |
| 3 | 3 | vidya := justfile_directory() + "/../vidya" |
| 4 | +jolt_native := justfile_directory() + "/../jolt-native" | |
| 4 | 5 | |
| 5 | 6 | default: |
| 6 | 7 | @just --list |
| @@ -9,6 +10,11 @@ default: | ||
| 9 | 10 | lib: |
| 10 | 11 | cd {{vidya}} && just ffi |
| 11 | 12 | |
| 13 | +# libjoltmoq — the AV media plane. Only needed for calls; everything else in | |
| 14 | +# the client runs without it. | |
| 15 | +av: | |
| 16 | + cd {{jolt_native}} && just build | |
| 17 | + | |
| 12 | 18 | # The app. Point LD_LIBRARY_PATH at whichever libvidya build you have. |
| 13 | 19 | run *args: |
| 14 | - LD_LIBRARY_PATH="{{vidya}}/build:{{vidya}}/ffi/target/release" jolt -M:frq {{args}} | |
| 20 | + LD_LIBRARY_PATH="{{vidya}}/build:{{vidya}}/ffi/target/release:{{jolt_native}}/target/release" jolt -M:frq {{args}} | |
| @@ -1,6 +1,7 @@ | |||
| 1 | set shell := ["bash", "-euo", "pipefail", "-c"] | 1 | set shell := ["bash", "-euo", "pipefail", "-c"] |
| 2 | 2 | ||
| 3 | vidya := justfile_directory() + "/../vidya" | 3 | vidya := justfile_directory() + "/../vidya" |
| 4 | +jolt_native := justfile_directory() + "/../jolt-native" | ||
| 4 | 5 | ||
| 5 | default: | 6 | default: |
| 6 | @just --list | 7 | @just --list |
| @@ -9,6 +10,11 @@ default: | |||
| 9 | lib: | 10 | lib: |
| 10 | cd {{vidya}} && just ffi | 11 | cd {{vidya}} && just ffi |
| 11 | 12 | ||
| 13 | +# libjoltmoq — the AV media plane. Only needed for calls; everything else in | ||
| 14 | +# the client runs without it. | ||
| 15 | +av: | ||
| 16 | + cd {{jolt_native}} && just build | ||
| 17 | + | ||
| 12 | # The app. Point LD_LIBRARY_PATH at whichever libvidya build you have. | 18 | # The app. Point LD_LIBRARY_PATH at whichever libvidya build you have. |
| 13 | run *args: | 19 | run *args: |
| 14 | - LD_LIBRARY_PATH="{{vidya}}/build:{{vidya}}/ffi/target/release" jolt -M:frq {{args}} | 20 | + LD_LIBRARY_PATH="{{vidya}}/build:{{vidya}}/ffi/target/release:{{jolt_native}}/target/release" jolt -M:frq {{args}} |
modified
src/frq/app.jolt +110 -1 | @@ -8,6 +8,7 @@ | ||
| 8 | 8 | [glimmer.ratom :as r :refer [atom]] |
| 9 | 9 | [glimmer.core :as ui] |
| 10 | 10 | [glimmer-vidya.core :as vidya] |
| 11 | + [frq.av :as av] | |
| 11 | 12 | [frq.avatars :as avatars] |
| 12 | 13 | [frq.clock :as clock] |
| 13 | 14 | [frq.media :as media] |
| @@ -519,6 +520,99 @@ | ||
| 519 | 520 | [:button {:label "← Back" :on-click #(reset! s/lightbox nil)}]] |
| 520 | 521 | [:image {:src path :fit true :on-click #(reset! s/lightbox nil)}]])) |
| 521 | 522 | |
| 523 | +(defn- call-tile | |
| 524 | + "One participant's picture. | |
| 525 | + | |
| 526 | + `:feed` rather than `:src`: these pixels never touch the disk and never | |
| 527 | + become a value here — the media plane hands the decoder's own buffer to | |
| 528 | + Vidya as a pointer, and the tag paints whatever arrived last under that name." | |
| 529 | + [key] | |
| 530 | + (let [mine? (= av/local-feed key)] | |
| 531 | + [:vbox {:key key :spacing 2} | |
| 532 | + [:image {:feed key :max-width 240 :max-height 180}] | |
| 533 | + [:dim-label {:label (if mine? "You" key)}]])) | |
| 534 | + | |
| 535 | +(defn- call-wall | |
| 536 | + "Everyone with a camera on, or nothing at all. | |
| 537 | + | |
| 538 | + A call with no video is the normal case and should look like one — a row of | |
| 539 | + empty frames would suggest something had failed to load." | |
| 540 | + [] | |
| 541 | + (let [tiles (av/tiles)] | |
| 542 | + [:vbox {:key :call-wall :spacing 6} | |
| 543 | + (when (seq tiles) | |
| 544 | + [:hbox {:spacing 8} | |
| 545 | + (mapv call-tile tiles)])])) | |
| 546 | + | |
| 547 | +(defn- call-controls | |
| 548 | + "What the person in a call can do about it. | |
| 549 | + | |
| 550 | + Mute and deafen are separate buttons because they are separate things: a | |
| 551 | + deafened microphone still carries your voice, and one control for both would | |
| 552 | + make the quieter of the two a surprise." | |
| 553 | + [] | |
| 554 | + (let [{:keys [muted? speaker-muted? camera? has-camera? has-mic? media]} @av/local-call] | |
| 555 | + [:vbox {:key :call-controls :spacing 6} | |
| 556 | + [:hbox {:spacing 8} | |
| 557 | + [:label {:label (case media | |
| 558 | + :waiting "Asking to join…" | |
| 559 | + :dialling "Connecting…" | |
| 560 | + :live "In call" | |
| 561 | + :failed "Call failed" | |
| 562 | + "In call")}] | |
| 563 | + ;; A microphone that is not there is worth saying so: the call works, | |
| 564 | + ;; and the person is listening rather than silent by choice. | |
| 565 | + (when (and (= :live media) (not has-mic?)) | |
| 566 | + [:dim-label {:label "· listening only"}])] | |
| 567 | + [:hbox {:spacing 8} | |
| 568 | + [:button {:label (if muted? "Unmute" "Mute") | |
| 569 | + :on-click #(av/set-muted! (not muted?))}] | |
| 570 | + [:button {:label (if speaker-muted? "Undeafen" "Deafen") | |
| 571 | + :on-click #(av/set-speaker-muted! (not speaker-muted?))}] | |
| 572 | + ;; Only offered when there is a camera to turn on. Nothing is more | |
| 573 | + ;; annoying than a control that does nothing and does not say why. | |
| 574 | + (when has-camera? | |
| 575 | + [:button {:label (if camera? "Stop video" "Start video") | |
| 576 | + :on-click #(av/set-camera! (not camera?))}]) | |
| 577 | + [:button {:label "Leave" :on-click #(s/leave-call!)}]] | |
| 578 | + (when-let [e @av/media-error] | |
| 579 | + [:dim-label {:label (str "⚠ " e)}])])) | |
| 580 | + | |
| 581 | +(defn call-bar | |
| 582 | + "The call in this channel, whatever state it is in. Always a node. | |
| 583 | + | |
| 584 | + Three cases, and the empty one matters as much as the others: a channel with | |
| 585 | + no call must render *something* here, because the reconciler matches children | |
| 586 | + by position and a banner that came and went would patch the message list into | |
| 587 | + a button." | |
| 588 | + [channel] | |
| 589 | + [:vbox {:key :call-bar :spacing 6} | |
| 590 | + (cond | |
| 591 | + (av/in-call? channel) | |
| 592 | + [:card {} | |
| 593 | + [:vbox {:spacing 6} | |
| 594 | + [call-controls] | |
| 595 | + [call-wall]]] | |
| 596 | + | |
| 597 | + ;; A call is open in this room and we are not in it. | |
| 598 | + (av/call-in channel) | |
| 599 | + (let [{:keys [session-id participants title]} (av/call-in channel)] | |
| 600 | + [:card {} | |
| 601 | + [:hbox {:spacing 8} | |
| 602 | + [:label {:label (str "📞 " (or title "Call in progress") | |
| 603 | + (if (and participants (pos? participants)) | |
| 604 | + (str " · " participants) | |
| 605 | + ""))}] | |
| 606 | + [:button {:label "Join" | |
| 607 | + :on-click #(s/join-call! channel session-id)}]]]) | |
| 608 | + | |
| 609 | + ;; We are in a call, but in a different room. Say which, since the | |
| 610 | + ;; controls are not on this screen to be found by looking. | |
| 611 | + (av/in-call?) | |
| 612 | + [:dim-label {:label (str "In a call in " (:channel @av/local-call))}] | |
| 613 | + | |
| 614 | + :else nil)]) | |
| 615 | + | |
| 522 | 616 | (defn chat-screen [] |
| 523 | 617 | (let [name @s/current |
| 524 | 618 | buffer (get @s/channels name)] |
| @@ -530,8 +624,17 @@ | ||
| 530 | 624 | [:vbox {:spacing 8 :margin 12 :margin-bottom 4} |
| 531 | 625 | [:hbox {:spacing 8} |
| 532 | 626 | [:button {:label "← Chats" :on-click #(reset! s/screen :chats)}] |
| 533 | - [:title {:label (or name "Chat")}]] | |
| 627 | + [:title {:label (or name "Chat")}] | |
| 628 | + ;; Only in a channel, and only when there is no call to join already — | |
| 629 | + ;; the bar below offers Join in that case, and two ways to end up in the | |
| 630 | + ;; same call is one more than anybody needs. | |
| 631 | + (when (and name | |
| 632 | + (str/starts-with? name "#") | |
| 633 | + (not (av/call-in name)) | |
| 634 | + (not (av/in-call?))) | |
| 635 | + [:button {:label "Call" :on-click #(s/start-call! name)}])] | |
| 534 | 636 | [error-note] |
| 637 | + [call-bar name] | |
| 535 | 638 | ;; :reserve leaves room for everything below: the jump button's row, the |
| 536 | 639 | ;; separator and the compose bar. It does not vary with whether the button |
| 537 | 640 | ;; is showing, and neither does that row — a reserve that changed would |
| @@ -679,4 +782,10 @@ | ||
| 679 | 782 | ;; user should be looking at while this happens, and if it fails, the |
| 680 | 783 | ;; error lands somewhere visible. |
| 681 | 784 | (vidya/after! 150 s/connect!)) |
| 785 | + ;; Calls arrive rather than being asked for, so the media plane is drained | |
| 786 | + ;; every frame whether or not one is up — the drain costs a single integer | |
| 787 | + ;; read when it is not. It has to be a timer: `frame-rgba!` and everything | |
| 788 | + ;; else that touches a node belongs to the loop thread, and this is glimmer's | |
| 789 | + ;; way of getting onto it. | |
| 790 | + (vidya/after! 0 av/install-pump!) | |
| 682 | 791 | (ui/run app :title "frq" :width 520 :height 860)) |
| @@ -8,6 +8,7 @@ | |||
| 8 | [glimmer.ratom :as r :refer [atom]] | 8 | [glimmer.ratom :as r :refer [atom]] |
| 9 | [glimmer.core :as ui] | 9 | [glimmer.core :as ui] |
| 10 | [glimmer-vidya.core :as vidya] | 10 | [glimmer-vidya.core :as vidya] |
| 11 | + [frq.av :as av] | ||
| 11 | [frq.avatars :as avatars] | 12 | [frq.avatars :as avatars] |
| 12 | [frq.clock :as clock] | 13 | [frq.clock :as clock] |
| 13 | [frq.media :as media] | 14 | [frq.media :as media] |
| @@ -519,6 +520,99 @@ | |||
| 519 | [:button {:label "← Back" :on-click #(reset! s/lightbox nil)}]] | 520 | [:button {:label "← Back" :on-click #(reset! s/lightbox nil)}]] |
| 520 | [:image {:src path :fit true :on-click #(reset! s/lightbox nil)}]])) | 521 | [:image {:src path :fit true :on-click #(reset! s/lightbox nil)}]])) |
| 521 | 522 | ||
| 523 | +(defn- call-tile | ||
| 524 | + "One participant's picture. | ||
| 525 | + | ||
| 526 | + `:feed` rather than `:src`: these pixels never touch the disk and never | ||
| 527 | + become a value here — the media plane hands the decoder's own buffer to | ||
| 528 | + Vidya as a pointer, and the tag paints whatever arrived last under that name." | ||
| 529 | + [key] | ||
| 530 | + (let [mine? (= av/local-feed key)] | ||
| 531 | + [:vbox {:key key :spacing 2} | ||
| 532 | + [:image {:feed key :max-width 240 :max-height 180}] | ||
| 533 | + [:dim-label {:label (if mine? "You" key)}]])) | ||
| 534 | + | ||
| 535 | +(defn- call-wall | ||
| 536 | + "Everyone with a camera on, or nothing at all. | ||
| 537 | + | ||
| 538 | + A call with no video is the normal case and should look like one — a row of | ||
| 539 | + empty frames would suggest something had failed to load." | ||
| 540 | + [] | ||
| 541 | + (let [tiles (av/tiles)] | ||
| 542 | + [:vbox {:key :call-wall :spacing 6} | ||
| 543 | + (when (seq tiles) | ||
| 544 | + [:hbox {:spacing 8} | ||
| 545 | + (mapv call-tile tiles)])])) | ||
| 546 | + | ||
| 547 | +(defn- call-controls | ||
| 548 | + "What the person in a call can do about it. | ||
| 549 | + | ||
| 550 | + Mute and deafen are separate buttons because they are separate things: a | ||
| 551 | + deafened microphone still carries your voice, and one control for both would | ||
| 552 | + make the quieter of the two a surprise." | ||
| 553 | + [] | ||
| 554 | + (let [{:keys [muted? speaker-muted? camera? has-camera? has-mic? media]} @av/local-call] | ||
| 555 | + [:vbox {:key :call-controls :spacing 6} | ||
| 556 | + [:hbox {:spacing 8} | ||
| 557 | + [:label {:label (case media | ||
| 558 | + :waiting "Asking to join…" | ||
| 559 | + :dialling "Connecting…" | ||
| 560 | + :live "In call" | ||
| 561 | + :failed "Call failed" | ||
| 562 | + "In call")}] | ||
| 563 | + ;; A microphone that is not there is worth saying so: the call works, | ||
| 564 | + ;; and the person is listening rather than silent by choice. | ||
| 565 | + (when (and (= :live media) (not has-mic?)) | ||
| 566 | + [:dim-label {:label "· listening only"}])] | ||
| 567 | + [:hbox {:spacing 8} | ||
| 568 | + [:button {:label (if muted? "Unmute" "Mute") | ||
| 569 | + :on-click #(av/set-muted! (not muted?))}] | ||
| 570 | + [:button {:label (if speaker-muted? "Undeafen" "Deafen") | ||
| 571 | + :on-click #(av/set-speaker-muted! (not speaker-muted?))}] | ||
| 572 | + ;; Only offered when there is a camera to turn on. Nothing is more | ||
| 573 | + ;; annoying than a control that does nothing and does not say why. | ||
| 574 | + (when has-camera? | ||
| 575 | + [:button {:label (if camera? "Stop video" "Start video") | ||
| 576 | + :on-click #(av/set-camera! (not camera?))}]) | ||
| 577 | + [:button {:label "Leave" :on-click #(s/leave-call!)}]] | ||
| 578 | + (when-let [e @av/media-error] | ||
| 579 | + [:dim-label {:label (str "⚠ " e)}])])) | ||
| 580 | + | ||
| 581 | +(defn call-bar | ||
| 582 | + "The call in this channel, whatever state it is in. Always a node. | ||
| 583 | + | ||
| 584 | + Three cases, and the empty one matters as much as the others: a channel with | ||
| 585 | + no call must render *something* here, because the reconciler matches children | ||
| 586 | + by position and a banner that came and went would patch the message list into | ||
| 587 | + a button." | ||
| 588 | + [channel] | ||
| 589 | + [:vbox {:key :call-bar :spacing 6} | ||
| 590 | + (cond | ||
| 591 | + (av/in-call? channel) | ||
| 592 | + [:card {} | ||
| 593 | + [:vbox {:spacing 6} | ||
| 594 | + [call-controls] | ||
| 595 | + [call-wall]]] | ||
| 596 | + | ||
| 597 | + ;; A call is open in this room and we are not in it. | ||
| 598 | + (av/call-in channel) | ||
| 599 | + (let [{:keys [session-id participants title]} (av/call-in channel)] | ||
| 600 | + [:card {} | ||
| 601 | + [:hbox {:spacing 8} | ||
| 602 | + [:label {:label (str "📞 " (or title "Call in progress") | ||
| 603 | + (if (and participants (pos? participants)) | ||
| 604 | + (str " · " participants) | ||
| 605 | + ""))}] | ||
| 606 | + [:button {:label "Join" | ||
| 607 | + :on-click #(s/join-call! channel session-id)}]]]) | ||
| 608 | + | ||
| 609 | + ;; We are in a call, but in a different room. Say which, since the | ||
| 610 | + ;; controls are not on this screen to be found by looking. | ||
| 611 | + (av/in-call?) | ||
| 612 | + [:dim-label {:label (str "In a call in " (:channel @av/local-call))}] | ||
| 613 | + | ||
| 614 | + :else nil)]) | ||
| 615 | + | ||
| 522 | (defn chat-screen [] | 616 | (defn chat-screen [] |
| 523 | (let [name @s/current | 617 | (let [name @s/current |
| 524 | buffer (get @s/channels name)] | 618 | buffer (get @s/channels name)] |
| @@ -530,8 +624,17 @@ | |||
| 530 | [:vbox {:spacing 8 :margin 12 :margin-bottom 4} | 624 | [:vbox {:spacing 8 :margin 12 :margin-bottom 4} |
| 531 | [:hbox {:spacing 8} | 625 | [:hbox {:spacing 8} |
| 532 | [:button {:label "← Chats" :on-click #(reset! s/screen :chats)}] | 626 | [:button {:label "← Chats" :on-click #(reset! s/screen :chats)}] |
| 533 | - [:title {:label (or name "Chat")}]] | 627 | + [:title {:label (or name "Chat")}] |
| 628 | + ;; Only in a channel, and only when there is no call to join already — | ||
| 629 | + ;; the bar below offers Join in that case, and two ways to end up in the | ||
| 630 | + ;; same call is one more than anybody needs. | ||
| 631 | + (when (and name | ||
| 632 | + (str/starts-with? name "#") | ||
| 633 | + (not (av/call-in name)) | ||
| 634 | + (not (av/in-call?))) | ||
| 635 | + [:button {:label "Call" :on-click #(s/start-call! name)}])] | ||
| 534 | [error-note] | 636 | [error-note] |
| 637 | + [call-bar name] | ||
| 535 | ;; :reserve leaves room for everything below: the jump button's row, the | 638 | ;; :reserve leaves room for everything below: the jump button's row, the |
| 536 | ;; separator and the compose bar. It does not vary with whether the button | 639 | ;; separator and the compose bar. It does not vary with whether the button |
| 537 | ;; is showing, and neither does that row — a reserve that changed would | 640 | ;; is showing, and neither does that row — a reserve that changed would |
| @@ -679,4 +782,10 @@ | |||
| 679 | ;; user should be looking at while this happens, and if it fails, the | 782 | ;; user should be looking at while this happens, and if it fails, the |
| 680 | ;; error lands somewhere visible. | 783 | ;; error lands somewhere visible. |
| 681 | (vidya/after! 150 s/connect!)) | 784 | (vidya/after! 150 s/connect!)) |
| 785 | + ;; Calls arrive rather than being asked for, so the media plane is drained | ||
| 786 | + ;; every frame whether or not one is up — the drain costs a single integer | ||
| 787 | + ;; read when it is not. It has to be a timer: `frame-rgba!` and everything | ||
| 788 | + ;; else that touches a node belongs to the loop thread, and this is glimmer's | ||
| 789 | + ;; way of getting onto it. | ||
| 790 | + (vidya/after! 0 av/install-pump!) | ||
| 682 | (ui/run app :title "frq" :width 520 :height 860)) | 791 | (ui/run app :title "frq" :width 520 :height 860)) |
added
src/frq/av.jolt +423 -0 | new file mode 100644 | ||
| @@ -0,0 +1,423 @@ | ||
| 1 | +(ns frq.av | |
| 2 | + "Calls: the signaling, in jolt, and a handle on the media plane, which is not. | |
| 3 | + | |
| 4 | + A freeq call has two halves. The one that is written here is *signaling* — | |
| 5 | + `+freeq.at/av-start`, `av-join` and `av-leave` go out as TAGMSGs and the | |
| 6 | + server broadcasts `+freeq.at/av-state` back, which is IRC and nothing more, | |
| 7 | + so it is written in the language the rest of the client is written in. | |
| 8 | + | |
| 9 | + The other half is audio and video over MoQ, and it is three thousand lines of | |
| 10 | + Opus, H.264, QUIC and capture that would be written badly a second time in | |
| 11 | + any other language. It lives in `libjoltmoq`, extracted from sleek, and this | |
| 12 | + namespace is the whole of what frq says to it. | |
| 13 | + | |
| 14 | + Two rules come from that side and shape everything here: | |
| 15 | + | |
| 16 | + * **Nothing calls back.** Status and video are *polled* — `pump!` drains both | |
| 17 | + and is called from a timer, which glimmer runs on the loop thread. | |
| 18 | + * **A video frame is borrowed**, valid only until the next poll. `pump!` | |
| 19 | + hands each one to Vidya as it arrives and never keeps one, which is also | |
| 20 | + why a frame never becomes a jolt value: it goes from the decoder's buffer | |
| 21 | + to the texture as a pointer, and is never copied on this side at all." | |
| 22 | + (:require [clojure.string :as str] | |
| 23 | + [glimmer.ratom :as r :refer [atom]] | |
| 24 | + [glimmer-vidya.core :as vidya] | |
| 25 | + [frq.irc :as irc] | |
| 26 | + [jolt.ffi :as ffi])) | |
| 27 | + | |
| 28 | +;; --- the media plane --------------------------------------------------------- | |
| 29 | +;; `libjoltmoq` is declared in deps.edn's `:jolt/native` beside libvidya, so the | |
| 30 | +;; loader has already found it by the time these resolve. | |
| 31 | + | |
| 32 | +(ffi/defcfn raw-init-logging "joltmoq_init_logging" [] :void) | |
| 33 | +(ffi/defcfn raw-start "joltmoq_start" | |
| 34 | + [:string :string :string :string :int :int :int :string :string :string] :int) | |
| 35 | +(ffi/defcfn raw-stop "joltmoq_stop" [] :void) | |
| 36 | +(ffi/defcfn raw-is-live "joltmoq_is_live" [] :int) | |
| 37 | + | |
| 38 | +(ffi/defcfn raw-set-muted "joltmoq_set_muted" [:int] :void) | |
| 39 | +(ffi/defcfn raw-set-speaker-muted "joltmoq_set_speaker_muted" [:int] :void) | |
| 40 | +(ffi/defcfn raw-set-camera "joltmoq_set_camera" [:int] :void) | |
| 41 | +(ffi/defcfn raw-set-camera-device "joltmoq_set_camera_device" [:string] :void) | |
| 42 | +(ffi/defcfn raw-set-mic-device "joltmoq_set_mic_device" [:string] :void) | |
| 43 | +(ffi/defcfn raw-set-speaker-device "joltmoq_set_speaker_device" [:string] :void) | |
| 44 | +(ffi/defcfn mic-level "joltmoq_mic_level" [] :double) | |
| 45 | + | |
| 46 | +(ffi/defcfn raw-poll-status "joltmoq_poll_status" [] :int) | |
| 47 | +(ffi/defcfn status-text "joltmoq_status_text" [] :string) | |
| 48 | +(ffi/defcfn raw-status-has-camera "joltmoq_status_has_camera" [] :int) | |
| 49 | +(ffi/defcfn raw-status-has-mic "joltmoq_status_has_mic" [] :int) | |
| 50 | + | |
| 51 | +(ffi/defcfn raw-frame-poll "joltmoq_frame_poll" [] :int) | |
| 52 | +(ffi/defcfn frame-key "joltmoq_frame_key" [] :string) | |
| 53 | +(ffi/defcfn frame-width "joltmoq_frame_width" [] :int) | |
| 54 | +(ffi/defcfn frame-height "joltmoq_frame_height" [] :int) | |
| 55 | +;; The one pointer that crosses. Read as `:pointer` rather than bytes on | |
| 56 | +;; purpose: it goes straight back out to `vidya/frame-rgba!`, so the pixels | |
| 57 | +;; never become a jolt value and are never copied on this side. | |
| 58 | +(ffi/defcfn frame-rgba "joltmoq_frame_rgba" [] :pointer) | |
| 59 | +(ffi/defcfn video-keys "joltmoq_video_keys" [] :string) | |
| 60 | + | |
| 61 | +(ffi/defcfn raw-cameras "joltmoq_cameras" [] :string) | |
| 62 | +(ffi/defcfn raw-microphones "joltmoq_microphones" [] :string) | |
| 63 | +(ffi/defcfn raw-speakers "joltmoq_speakers" [] :string) | |
| 64 | + | |
| 65 | +(ffi/defcfn raw-sfu-url "joltmoq_sfu_url" [:string :string :string] :string) | |
| 66 | +(ffi/defcfn raw-can-dial "joltmoq_can_dial" [:string :string] :int) | |
| 67 | +(ffi/defcfn new-instance "joltmoq_new_instance" [] :string) | |
| 68 | + | |
| 69 | +;; C has no booleans and no absence; both are converted here rather than at | |
| 70 | +;; every call site, so the rest of this file is written in jolt's own terms. | |
| 71 | +(def ^:private status-none 0) | |
| 72 | +(def ^:private status-live 1) | |
| 73 | +(def ^:private status-ended 2) | |
| 74 | +(def ^:private status-failed 3) | |
| 75 | + | |
| 76 | +(defn- flag [b] (if b 1 0)) | |
| 77 | +(defn- pref [s] (or s "")) | |
| 78 | + | |
| 79 | +(defn live? [] (not (zero? (raw-is-live)))) | |
| 80 | +(defn can-dial? | |
| 81 | + "Whether dialling this server is worth attempting. A remote SFU with no token | |
| 82 | + accepts the connection and closes it, and the MoQ client then retries in a | |
| 83 | + tight loop that looks exactly like a hang." | |
| 84 | + [server jwt] | |
| 85 | + (not (zero? (raw-can-dial (pref server) (pref jwt))))) | |
| 86 | + | |
| 87 | +(defn sfu-url | |
| 88 | + "The SFU to dial for this server, or nil when the server is not one a URL can | |
| 89 | + be made of." | |
| 90 | + [server jwt instance] | |
| 91 | + (let [u (raw-sfu-url (pref server) (pref jwt) (pref instance))] | |
| 92 | + (when (seq u) u))) | |
| 93 | + | |
| 94 | +(defn- parse-devices | |
| 95 | + "`id\\tname\\tdefault` a line into `{:id :name :default?}`. | |
| 96 | + | |
| 97 | + Tab and newline delimit because a device name may hold anything else — a | |
| 98 | + webcam called \"EMEET SmartCam C960, Mono\" has spaces and a comma in it, and | |
| 99 | + splitting on those gives nonsense." | |
| 100 | + [text] | |
| 101 | + (->> (str/split-lines (or text "")) | |
| 102 | + (remove str/blank?) | |
| 103 | + (mapv (fn [line] | |
| 104 | + (let [[id name default] (str/split line #"\t")] | |
| 105 | + {:id id | |
| 106 | + :name (or name id) | |
| 107 | + :default? (= "1" default)}))))) | |
| 108 | + | |
| 109 | +(defn cameras [] (parse-devices (raw-cameras))) | |
| 110 | +(defn microphones [] (parse-devices (raw-microphones))) | |
| 111 | +(defn speakers [] (parse-devices (raw-speakers))) | |
| 112 | + | |
| 113 | +;; --- the signaling tags ------------------------------------------------------ | |
| 114 | +;; Every one of these is a TAGMSG to the channel. The server answers with an | |
| 115 | +;; `+freeq.at/av-state` broadcast, which is what actually moves this client's | |
| 116 | +;; state — nothing below assumes a request succeeded. | |
| 117 | + | |
| 118 | +(defn start-tags | |
| 119 | + "Open a call on this channel." | |
| 120 | + [instance title] | |
| 121 | + (cond-> {"+freeq.at/av-start" "" | |
| 122 | + "+freeq.at/av-instance" instance} | |
| 123 | + (seq title) (assoc "+freeq.at/av-title" title))) | |
| 124 | + | |
| 125 | +(defn join-tags | |
| 126 | + "Join the call already open on this channel." | |
| 127 | + [session-id instance] | |
| 128 | + {"+freeq.at/av-join" "" | |
| 129 | + "+freeq.at/av-id" session-id | |
| 130 | + "+freeq.at/av-instance" instance}) | |
| 131 | + | |
| 132 | +(defn leave-tags | |
| 133 | + [session-id instance] | |
| 134 | + {"+freeq.at/av-leave" "" | |
| 135 | + "+freeq.at/av-id" session-id | |
| 136 | + "+freeq.at/av-instance" instance}) | |
| 137 | + | |
| 138 | +(defn parse-state | |
| 139 | + "An `+freeq.at/av-state` broadcast, or nil for any other TAGMSG. | |
| 140 | + | |
| 141 | + Safe to apply to every TAGMSG that arrives: a reaction answers nil." | |
| 142 | + [tags] | |
| 143 | + (let [action (irc/tag-value tags "+freeq.at/av-state")] | |
| 144 | + (when (contains? #{"started" "joined" "left" "ended"} action) | |
| 145 | + {:action (keyword action) | |
| 146 | + :session-id (or (irc/tag-value tags "+freeq.at/av-id") "") | |
| 147 | + :actor (irc/tag-value tags "+freeq.at/av-actor") | |
| 148 | + :participants (when-let [p (irc/tag-value tags "+freeq.at/av-participants")] | |
| 149 | + (try (Integer/parseInt p) (catch Exception _ nil))) | |
| 150 | + :title (let [t (irc/tag-value tags "+freeq.at/av-title")] | |
| 151 | + (when (seq t) t))}))) | |
| 152 | + | |
| 153 | +(defn state-message | |
| 154 | + "The system line a state change is worth showing as." | |
| 155 | + [{:keys [action actor participants title]}] | |
| 156 | + (let [who (or actor "someone") | |
| 157 | + n (if participants (str " · " participants " in call") "")] | |
| 158 | + (case action | |
| 159 | + :started (str "Call started by " who | |
| 160 | + (if (seq title) (str " “" title "”") "") | |
| 161 | + n) | |
| 162 | + :joined (str who " joined the call" n) | |
| 163 | + :left (str who " left the call" n) | |
| 164 | + :ended (str "Call ended" n) | |
| 165 | + ""))) | |
| 166 | + | |
| 167 | +;; --- what the UI reads ------------------------------------------------------- | |
| 168 | + | |
| 169 | +;; channel -> {:session-id :title :participants :last-actor}. What the server | |
| 170 | +;; says is happening in a room, whether or not we are in it: this is what puts | |
| 171 | +;; a "join the call" banner above a channel nobody here has joined. | |
| 172 | +(defonce channel-calls (atom {})) | |
| 173 | + | |
| 174 | +;; nil, or the call this device is in. `:media` is how far the other half has | |
| 175 | +;; got: :dialling until the plane says otherwise, then :live or :failed. | |
| 176 | +(defonce local-call (atom nil)) | |
| 177 | + | |
| 178 | +;; The last thing the media plane failed with, for the line under the controls. | |
| 179 | +(defonce media-error (atom nil)) | |
| 180 | + | |
| 181 | +;; Feeds pushed to Vidya, so the ones that stop can be dropped again. Without | |
| 182 | +;; this the last frame of someone who left hangs on the wall for the rest of | |
| 183 | +;; the call. | |
| 184 | +(defonce ^:private painted-feeds (atom #{})) | |
| 185 | + | |
| 186 | +;; The self-view is keyed this way by the media plane; the UI wants to know | |
| 187 | +;; which tile is its own, to label it and to mirror nothing else. | |
| 188 | +(def local-feed "__local__") | |
| 189 | + | |
| 190 | +(declare stop-media!) | |
| 191 | + | |
| 192 | +(defn call-in [channel] (get @channel-calls channel)) | |
| 193 | + | |
| 194 | +(defn in-call? | |
| 195 | + ([] (some? @local-call)) | |
| 196 | + ([channel] (= channel (:channel @local-call)))) | |
| 197 | + | |
| 198 | +(defn apply-state! | |
| 199 | + "Fold an `+freeq.at/av-state` broadcast into what we know about `channel`. | |
| 200 | + | |
| 201 | + The server is the authority on who is in a call, so this only writes what it | |
| 202 | + was told. Ending clears the room; anything else updates the tally in place, | |
| 203 | + because a `left` that omits the count should not reset it to nothing." | |
| 204 | + [channel st] | |
| 205 | + (let [{:keys [action session-id actor participants title]} st] | |
| 206 | + (if (= :ended action) | |
| 207 | + (swap! channel-calls dissoc channel) | |
| 208 | + (swap! channel-calls update channel | |
| 209 | + (fn [c] | |
| 210 | + (cond-> (or c {:participants 0}) | |
| 211 | + true (assoc :session-id session-id) | |
| 212 | + participants (assoc :participants participants) | |
| 213 | + ;; A `started` with no count is one person: whoever started it. | |
| 214 | + (and (nil? participants) | |
| 215 | + (= :started action) | |
| 216 | + (zero? (:participants (or c {:participants 0})))) | |
| 217 | + (assoc :participants 1) | |
| 218 | + actor (assoc :last-actor actor) | |
| 219 | + title (assoc :title title))))) | |
| 220 | + ;; A call we are in that has ended is one we are no longer in, whoever | |
| 221 | + ;; ended it. Leaving the local call set would leave the controls up over | |
| 222 | + ;; a session the SFU has already forgotten. | |
| 223 | + (when (and (= :ended action) (in-call? channel)) | |
| 224 | + (stop-media!)))) | |
| 225 | + | |
| 226 | +;; --- the media plane, as this client uses it --------------------------------- | |
| 227 | + | |
| 228 | +(defn- drop-feeds! | |
| 229 | + "Stop painting every feed we have been pushing." | |
| 230 | + [] | |
| 231 | + (doseq [k @painted-feeds] (vidya/frame-drop! k)) | |
| 232 | + (reset! painted-feeds #{})) | |
| 233 | + | |
| 234 | +(defn stop-media! | |
| 235 | + "Leave the media plane and forget the call. Signaling is the caller's to send | |
| 236 | + — `leave!` does both, and this is what is left when the server ended it for | |
| 237 | + us." | |
| 238 | + [] | |
| 239 | + (raw-stop) | |
| 240 | + (drop-feeds!) | |
| 241 | + (reset! local-call nil) | |
| 242 | + (reset! media-error nil)) | |
| 243 | + | |
| 244 | +(defn- start-media! | |
| 245 | + "Dial the SFU for the call we have already joined over IRC. | |
| 246 | + | |
| 247 | + Called once the server has minted a token for us, not when we asked to join: | |
| 248 | + a remote SFU refuses a connection without one and the MoQ client then retries | |
| 249 | + in a loop that looks exactly like a hang." | |
| 250 | + [server] | |
| 251 | + (when-let [{:keys [session-id instance token muted? speaker-muted? camera? | |
| 252 | + camera-id mic-id speaker-id nick]} @local-call] | |
| 253 | + (if-let [url (sfu-url server token instance)] | |
| 254 | + (do | |
| 255 | + (swap! local-call assoc :media :dialling) | |
| 256 | + (reset! media-error nil) | |
| 257 | + (when (zero? (raw-start url session-id (pref nick) instance | |
| 258 | + (flag muted?) (flag speaker-muted?) (flag camera?) | |
| 259 | + (pref camera-id) (pref mic-id) (pref speaker-id))) | |
| 260 | + (swap! local-call assoc :media :failed) | |
| 261 | + (reset! media-error "could not start the media plane"))) | |
| 262 | + (do (swap! local-call assoc :media :failed) | |
| 263 | + (reset! media-error (str "no SFU for " server)))))) | |
| 264 | + | |
| 265 | +(defn apply-token! | |
| 266 | + "The server minted us an SFU token — dial the media plane with it. | |
| 267 | + | |
| 268 | + This arrives as a TAGMSG directed at our own nick rather than at the channel, | |
| 269 | + so the buffer it came in on says nothing about which call it is for; the | |
| 270 | + session id in the tag does. One that names a different session than ours is | |
| 271 | + not ours." | |
| 272 | + [server session-id token] | |
| 273 | + (when-let [lc @local-call] | |
| 274 | + (when (or (str/blank? session-id) | |
| 275 | + (str/blank? (:session-id lc)) | |
| 276 | + (= session-id (:session-id lc))) | |
| 277 | + (swap! local-call | |
| 278 | + #(-> % | |
| 279 | + (assoc :token token :awaiting-start? false) | |
| 280 | + (cond-> (seq session-id) (assoc :session-id session-id)))) | |
| 281 | + (start-media! server)))) | |
| 282 | + | |
| 283 | +(defn begin! | |
| 284 | + "Record that this device is joining `channel`, before the server has agreed. | |
| 285 | + | |
| 286 | + Optimistic on purpose: the controls appear on the press rather than a round | |
| 287 | + trip later. `:awaiting-start?` is what a start that has not been answered yet | |
| 288 | + looks like, and is how a collision is recognised as ours." | |
| 289 | + [{:keys [channel session-id nick muted? speaker-muted? camera? | |
| 290 | + camera-id mic-id speaker-id]}] | |
| 291 | + (let [instance (new-instance)] | |
| 292 | + (reset! media-error nil) | |
| 293 | + (reset! local-call | |
| 294 | + {:channel channel | |
| 295 | + :session-id (or session-id "") | |
| 296 | + :instance instance | |
| 297 | + :nick nick | |
| 298 | + :token nil | |
| 299 | + :awaiting-start? (str/blank? (or session-id "")) | |
| 300 | + :muted? (boolean muted?) | |
| 301 | + :speaker-muted? (boolean speaker-muted?) | |
| 302 | + :camera? (boolean camera?) | |
| 303 | + :camera-id camera-id | |
| 304 | + :mic-id mic-id | |
| 305 | + :speaker-id speaker-id | |
| 306 | + :media :waiting}) | |
| 307 | + instance)) | |
| 308 | + | |
| 309 | +;; --- controls ---------------------------------------------------------------- | |
| 310 | +;; Each writes the cell the UI reads *and* tells the media plane, so a control | |
| 311 | +;; answers on the press rather than a frame later. Muting the microphone and | |
| 312 | +;; muting the speaker are deliberately separate: deafening yourself still lets | |
| 313 | +;; peers hear you. | |
| 314 | + | |
| 315 | +(defn set-muted! [muted?] | |
| 316 | + (swap! local-call assoc :muted? muted?) | |
| 317 | + (raw-set-muted (flag muted?))) | |
| 318 | + | |
| 319 | +(defn set-speaker-muted! [muted?] | |
| 320 | + (swap! local-call assoc :speaker-muted? muted?) | |
| 321 | + (raw-set-speaker-muted (flag muted?))) | |
| 322 | + | |
| 323 | +(defn set-camera! [on?] | |
| 324 | + (swap! local-call assoc :camera? on?) | |
| 325 | + (raw-set-camera (flag on?)) | |
| 326 | + ;; The tile goes when the camera does: the plane stops publishing, so no | |
| 327 | + ;; frame arrives to replace the last one. | |
| 328 | + (when-not on? | |
| 329 | + (vidya/frame-drop! local-feed) | |
| 330 | + (swap! painted-feeds disj local-feed))) | |
| 331 | + | |
| 332 | +(defn set-camera-device! [id] | |
| 333 | + (swap! local-call assoc :camera-id id) | |
| 334 | + (raw-set-camera-device (pref id))) | |
| 335 | + | |
| 336 | +(defn set-mic-device! [id] | |
| 337 | + (swap! local-call assoc :mic-id id) | |
| 338 | + (raw-set-mic-device (pref id))) | |
| 339 | + | |
| 340 | +(defn set-speaker-device! [id] | |
| 341 | + (swap! local-call assoc :speaker-id id) | |
| 342 | + (raw-set-speaker-device (pref id))) | |
| 343 | + | |
| 344 | +;; --- the pump ---------------------------------------------------------------- | |
| 345 | + | |
| 346 | +(defn- pump-status! | |
| 347 | + "Drain what the media plane has learned since the last frame." | |
| 348 | + [] | |
| 349 | + (loop [] | |
| 350 | + (let [code (raw-poll-status)] | |
| 351 | + (when-not (= code status-none) | |
| 352 | + (cond | |
| 353 | + (= code status-live) | |
| 354 | + (swap! local-call #(when % | |
| 355 | + (assoc % :media :live | |
| 356 | + :has-camera? (not (zero? (raw-status-has-camera))) | |
| 357 | + :has-mic? (not (zero? (raw-status-has-mic)))))) | |
| 358 | + | |
| 359 | + (= code status-ended) | |
| 360 | + (stop-media!) | |
| 361 | + | |
| 362 | + (= code status-failed) | |
| 363 | + (do (reset! media-error (status-text)) | |
| 364 | + (swap! local-call #(when % (assoc % :media :failed))))) | |
| 365 | + (recur))))) | |
| 366 | + | |
| 367 | +(defn- pump-frames! | |
| 368 | + "Hand every new frame straight to Vidya. | |
| 369 | + | |
| 370 | + The pointer is borrowed until the next poll, so it is used and dropped inside | |
| 371 | + this loop and never held. Nothing is copied on this side: the pixels go from | |
| 372 | + the decoder's own buffer to a texture without becoming a jolt value at all, | |
| 373 | + which is the only way a call at thirty frames a second is affordable here." | |
| 374 | + [] | |
| 375 | + (loop [] | |
| 376 | + (when-not (zero? (raw-frame-poll)) | |
| 377 | + (let [key (frame-key) | |
| 378 | + w (frame-width) | |
| 379 | + h (frame-height) | |
| 380 | + px (frame-rgba)] | |
| 381 | + (when (and (seq key) (pos? w) (pos? h) (not (ffi/null? px))) | |
| 382 | + (vidya/frame-rgba! key w h px) | |
| 383 | + (swap! painted-feeds conj key))) | |
| 384 | + (recur)))) | |
| 385 | + | |
| 386 | +(defn- forget-gone-feeds! | |
| 387 | + "Drop the tiles of everyone who has stopped publishing." | |
| 388 | + [] | |
| 389 | + (let [live (set (remove str/blank? (str/split-lines (or (video-keys) "")))) | |
| 390 | + gone (remove live @painted-feeds)] | |
| 391 | + (doseq [k gone] | |
| 392 | + (vidya/frame-drop! k) | |
| 393 | + (swap! painted-feeds disj k)))) | |
| 394 | + | |
| 395 | +(defn pump! | |
| 396 | + "One frame's worth of the media plane. Cheap when no call is up. | |
| 397 | + | |
| 398 | + Runs on the loop thread — `vidya/frame-rgba!` may not be called from anywhere | |
| 399 | + else, and neither may anything that touches a node." | |
| 400 | + [] | |
| 401 | + (when (live?) | |
| 402 | + (pump-status!) | |
| 403 | + (pump-frames!) | |
| 404 | + (forget-gone-feeds!))) | |
| 405 | + | |
| 406 | +(defn install-pump! | |
| 407 | + "Start pumping the media plane every frame. Returns a timer id. | |
| 408 | + | |
| 409 | + Sixteen milliseconds rather than a longer gap because this is where video | |
| 410 | + arrives: polling slower than the window paints would show every other frame." | |
| 411 | + [] | |
| 412 | + (vidya/every! 16 pump!)) | |
| 413 | + | |
| 414 | +(defn tiles | |
| 415 | + "Everyone with a picture in the current call, the self-view last. | |
| 416 | + | |
| 417 | + Last because it is the one the person can already see the subject of, so it | |
| 418 | + belongs where it will not push a face they are talking to off the row." | |
| 419 | + [] | |
| 420 | + (let [ks (remove str/blank? (str/split-lines (or (video-keys) ""))) | |
| 421 | + mine (filter #(= local-feed %) ks) | |
| 422 | + others (sort (remove #(= local-feed %) ks))] | |
| 423 | + (vec (concat others mine)))) | |
| new file mode 100644 | |||
| @@ -0,0 +1,423 @@ | |||
| 1 | +(ns frq.av | ||
| 2 | + "Calls: the signaling, in jolt, and a handle on the media plane, which is not. | ||
| 3 | + | ||
| 4 | + A freeq call has two halves. The one that is written here is *signaling* — | ||
| 5 | + `+freeq.at/av-start`, `av-join` and `av-leave` go out as TAGMSGs and the | ||
| 6 | + server broadcasts `+freeq.at/av-state` back, which is IRC and nothing more, | ||
| 7 | + so it is written in the language the rest of the client is written in. | ||
| 8 | + | ||
| 9 | + The other half is audio and video over MoQ, and it is three thousand lines of | ||
| 10 | + Opus, H.264, QUIC and capture that would be written badly a second time in | ||
| 11 | + any other language. It lives in `libjoltmoq`, extracted from sleek, and this | ||
| 12 | + namespace is the whole of what frq says to it. | ||
| 13 | + | ||
| 14 | + Two rules come from that side and shape everything here: | ||
| 15 | + | ||
| 16 | + * **Nothing calls back.** Status and video are *polled* — `pump!` drains both | ||
| 17 | + and is called from a timer, which glimmer runs on the loop thread. | ||
| 18 | + * **A video frame is borrowed**, valid only until the next poll. `pump!` | ||
| 19 | + hands each one to Vidya as it arrives and never keeps one, which is also | ||
| 20 | + why a frame never becomes a jolt value: it goes from the decoder's buffer | ||
| 21 | + to the texture as a pointer, and is never copied on this side at all." | ||
| 22 | + (:require [clojure.string :as str] | ||
| 23 | + [glimmer.ratom :as r :refer [atom]] | ||
| 24 | + [glimmer-vidya.core :as vidya] | ||
| 25 | + [frq.irc :as irc] | ||
| 26 | + [jolt.ffi :as ffi])) | ||
| 27 | + | ||
| 28 | +;; --- the media plane --------------------------------------------------------- | ||
| 29 | +;; `libjoltmoq` is declared in deps.edn's `:jolt/native` beside libvidya, so the | ||
| 30 | +;; loader has already found it by the time these resolve. | ||
| 31 | + | ||
| 32 | +(ffi/defcfn raw-init-logging "joltmoq_init_logging" [] :void) | ||
| 33 | +(ffi/defcfn raw-start "joltmoq_start" | ||
| 34 | + [:string :string :string :string :int :int :int :string :string :string] :int) | ||
| 35 | +(ffi/defcfn raw-stop "joltmoq_stop" [] :void) | ||
| 36 | +(ffi/defcfn raw-is-live "joltmoq_is_live" [] :int) | ||
| 37 | + | ||
| 38 | +(ffi/defcfn raw-set-muted "joltmoq_set_muted" [:int] :void) | ||
| 39 | +(ffi/defcfn raw-set-speaker-muted "joltmoq_set_speaker_muted" [:int] :void) | ||
| 40 | +(ffi/defcfn raw-set-camera "joltmoq_set_camera" [:int] :void) | ||
| 41 | +(ffi/defcfn raw-set-camera-device "joltmoq_set_camera_device" [:string] :void) | ||
| 42 | +(ffi/defcfn raw-set-mic-device "joltmoq_set_mic_device" [:string] :void) | ||
| 43 | +(ffi/defcfn raw-set-speaker-device "joltmoq_set_speaker_device" [:string] :void) | ||
| 44 | +(ffi/defcfn mic-level "joltmoq_mic_level" [] :double) | ||
| 45 | + | ||
| 46 | +(ffi/defcfn raw-poll-status "joltmoq_poll_status" [] :int) | ||
| 47 | +(ffi/defcfn status-text "joltmoq_status_text" [] :string) | ||
| 48 | +(ffi/defcfn raw-status-has-camera "joltmoq_status_has_camera" [] :int) | ||
| 49 | +(ffi/defcfn raw-status-has-mic "joltmoq_status_has_mic" [] :int) | ||
| 50 | + | ||
| 51 | +(ffi/defcfn raw-frame-poll "joltmoq_frame_poll" [] :int) | ||
| 52 | +(ffi/defcfn frame-key "joltmoq_frame_key" [] :string) | ||
| 53 | +(ffi/defcfn frame-width "joltmoq_frame_width" [] :int) | ||
| 54 | +(ffi/defcfn frame-height "joltmoq_frame_height" [] :int) | ||
| 55 | +;; The one pointer that crosses. Read as `:pointer` rather than bytes on | ||
| 56 | +;; purpose: it goes straight back out to `vidya/frame-rgba!`, so the pixels | ||
| 57 | +;; never become a jolt value and are never copied on this side. | ||
| 58 | +(ffi/defcfn frame-rgba "joltmoq_frame_rgba" [] :pointer) | ||
| 59 | +(ffi/defcfn video-keys "joltmoq_video_keys" [] :string) | ||
| 60 | + | ||
| 61 | +(ffi/defcfn raw-cameras "joltmoq_cameras" [] :string) | ||
| 62 | +(ffi/defcfn raw-microphones "joltmoq_microphones" [] :string) | ||
| 63 | +(ffi/defcfn raw-speakers "joltmoq_speakers" [] :string) | ||
| 64 | + | ||
| 65 | +(ffi/defcfn raw-sfu-url "joltmoq_sfu_url" [:string :string :string] :string) | ||
| 66 | +(ffi/defcfn raw-can-dial "joltmoq_can_dial" [:string :string] :int) | ||
| 67 | +(ffi/defcfn new-instance "joltmoq_new_instance" [] :string) | ||
| 68 | + | ||
| 69 | +;; C has no booleans and no absence; both are converted here rather than at | ||
| 70 | +;; every call site, so the rest of this file is written in jolt's own terms. | ||
| 71 | +(def ^:private status-none 0) | ||
| 72 | +(def ^:private status-live 1) | ||
| 73 | +(def ^:private status-ended 2) | ||
| 74 | +(def ^:private status-failed 3) | ||
| 75 | + | ||
| 76 | +(defn- flag [b] (if b 1 0)) | ||
| 77 | +(defn- pref [s] (or s "")) | ||
| 78 | + | ||
| 79 | +(defn live? [] (not (zero? (raw-is-live)))) | ||
| 80 | +(defn can-dial? | ||
| 81 | + "Whether dialling this server is worth attempting. A remote SFU with no token | ||
| 82 | + accepts the connection and closes it, and the MoQ client then retries in a | ||
| 83 | + tight loop that looks exactly like a hang." | ||
| 84 | + [server jwt] | ||
| 85 | + (not (zero? (raw-can-dial (pref server) (pref jwt))))) | ||
| 86 | + | ||
| 87 | +(defn sfu-url | ||
| 88 | + "The SFU to dial for this server, or nil when the server is not one a URL can | ||
| 89 | + be made of." | ||
| 90 | + [server jwt instance] | ||
| 91 | + (let [u (raw-sfu-url (pref server) (pref jwt) (pref instance))] | ||
| 92 | + (when (seq u) u))) | ||
| 93 | + | ||
| 94 | +(defn- parse-devices | ||
| 95 | + "`id\\tname\\tdefault` a line into `{:id :name :default?}`. | ||
| 96 | + | ||
| 97 | + Tab and newline delimit because a device name may hold anything else — a | ||
| 98 | + webcam called \"EMEET SmartCam C960, Mono\" has spaces and a comma in it, and | ||
| 99 | + splitting on those gives nonsense." | ||
| 100 | + [text] | ||
| 101 | + (->> (str/split-lines (or text "")) | ||
| 102 | + (remove str/blank?) | ||
| 103 | + (mapv (fn [line] | ||
| 104 | + (let [[id name default] (str/split line #"\t")] | ||
| 105 | + {:id id | ||
| 106 | + :name (or name id) | ||
| 107 | + :default? (= "1" default)}))))) | ||
| 108 | + | ||
| 109 | +(defn cameras [] (parse-devices (raw-cameras))) | ||
| 110 | +(defn microphones [] (parse-devices (raw-microphones))) | ||
| 111 | +(defn speakers [] (parse-devices (raw-speakers))) | ||
| 112 | + | ||
| 113 | +;; --- the signaling tags ------------------------------------------------------ | ||
| 114 | +;; Every one of these is a TAGMSG to the channel. The server answers with an | ||
| 115 | +;; `+freeq.at/av-state` broadcast, which is what actually moves this client's | ||
| 116 | +;; state — nothing below assumes a request succeeded. | ||
| 117 | + | ||
| 118 | +(defn start-tags | ||
| 119 | + "Open a call on this channel." | ||
| 120 | + [instance title] | ||
| 121 | + (cond-> {"+freeq.at/av-start" "" | ||
| 122 | + "+freeq.at/av-instance" instance} | ||
| 123 | + (seq title) (assoc "+freeq.at/av-title" title))) | ||
| 124 | + | ||
| 125 | +(defn join-tags | ||
| 126 | + "Join the call already open on this channel." | ||
| 127 | + [session-id instance] | ||
| 128 | + {"+freeq.at/av-join" "" | ||
| 129 | + "+freeq.at/av-id" session-id | ||
| 130 | + "+freeq.at/av-instance" instance}) | ||
| 131 | + | ||
| 132 | +(defn leave-tags | ||
| 133 | + [session-id instance] | ||
| 134 | + {"+freeq.at/av-leave" "" | ||
| 135 | + "+freeq.at/av-id" session-id | ||
| 136 | + "+freeq.at/av-instance" instance}) | ||
| 137 | + | ||
| 138 | +(defn parse-state | ||
| 139 | + "An `+freeq.at/av-state` broadcast, or nil for any other TAGMSG. | ||
| 140 | + | ||
| 141 | + Safe to apply to every TAGMSG that arrives: a reaction answers nil." | ||
| 142 | + [tags] | ||
| 143 | + (let [action (irc/tag-value tags "+freeq.at/av-state")] | ||
| 144 | + (when (contains? #{"started" "joined" "left" "ended"} action) | ||
| 145 | + {:action (keyword action) | ||
| 146 | + :session-id (or (irc/tag-value tags "+freeq.at/av-id") "") | ||
| 147 | + :actor (irc/tag-value tags "+freeq.at/av-actor") | ||
| 148 | + :participants (when-let [p (irc/tag-value tags "+freeq.at/av-participants")] | ||
| 149 | + (try (Integer/parseInt p) (catch Exception _ nil))) | ||
| 150 | + :title (let [t (irc/tag-value tags "+freeq.at/av-title")] | ||
| 151 | + (when (seq t) t))}))) | ||
| 152 | + | ||
| 153 | +(defn state-message | ||
| 154 | + "The system line a state change is worth showing as." | ||
| 155 | + [{:keys [action actor participants title]}] | ||
| 156 | + (let [who (or actor "someone") | ||
| 157 | + n (if participants (str " · " participants " in call") "")] | ||
| 158 | + (case action | ||
| 159 | + :started (str "Call started by " who | ||
| 160 | + (if (seq title) (str " “" title "”") "") | ||
| 161 | + n) | ||
| 162 | + :joined (str who " joined the call" n) | ||
| 163 | + :left (str who " left the call" n) | ||
| 164 | + :ended (str "Call ended" n) | ||
| 165 | + ""))) | ||
| 166 | + | ||
| 167 | +;; --- what the UI reads ------------------------------------------------------- | ||
| 168 | + | ||
| 169 | +;; channel -> {:session-id :title :participants :last-actor}. What the server | ||
| 170 | +;; says is happening in a room, whether or not we are in it: this is what puts | ||
| 171 | +;; a "join the call" banner above a channel nobody here has joined. | ||
| 172 | +(defonce channel-calls (atom {})) | ||
| 173 | + | ||
| 174 | +;; nil, or the call this device is in. `:media` is how far the other half has | ||
| 175 | +;; got: :dialling until the plane says otherwise, then :live or :failed. | ||
| 176 | +(defonce local-call (atom nil)) | ||
| 177 | + | ||
| 178 | +;; The last thing the media plane failed with, for the line under the controls. | ||
| 179 | +(defonce media-error (atom nil)) | ||
| 180 | + | ||
| 181 | +;; Feeds pushed to Vidya, so the ones that stop can be dropped again. Without | ||
| 182 | +;; this the last frame of someone who left hangs on the wall for the rest of | ||
| 183 | +;; the call. | ||
| 184 | +(defonce ^:private painted-feeds (atom #{})) | ||
| 185 | + | ||
| 186 | +;; The self-view is keyed this way by the media plane; the UI wants to know | ||
| 187 | +;; which tile is its own, to label it and to mirror nothing else. | ||
| 188 | +(def local-feed "__local__") | ||
| 189 | + | ||
| 190 | +(declare stop-media!) | ||
| 191 | + | ||
| 192 | +(defn call-in [channel] (get @channel-calls channel)) | ||
| 193 | + | ||
| 194 | +(defn in-call? | ||
| 195 | + ([] (some? @local-call)) | ||
| 196 | + ([channel] (= channel (:channel @local-call)))) | ||
| 197 | + | ||
| 198 | +(defn apply-state! | ||
| 199 | + "Fold an `+freeq.at/av-state` broadcast into what we know about `channel`. | ||
| 200 | + | ||
| 201 | + The server is the authority on who is in a call, so this only writes what it | ||
| 202 | + was told. Ending clears the room; anything else updates the tally in place, | ||
| 203 | + because a `left` that omits the count should not reset it to nothing." | ||
| 204 | + [channel st] | ||
| 205 | + (let [{:keys [action session-id actor participants title]} st] | ||
| 206 | + (if (= :ended action) | ||
| 207 | + (swap! channel-calls dissoc channel) | ||
| 208 | + (swap! channel-calls update channel | ||
| 209 | + (fn [c] | ||
| 210 | + (cond-> (or c {:participants 0}) | ||
| 211 | + true (assoc :session-id session-id) | ||
| 212 | + participants (assoc :participants participants) | ||
| 213 | + ;; A `started` with no count is one person: whoever started it. | ||
| 214 | + (and (nil? participants) | ||
| 215 | + (= :started action) | ||
| 216 | + (zero? (:participants (or c {:participants 0})))) | ||
| 217 | + (assoc :participants 1) | ||
| 218 | + actor (assoc :last-actor actor) | ||
| 219 | + title (assoc :title title))))) | ||
| 220 | + ;; A call we are in that has ended is one we are no longer in, whoever | ||
| 221 | + ;; ended it. Leaving the local call set would leave the controls up over | ||
| 222 | + ;; a session the SFU has already forgotten. | ||
| 223 | + (when (and (= :ended action) (in-call? channel)) | ||
| 224 | + (stop-media!)))) | ||
| 225 | + | ||
| 226 | +;; --- the media plane, as this client uses it --------------------------------- | ||
| 227 | + | ||
| 228 | +(defn- drop-feeds! | ||
| 229 | + "Stop painting every feed we have been pushing." | ||
| 230 | + [] | ||
| 231 | + (doseq [k @painted-feeds] (vidya/frame-drop! k)) | ||
| 232 | + (reset! painted-feeds #{})) | ||
| 233 | + | ||
| 234 | +(defn stop-media! | ||
| 235 | + "Leave the media plane and forget the call. Signaling is the caller's to send | ||
| 236 | + — `leave!` does both, and this is what is left when the server ended it for | ||
| 237 | + us." | ||
| 238 | + [] | ||
| 239 | + (raw-stop) | ||
| 240 | + (drop-feeds!) | ||
| 241 | + (reset! local-call nil) | ||
| 242 | + (reset! media-error nil)) | ||
| 243 | + | ||
| 244 | +(defn- start-media! | ||
| 245 | + "Dial the SFU for the call we have already joined over IRC. | ||
| 246 | + | ||
| 247 | + Called once the server has minted a token for us, not when we asked to join: | ||
| 248 | + a remote SFU refuses a connection without one and the MoQ client then retries | ||
| 249 | + in a loop that looks exactly like a hang." | ||
| 250 | + [server] | ||
| 251 | + (when-let [{:keys [session-id instance token muted? speaker-muted? camera? | ||
| 252 | + camera-id mic-id speaker-id nick]} @local-call] | ||
| 253 | + (if-let [url (sfu-url server token instance)] | ||
| 254 | + (do | ||
| 255 | + (swap! local-call assoc :media :dialling) | ||
| 256 | + (reset! media-error nil) | ||
| 257 | + (when (zero? (raw-start url session-id (pref nick) instance | ||
| 258 | + (flag muted?) (flag speaker-muted?) (flag camera?) | ||
| 259 | + (pref camera-id) (pref mic-id) (pref speaker-id))) | ||
| 260 | + (swap! local-call assoc :media :failed) | ||
| 261 | + (reset! media-error "could not start the media plane"))) | ||
| 262 | + (do (swap! local-call assoc :media :failed) | ||
| 263 | + (reset! media-error (str "no SFU for " server)))))) | ||
| 264 | + | ||
| 265 | +(defn apply-token! | ||
| 266 | + "The server minted us an SFU token — dial the media plane with it. | ||
| 267 | + | ||
| 268 | + This arrives as a TAGMSG directed at our own nick rather than at the channel, | ||
| 269 | + so the buffer it came in on says nothing about which call it is for; the | ||
| 270 | + session id in the tag does. One that names a different session than ours is | ||
| 271 | + not ours." | ||
| 272 | + [server session-id token] | ||
| 273 | + (when-let [lc @local-call] | ||
| 274 | + (when (or (str/blank? session-id) | ||
| 275 | + (str/blank? (:session-id lc)) | ||
| 276 | + (= session-id (:session-id lc))) | ||
| 277 | + (swap! local-call | ||
| 278 | + #(-> % | ||
| 279 | + (assoc :token token :awaiting-start? false) | ||
| 280 | + (cond-> (seq session-id) (assoc :session-id session-id)))) | ||
| 281 | + (start-media! server)))) | ||
| 282 | + | ||
| 283 | +(defn begin! | ||
| 284 | + "Record that this device is joining `channel`, before the server has agreed. | ||
| 285 | + | ||
| 286 | + Optimistic on purpose: the controls appear on the press rather than a round | ||
| 287 | + trip later. `:awaiting-start?` is what a start that has not been answered yet | ||
| 288 | + looks like, and is how a collision is recognised as ours." | ||
| 289 | + [{:keys [channel session-id nick muted? speaker-muted? camera? | ||
| 290 | + camera-id mic-id speaker-id]}] | ||
| 291 | + (let [instance (new-instance)] | ||
| 292 | + (reset! media-error nil) | ||
| 293 | + (reset! local-call | ||
| 294 | + {:channel channel | ||
| 295 | + :session-id (or session-id "") | ||
| 296 | + :instance instance | ||
| 297 | + :nick nick | ||
| 298 | + :token nil | ||
| 299 | + :awaiting-start? (str/blank? (or session-id "")) | ||
| 300 | + :muted? (boolean muted?) | ||
| 301 | + :speaker-muted? (boolean speaker-muted?) | ||
| 302 | + :camera? (boolean camera?) | ||
| 303 | + :camera-id camera-id | ||
| 304 | + :mic-id mic-id | ||
| 305 | + :speaker-id speaker-id | ||
| 306 | + :media :waiting}) | ||
| 307 | + instance)) | ||
| 308 | + | ||
| 309 | +;; --- controls ---------------------------------------------------------------- | ||
| 310 | +;; Each writes the cell the UI reads *and* tells the media plane, so a control | ||
| 311 | +;; answers on the press rather than a frame later. Muting the microphone and | ||
| 312 | +;; muting the speaker are deliberately separate: deafening yourself still lets | ||
| 313 | +;; peers hear you. | ||
| 314 | + | ||
| 315 | +(defn set-muted! [muted?] | ||
| 316 | + (swap! local-call assoc :muted? muted?) | ||
| 317 | + (raw-set-muted (flag muted?))) | ||
| 318 | + | ||
| 319 | +(defn set-speaker-muted! [muted?] | ||
| 320 | + (swap! local-call assoc :speaker-muted? muted?) | ||
| 321 | + (raw-set-speaker-muted (flag muted?))) | ||
| 322 | + | ||
| 323 | +(defn set-camera! [on?] | ||
| 324 | + (swap! local-call assoc :camera? on?) | ||
| 325 | + (raw-set-camera (flag on?)) | ||
| 326 | + ;; The tile goes when the camera does: the plane stops publishing, so no | ||
| 327 | + ;; frame arrives to replace the last one. | ||
| 328 | + (when-not on? | ||
| 329 | + (vidya/frame-drop! local-feed) | ||
| 330 | + (swap! painted-feeds disj local-feed))) | ||
| 331 | + | ||
| 332 | +(defn set-camera-device! [id] | ||
| 333 | + (swap! local-call assoc :camera-id id) | ||
| 334 | + (raw-set-camera-device (pref id))) | ||
| 335 | + | ||
| 336 | +(defn set-mic-device! [id] | ||
| 337 | + (swap! local-call assoc :mic-id id) | ||
| 338 | + (raw-set-mic-device (pref id))) | ||
| 339 | + | ||
| 340 | +(defn set-speaker-device! [id] | ||
| 341 | + (swap! local-call assoc :speaker-id id) | ||
| 342 | + (raw-set-speaker-device (pref id))) | ||
| 343 | + | ||
| 344 | +;; --- the pump ---------------------------------------------------------------- | ||
| 345 | + | ||
| 346 | +(defn- pump-status! | ||
| 347 | + "Drain what the media plane has learned since the last frame." | ||
| 348 | + [] | ||
| 349 | + (loop [] | ||
| 350 | + (let [code (raw-poll-status)] | ||
| 351 | + (when-not (= code status-none) | ||
| 352 | + (cond | ||
| 353 | + (= code status-live) | ||
| 354 | + (swap! local-call #(when % | ||
| 355 | + (assoc % :media :live | ||
| 356 | + :has-camera? (not (zero? (raw-status-has-camera))) | ||
| 357 | + :has-mic? (not (zero? (raw-status-has-mic)))))) | ||
| 358 | + | ||
| 359 | + (= code status-ended) | ||
| 360 | + (stop-media!) | ||
| 361 | + | ||
| 362 | + (= code status-failed) | ||
| 363 | + (do (reset! media-error (status-text)) | ||
| 364 | + (swap! local-call #(when % (assoc % :media :failed))))) | ||
| 365 | + (recur))))) | ||
| 366 | + | ||
| 367 | +(defn- pump-frames! | ||
| 368 | + "Hand every new frame straight to Vidya. | ||
| 369 | + | ||
| 370 | + The pointer is borrowed until the next poll, so it is used and dropped inside | ||
| 371 | + this loop and never held. Nothing is copied on this side: the pixels go from | ||
| 372 | + the decoder's own buffer to a texture without becoming a jolt value at all, | ||
| 373 | + which is the only way a call at thirty frames a second is affordable here." | ||
| 374 | + [] | ||
| 375 | + (loop [] | ||
| 376 | + (when-not (zero? (raw-frame-poll)) | ||
| 377 | + (let [key (frame-key) | ||
| 378 | + w (frame-width) | ||
| 379 | + h (frame-height) | ||
| 380 | + px (frame-rgba)] | ||
| 381 | + (when (and (seq key) (pos? w) (pos? h) (not (ffi/null? px))) | ||
| 382 | + (vidya/frame-rgba! key w h px) | ||
| 383 | + (swap! painted-feeds conj key))) | ||
| 384 | + (recur)))) | ||
| 385 | + | ||
| 386 | +(defn- forget-gone-feeds! | ||
| 387 | + "Drop the tiles of everyone who has stopped publishing." | ||
| 388 | + [] | ||
| 389 | + (let [live (set (remove str/blank? (str/split-lines (or (video-keys) "")))) | ||
| 390 | + gone (remove live @painted-feeds)] | ||
| 391 | + (doseq [k gone] | ||
| 392 | + (vidya/frame-drop! k) | ||
| 393 | + (swap! painted-feeds disj k)))) | ||
| 394 | + | ||
| 395 | +(defn pump! | ||
| 396 | + "One frame's worth of the media plane. Cheap when no call is up. | ||
| 397 | + | ||
| 398 | + Runs on the loop thread — `vidya/frame-rgba!` may not be called from anywhere | ||
| 399 | + else, and neither may anything that touches a node." | ||
| 400 | + [] | ||
| 401 | + (when (live?) | ||
| 402 | + (pump-status!) | ||
| 403 | + (pump-frames!) | ||
| 404 | + (forget-gone-feeds!))) | ||
| 405 | + | ||
| 406 | +(defn install-pump! | ||
| 407 | + "Start pumping the media plane every frame. Returns a timer id. | ||
| 408 | + | ||
| 409 | + Sixteen milliseconds rather than a longer gap because this is where video | ||
| 410 | + arrives: polling slower than the window paints would show every other frame." | ||
| 411 | + [] | ||
| 412 | + (vidya/every! 16 pump!)) | ||
| 413 | + | ||
| 414 | +(defn tiles | ||
| 415 | + "Everyone with a picture in the current call, the self-view last. | ||
| 416 | + | ||
| 417 | + Last because it is the one the person can already see the subject of, so it | ||
| 418 | + belongs where it will not push a face they are talking to off the row." | ||
| 419 | + [] | ||
| 420 | + (let [ks (remove str/blank? (str/split-lines (or (video-keys) ""))) | ||
| 421 | + mine (filter #(= local-feed %) ks) | ||
| 422 | + others (sort (remove #(= local-feed %) ks))] | ||
| 423 | + (vec (concat others mine)))) | ||
modified
src/frq/state.jolt +105 -2 | @@ -8,6 +8,7 @@ | ||
| 8 | 8 | [glimmer.ratom :as r :refer [atom]] |
| 9 | 9 | [jolt.host :as host] |
| 10 | 10 | [frq.atproto :as atproto] |
| 11 | + [frq.av :as av] | |
| 11 | 12 | [frq.clock :as clock] |
| 12 | 13 | [frq.emoji :as emoji] |
| 13 | 14 | [frq.irc :as irc] |
| @@ -231,7 +232,53 @@ | ||
| 231 | 232 | msgs)) |
| 232 | 233 | m))))) |
| 233 | 234 | |
| 234 | -(declare join!) | |
| 235 | +(declare join! join-call!) | |
| 236 | + | |
| 237 | +;; --- calls ------------------------------------------------------------------- | |
| 238 | +;; Signaling only. The audio and video themselves are `frq.av`'s, and behind it | |
| 239 | +;; libjoltmoq's; what happens here is that the server's broadcasts become state | |
| 240 | +;; the screens can read, and a press becomes a TAGMSG. | |
| 241 | + | |
| 242 | +(defn apply-call-state! | |
| 243 | + "A `+freeq.at/av-state` broadcast: fold it in, and say so in the buffer. | |
| 244 | + | |
| 245 | + The system line is worth the space — a call is the one thing that happens in | |
| 246 | + a channel while nobody types, and without a line saying so the only trace of | |
| 247 | + someone joining is a number quietly changing in a banner." | |
| 248 | + [channel st] | |
| 249 | + (av/apply-state! channel st) | |
| 250 | + (let [line (av/state-message st)] | |
| 251 | + (when (seq line) | |
| 252 | + (push-message! channel "*" line)))) | |
| 253 | + | |
| 254 | +(defn apply-call-error! | |
| 255 | + "A `+freeq.at/av-error`. Most say the call failed; one says we lost a race. | |
| 256 | + | |
| 257 | + `start-collision` means our `av-start` and someone else's crossed and theirs | |
| 258 | + won. The server names the winning session, so the answer is to join that one | |
| 259 | + rather than to report an error for something the person asked for and can | |
| 260 | + have — they wanted to be in a call in this room, and there is one." | |
| 261 | + [tags code] | |
| 262 | + (let [reason (or (irc/tag-value tags "+freeq.at/av-reason") code) | |
| 263 | + session-id (irc/tag-value tags "+freeq.at/av-id") | |
| 264 | + lc @av/local-call | |
| 265 | + channel (:channel lc)] | |
| 266 | + (if (and (= "start-collision" code) (seq session-id) channel | |
| 267 | + (or (:awaiting-start? lc) (str/blank? (:session-id lc)))) | |
| 268 | + (do | |
| 269 | + (push-message! channel "*" "Call already open — joining it instead") | |
| 270 | + (av/stop-media!) | |
| 271 | + (join-call! channel session-id)) | |
| 272 | + (do | |
| 273 | + (when channel | |
| 274 | + (push-message! channel "*" (str "Call error: " reason))) | |
| 275 | + ;; Only tear down a call the error is actually about. A `join-failed` | |
| 276 | + ;; naming someone else's session is not ours to act on. | |
| 277 | + (when (and lc | |
| 278 | + (or (str/blank? (or session-id "")) | |
| 279 | + (str/blank? (:session-id lc)) | |
| 280 | + (= session-id (:session-id lc)))) | |
| 281 | + (av/stop-media!)))))) | |
| 235 | 282 | |
| 236 | 283 | (defn apply-msg! |
| 237 | 284 | "Fold one parsed IRC message into the state." |
| @@ -280,10 +327,22 @@ | ||
| 280 | 327 | (irc/tag-value tags "+draft/reply")) |
| 281 | 328 | add (or (irc/tag-value tags "+react") |
| 282 | 329 | (irc/tag-value tags "+draft/react")) |
| 283 | - remove-it (irc/tag-value tags "+freeq.at/unreact")] | |
| 330 | + remove-it (irc/tag-value tags "+freeq.at/unreact") | |
| 331 | + call-state (av/parse-state tags) | |
| 332 | + ;; The token is directed at our own nick rather than at | |
| 333 | + ;; the channel, so `buffer` is a DM key here and says | |
| 334 | + ;; nothing about which call it is for. The session id in | |
| 335 | + ;; the tag is what does. | |
| 336 | + token (irc/tag-value tags "+freeq.at/av-token") | |
| 337 | + call-error (irc/tag-value tags "+freeq.at/av-error")] | |
| 284 | 338 | (cond |
| 285 | 339 | add (update-reaction! buffer msgid add from true) |
| 286 | 340 | remove-it (update-reaction! buffer msgid remove-it from false) |
| 341 | + call-state (apply-call-state! buffer call-state) | |
| 342 | + token (av/apply-token! @form-host | |
| 343 | + (irc/tag-value tags "+freeq.at/av-id") | |
| 344 | + token) | |
| 345 | + call-error (apply-call-error! tags call-error) | |
| 287 | 346 | :else nil)) |
| 288 | 347 | "JOIN" (let [ch (first params)] |
| 289 | 348 | (if (= from @form-nick) |
| @@ -697,6 +756,50 @@ | ||
| 697 | 756 | (irc/unreact! c channel msgid emoji))) |
| 698 | 757 | (update-reaction! channel msgid emoji @form-nick on?)))) |
| 699 | 758 | |
| 759 | +(defn start-call! | |
| 760 | + "Open a call on this channel. | |
| 761 | + | |
| 762 | + Optimistic: the controls appear on the press. What comes back settles it — | |
| 763 | + an `av-state` says the room has a call, an `av-token` starts the media, and | |
| 764 | + a `start-collision` means someone beat us to it and we join theirs instead." | |
| 765 | + [channel] | |
| 766 | + (when-let [c @conn] | |
| 767 | + (let [nick (or (:nick @session) @form-nick) | |
| 768 | + instance (av/begin! {:channel channel | |
| 769 | + :nick nick | |
| 770 | + :muted? false | |
| 771 | + :speaker-muted? false | |
| 772 | + ;; Audio first, always. A call that opened with | |
| 773 | + ;; the camera on would be a call that showed | |
| 774 | + ;; someone's room before they had agreed to. | |
| 775 | + :camera? false})] | |
| 776 | + (irc/tagmsg! c channel (av/start-tags instance nil))))) | |
| 777 | + | |
| 778 | +(defn join-call! | |
| 779 | + "Join the call already open on this channel." | |
| 780 | + [channel session-id] | |
| 781 | + (when-let [c @conn] | |
| 782 | + (let [nick (or (:nick @session) @form-nick) | |
| 783 | + instance (av/begin! {:channel channel | |
| 784 | + :session-id session-id | |
| 785 | + :nick nick | |
| 786 | + :muted? false | |
| 787 | + :speaker-muted? false | |
| 788 | + :camera? false})] | |
| 789 | + (irc/tagmsg! c channel (av/join-tags session-id instance))))) | |
| 790 | + | |
| 791 | +(defn leave-call! | |
| 792 | + "Leave the call, telling the room and the SFU both. | |
| 793 | + | |
| 794 | + The media plane goes down first and on its own account: the person pressed | |
| 795 | + leave, so the microphone should be shut whether or not the TAGMSG gets out." | |
| 796 | + [] | |
| 797 | + (when-let [{:keys [channel session-id instance]} @av/local-call] | |
| 798 | + (av/stop-media!) | |
| 799 | + (when-let [c @conn] | |
| 800 | + (when (seq session-id) | |
| 801 | + (irc/tagmsg! c channel (av/leave-tags session-id instance)))))) | |
| 802 | + | |
| 700 | 803 | (defn channel-list |
| 701 | 804 | "Buffers most recently opened first, filtered by the search box. |
| 702 | 805 | |
| @@ -8,6 +8,7 @@ | |||
| 8 | [glimmer.ratom :as r :refer [atom]] | 8 | [glimmer.ratom :as r :refer [atom]] |
| 9 | [jolt.host :as host] | 9 | [jolt.host :as host] |
| 10 | [frq.atproto :as atproto] | 10 | [frq.atproto :as atproto] |
| 11 | + [frq.av :as av] | ||
| 11 | [frq.clock :as clock] | 12 | [frq.clock :as clock] |
| 12 | [frq.emoji :as emoji] | 13 | [frq.emoji :as emoji] |
| 13 | [frq.irc :as irc] | 14 | [frq.irc :as irc] |
| @@ -231,7 +232,53 @@ | |||
| 231 | msgs)) | 232 | msgs)) |
| 232 | m))))) | 233 | m))))) |
| 233 | 234 | ||
| 234 | -(declare join!) | 235 | +(declare join! join-call!) |
| 236 | + | ||
| 237 | +;; --- calls ------------------------------------------------------------------- | ||
| 238 | +;; Signaling only. The audio and video themselves are `frq.av`'s, and behind it | ||
| 239 | +;; libjoltmoq's; what happens here is that the server's broadcasts become state | ||
| 240 | +;; the screens can read, and a press becomes a TAGMSG. | ||
| 241 | + | ||
| 242 | +(defn apply-call-state! | ||
| 243 | + "A `+freeq.at/av-state` broadcast: fold it in, and say so in the buffer. | ||
| 244 | + | ||
| 245 | + The system line is worth the space — a call is the one thing that happens in | ||
| 246 | + a channel while nobody types, and without a line saying so the only trace of | ||
| 247 | + someone joining is a number quietly changing in a banner." | ||
| 248 | + [channel st] | ||
| 249 | + (av/apply-state! channel st) | ||
| 250 | + (let [line (av/state-message st)] | ||
| 251 | + (when (seq line) | ||
| 252 | + (push-message! channel "*" line)))) | ||
| 253 | + | ||
| 254 | +(defn apply-call-error! | ||
| 255 | + "A `+freeq.at/av-error`. Most say the call failed; one says we lost a race. | ||
| 256 | + | ||
| 257 | + `start-collision` means our `av-start` and someone else's crossed and theirs | ||
| 258 | + won. The server names the winning session, so the answer is to join that one | ||
| 259 | + rather than to report an error for something the person asked for and can | ||
| 260 | + have — they wanted to be in a call in this room, and there is one." | ||
| 261 | + [tags code] | ||
| 262 | + (let [reason (or (irc/tag-value tags "+freeq.at/av-reason") code) | ||
| 263 | + session-id (irc/tag-value tags "+freeq.at/av-id") | ||
| 264 | + lc @av/local-call | ||
| 265 | + channel (:channel lc)] | ||
| 266 | + (if (and (= "start-collision" code) (seq session-id) channel | ||
| 267 | + (or (:awaiting-start? lc) (str/blank? (:session-id lc)))) | ||
| 268 | + (do | ||
| 269 | + (push-message! channel "*" "Call already open — joining it instead") | ||
| 270 | + (av/stop-media!) | ||
| 271 | + (join-call! channel session-id)) | ||
| 272 | + (do | ||
| 273 | + (when channel | ||
| 274 | + (push-message! channel "*" (str "Call error: " reason))) | ||
| 275 | + ;; Only tear down a call the error is actually about. A `join-failed` | ||
| 276 | + ;; naming someone else's session is not ours to act on. | ||
| 277 | + (when (and lc | ||
| 278 | + (or (str/blank? (or session-id "")) | ||
| 279 | + (str/blank? (:session-id lc)) | ||
| 280 | + (= session-id (:session-id lc)))) | ||
| 281 | + (av/stop-media!)))))) | ||
| 235 | 282 | ||
| 236 | (defn apply-msg! | 283 | (defn apply-msg! |
| 237 | "Fold one parsed IRC message into the state." | 284 | "Fold one parsed IRC message into the state." |
| @@ -280,10 +327,22 @@ | |||
| 280 | (irc/tag-value tags "+draft/reply")) | 327 | (irc/tag-value tags "+draft/reply")) |
| 281 | add (or (irc/tag-value tags "+react") | 328 | add (or (irc/tag-value tags "+react") |
| 282 | (irc/tag-value tags "+draft/react")) | 329 | (irc/tag-value tags "+draft/react")) |
| 283 | - remove-it (irc/tag-value tags "+freeq.at/unreact")] | 330 | + remove-it (irc/tag-value tags "+freeq.at/unreact") |
| 331 | + call-state (av/parse-state tags) | ||
| 332 | + ;; The token is directed at our own nick rather than at | ||
| 333 | + ;; the channel, so `buffer` is a DM key here and says | ||
| 334 | + ;; nothing about which call it is for. The session id in | ||
| 335 | + ;; the tag is what does. | ||
| 336 | + token (irc/tag-value tags "+freeq.at/av-token") | ||
| 337 | + call-error (irc/tag-value tags "+freeq.at/av-error")] | ||
| 284 | (cond | 338 | (cond |
| 285 | add (update-reaction! buffer msgid add from true) | 339 | add (update-reaction! buffer msgid add from true) |
| 286 | remove-it (update-reaction! buffer msgid remove-it from false) | 340 | remove-it (update-reaction! buffer msgid remove-it from false) |
| 341 | + call-state (apply-call-state! buffer call-state) | ||
| 342 | + token (av/apply-token! @form-host | ||
| 343 | + (irc/tag-value tags "+freeq.at/av-id") | ||
| 344 | + token) | ||
| 345 | + call-error (apply-call-error! tags call-error) | ||
| 287 | :else nil)) | 346 | :else nil)) |
| 288 | "JOIN" (let [ch (first params)] | 347 | "JOIN" (let [ch (first params)] |
| 289 | (if (= from @form-nick) | 348 | (if (= from @form-nick) |
| @@ -697,6 +756,50 @@ | |||
| 697 | (irc/unreact! c channel msgid emoji))) | 756 | (irc/unreact! c channel msgid emoji))) |
| 698 | (update-reaction! channel msgid emoji @form-nick on?)))) | 757 | (update-reaction! channel msgid emoji @form-nick on?)))) |
| 699 | 758 | ||
| 759 | +(defn start-call! | ||
| 760 | + "Open a call on this channel. | ||
| 761 | + | ||
| 762 | + Optimistic: the controls appear on the press. What comes back settles it — | ||
| 763 | + an `av-state` says the room has a call, an `av-token` starts the media, and | ||
| 764 | + a `start-collision` means someone beat us to it and we join theirs instead." | ||
| 765 | + [channel] | ||
| 766 | + (when-let [c @conn] | ||
| 767 | + (let [nick (or (:nick @session) @form-nick) | ||
| 768 | + instance (av/begin! {:channel channel | ||
| 769 | + :nick nick | ||
| 770 | + :muted? false | ||
| 771 | + :speaker-muted? false | ||
| 772 | + ;; Audio first, always. A call that opened with | ||
| 773 | + ;; the camera on would be a call that showed | ||
| 774 | + ;; someone's room before they had agreed to. | ||
| 775 | + :camera? false})] | ||
| 776 | + (irc/tagmsg! c channel (av/start-tags instance nil))))) | ||
| 777 | + | ||
| 778 | +(defn join-call! | ||
| 779 | + "Join the call already open on this channel." | ||
| 780 | + [channel session-id] | ||
| 781 | + (when-let [c @conn] | ||
| 782 | + (let [nick (or (:nick @session) @form-nick) | ||
| 783 | + instance (av/begin! {:channel channel | ||
| 784 | + :session-id session-id | ||
| 785 | + :nick nick | ||
| 786 | + :muted? false | ||
| 787 | + :speaker-muted? false | ||
| 788 | + :camera? false})] | ||
| 789 | + (irc/tagmsg! c channel (av/join-tags session-id instance))))) | ||
| 790 | + | ||
| 791 | +(defn leave-call! | ||
| 792 | + "Leave the call, telling the room and the SFU both. | ||
| 793 | + | ||
| 794 | + The media plane goes down first and on its own account: the person pressed | ||
| 795 | + leave, so the microphone should be shut whether or not the TAGMSG gets out." | ||
| 796 | + [] | ||
| 797 | + (when-let [{:keys [channel session-id instance]} @av/local-call] | ||
| 798 | + (av/stop-media!) | ||
| 799 | + (when-let [c @conn] | ||
| 800 | + (when (seq session-id) | ||
| 801 | + (irc/tagmsg! c channel (av/leave-tags session-id instance)))))) | ||
| 802 | + | ||
| 700 | (defn channel-list | 803 | (defn channel-list |
| 701 | "Buffers most recently opened first, filtered by the search box. | 804 | "Buffers most recently opened first, filtered by the search box. |
| 702 | 805 | ||