rickub
Sign in Get started
nandi/jolt-nativepublic⑂ Fork 0
⑂ c2d912f
Branches
main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/jolt-native.git
git clone ssh://git@rickub.com/nandi/jolt-native.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Draw a picture in a terminal, over the Kitty graphics protocol

An `:image` was a tag with a `src`, which this measured as nothing and
painted as nothing: a picture in a message was the link that fetched it
and no more. A cell grid has no pixels, so it cannot be otherwise —
unless the terminal is asked to draw over the cells, which is what the
Kitty protocol is for and what Ghostty, kitty and WezTerm speak.

So the painter reserves the cells a picture's shape asks for and records
where they are, and the flush turns that into an escape. The file goes
over as it is, with `f=100`: nothing here decodes an image, which is the
bargain the window backend already makes — libvidya decodes PNG and
nothing else, and frq caches nothing else. The size comes out of the
twenty-four byte header, the size of a cell comes from the terminal, and
between them they give the rows and columns with the picture's shape
kept.

A picture crosses the wire once. Afterwards only its placement moves, and
a placement that has not moved is not sent again — the wire is quiet
while a reader reads. One inside a scroll is cropped from the source
rather than dropped, so a backlog scrolls past a picture a row at a time
instead of losing it whole at the edge or painting it over the compose
bar. Closing the session deletes them: a placement is the terminal's, and
outlives the alternate screen it was made on.

Off unless the terminal is one that answers. The protocol's own query
comes back as input, which is too late for the layout that has to decide
how many rows a picture takes — and a terminal without the protocol would
print the escape, which is a screenful of base64 where the conversation
was. So it is `$TERM`, `$TERM_PROGRAM` and kitty's own variable that say,
with `JOLT_TUI_GRAPHICS` to force it either way. Where the answer is no,
the cells carry a dim note that a picture is there and the link above it
is the way to it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nandi committed 2026-09-03T01:29:15-07:00 Browse files
c2d912f parent: 3dd441e
added crates/jolt-tui/src/graphics.rs +354 -0
new file mode 100644
@@ -0,0 +1,354 @@
1+//! Pictures in a terminal, over the Kitty graphics protocol.
2+//!
3+//! A cell grid has no pixels, so an `:image` is not painted like everything
4+//! else here: the painter reserves the cells and records where they are, and
5+//! this turns that into the escape sequences that put a picture there. The
6+//! terminal draws it over the blank cells the grid left behind.
7+//!
8+//! PNG only, and deliberately: the file is handed over as it is, with `f=100`,
9+//! so nothing here decodes an image. That is the same bargain the window
10+//! backend makes — `libvidya` decodes PNG and nothing else, and frq only ever
11+//! caches PNG — and it is what keeps a terminal backend free of an image
12+//! library.
13+//!
14+//! Not every terminal has the protocol, and one that has not would print the
15+//! escape as text: a screenful of base64 where a conversation was. So this is
16+//! off unless the terminal is one that is known to answer — `$TERM`,
17+//! `$TERM_PROGRAM` and kitty's own variable say so — and `JOLT_TUI_GRAPHICS`
18+//! forces it either way for a terminal this does not know about yet.
19+
20+use std::collections::HashMap;
21+use std::io::{self, Write};
22+
23+use crate::screen::Rect;
24+
25+/// Where one picture goes, as the painter left it.
26+#[derive(Clone, Debug, PartialEq, Eq)]
27+pub struct Placement {
28+ /// The node it belongs to, which is what makes it the same picture across
29+ /// frames while it scrolls.
30+ pub node: u32,
31+ /// The file, which is what makes it the same *picture* — two nodes showing
32+ /// one file are transmitted once.
33+ pub path: String,
34+ /// The cells it was given.
35+ pub area: Rect,
36+ /// Rows of the picture cut off the top and the bottom by whatever it is
37+ /// scrolling inside. A partly-visible picture is placed partly, rather
38+ /// than whole and over its neighbours, or not at all and flickering.
39+ pub crop_top: u16,
40+ pub crop_bottom: u16,
41+}
42+
43+/// A picture's size in pixels, read out of the PNG header.
44+///
45+/// The eight-byte signature, then the first chunk, which the format says is
46+/// `IHDR`: length, type, width, height. Sixteen bytes of a file, so this is a
47+/// read of the head rather than a decode of the whole.
48+pub fn png_size(bytes: &[u8]) -> Option<(u32, u32)> {
49+ const SIGNATURE: [u8; 8] = [0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A];
50+ if bytes.len() < 24 || bytes[..8] != SIGNATURE || &bytes[12..16] != b"IHDR" {
51+ return None;
52+ }
53+ let read = |at: usize| {
54+ u32::from_be_bytes([bytes[at], bytes[at + 1], bytes[at + 2], bytes[at + 3]])
55+ };
56+ let (w, h) = (read(16), read(20));
57+ (w > 0 && h > 0).then_some((w, h))
58+}
59+
60+/// Whether this terminal draws pictures.
61+///
62+/// A name rather than a question asked over the wire: the protocol's query is
63+/// an escape whose answer comes back as input, and the answer arrives while
64+/// the first frame is being painted — too late for the layout that has to
65+/// decide how many rows a picture takes. Names are what a terminal is
66+/// identified by everywhere else here, and `JOLT_TUI_GRAPHICS=1` is the way in
67+/// for one this list has not learned yet.
68+pub fn supported() -> bool {
69+ #[cfg(test)]
70+ if let Some(forced) = FORCED.with(|f| f.get()) {
71+ return forced;
72+ }
73+ match std::env::var("JOLT_TUI_GRAPHICS").as_deref() {
74+ Ok("1") | Ok("true") => return true,
75+ Ok("0") | Ok("false") => return false,
76+ _ => {}
77+ }
78+ if std::env::var_os("KITTY_WINDOW_ID").is_some()
79+ || std::env::var_os("GHOSTTY_RESOURCES_DIR").is_some()
80+ {
81+ return true;
82+ }
83+ let known = |name: String| {
84+ let name = name.to_ascii_lowercase();
85+ name.contains("kitty") || name.contains("ghostty") || name.contains("wezterm")
86+ };
87+ std::env::var("TERM").map(known).unwrap_or(false)
88+ || std::env::var("TERM_PROGRAM").map(known).unwrap_or(false)
89+}
90+
91+/// One cell in pixels. The terminal is asked; where it will not say — a
92+/// multiplexer in the way, a terminal that answers zero — the usual size of a
93+/// cell in a terminal font stands in, which is wrong by a little rather than
94+/// by an order of magnitude.
95+pub fn cell_pixels(columns: u16, rows: u16, width: u16, height: u16) -> (u16, u16) {
96+ let w = if columns > 0 && width > 0 {
97+ width / columns
98+ } else {
99+ 0
100+ };
101+ let h = if rows > 0 && height > 0 {
102+ height / rows
103+ } else {
104+ 0
105+ };
106+ (if w > 0 { w } else { 8 }, if h > 0 { h } else { 16 })
107+}
108+
109+thread_local! {
110+ /// An answer for [`supported`] that does not come from the environment —
111+ /// what a test says, so one asserting on a screen with a picture in it
112+ /// does not depend on the terminal it happens to be run from.
113+ #[cfg(test)]
114+ static FORCED: std::cell::Cell<Option<bool>> = const { std::cell::Cell::new(None) };
115+ /// What a cell measures, and the size of every picture on screen follows
116+ /// from it. The terminal is asked once when it opens and again when it is
117+ /// resized; a headless session keeps the default, which is what makes a
118+ /// test of a layout with a picture in it repeatable.
119+ static CELL: std::cell::Cell<(u16, u16)> = const { std::cell::Cell::new((8, 16)) };
120+ /// Sizes read out of PNG headers, by path. A layout asks for every picture
121+ /// on screen every frame, and a file's header does not change under it —
122+ /// frq writes a cached picture once, under a name of its own.
123+ static SIZES: std::cell::RefCell<HashMap<String, Option<(u32, u32)>>> =
124+ std::cell::RefCell::new(HashMap::new());
125+}
126+
127+/// Say yes or no for this thread, whatever the terminal is. `None` gives the
128+/// question back to the environment.
129+#[cfg(test)]
130+pub fn force(answer: Option<bool>) {
131+ FORCED.with(|f| f.set(answer));
132+}
133+
134+pub fn set_cell(cell: (u16, u16)) {
135+ CELL.with(|c| c.set(cell));
136+}
137+
138+pub fn cell() -> (u16, u16) {
139+ CELL.with(|c| c.get())
140+}
141+
142+/// The size of the picture at `path`, remembered.
143+pub fn size_of(path: &str) -> Option<(u32, u32)> {
144+ SIZES.with(|sizes| {
145+ if let Some(known) = sizes.borrow().get(path) {
146+ return *known;
147+ }
148+ let size = std::fs::read(path).ok().and_then(|bytes| png_size(&bytes));
149+ sizes.borrow_mut().insert(path.to_owned(), size);
150+ size
151+ })
152+}
153+
154+/// How many cells a picture takes, given what it may not exceed.
155+///
156+/// The caller's `:max-width` and `:max-height` are the bounds — frq measures
157+/// those against the window, and against a terminal they arrive as cells — and
158+/// the picture is fitted inside them with its shape kept. A picture whose size
159+/// cannot be read at all is given a modest box: something is there, and the
160+/// line above it says what.
161+pub fn cells_for(path: &str, max_cols: u16, max_rows: u16) -> (u16, u16) {
162+ let (max_cols, max_rows) = (max_cols.max(1), max_rows.max(1));
163+ let Some((pw, ph)) = size_of(path) else {
164+ return (max_cols.min(24), max_rows.min(3));
165+ };
166+ let (cw, ch) = cell();
167+ let natural_cols = (pw as f64 / cw as f64).ceil().max(1.0);
168+ let natural_rows = (ph as f64 / ch as f64).ceil().max(1.0);
169+ // Whichever bound it meets first, and never scaled up: a small picture is
170+ // small, the way it is in the window.
171+ let scale = (max_cols as f64 / natural_cols)
172+ .min(max_rows as f64 / natural_rows)
173+ .min(1.0);
174+ (
175+ ((natural_cols * scale).round() as u16).max(1),
176+ ((natural_rows * scale).round() as u16).max(1),
177+ )
178+}
179+
180+/// The pictures on screen, and what the terminal has been told about them.
181+#[derive(Default)]
182+pub struct Graphics {
183+ /// File -> the id it was transmitted under. A picture crosses the wire
184+ /// once, however many frames it is on screen for and wherever it scrolls.
185+ sent: HashMap<String, u32>,
186+ /// Node -> what it was last placed as. A placement that has not moved is
187+ /// left alone; the wire is quiet while a reader reads.
188+ placed: HashMap<u32, Placement>,
189+ next_id: u32,
190+}
191+
192+impl Graphics {
193+ /// Bring the terminal's idea of what is on screen into line with `now`.
194+ ///
195+ /// Deletions first: a picture that has moved is deleted and placed again,
196+ /// and doing it in that order means the cells it used to be over are the
197+ /// terminal's own to redraw rather than a hole under the new placement.
198+ pub fn sync(
199+ &mut self,
200+ out: &mut impl Write,
201+ now: &[Placement],
202+ cell: (u16, u16),
203+ ) -> io::Result<()> {
204+ for (node, was) in std::mem::take(&mut self.placed) {
205+ let still = now.iter().any(|p| p.node == node && *p == was);
206+ if still {
207+ self.placed.insert(node, was);
208+ } else {
209+ write!(out, "\x1b_Ga=d,d=i,i={},p={},q=2\x1b\\", self.id(&was.path), node)?;
210+ }
211+ }
212+ for placement in now {
213+ if self.placed.get(&placement.node) == Some(placement) {
214+ continue;
215+ }
216+ self.place(out, placement, cell)?;
217+ self.placed.insert(placement.node, placement.clone());
218+ }
219+ Ok(())
220+ }
221+
222+ /// Every picture forgotten, and the terminal told to drop them all. What
223+ /// closing a session, or leaving the alternate screen, has to do: a
224+ /// placement outlives the frame it was made in.
225+ pub fn clear(&mut self, out: &mut impl Write) -> io::Result<()> {
226+ self.sent.clear();
227+ self.placed.clear();
228+ write!(out, "\x1b_Ga=d,d=A,q=2\x1b\\")
229+ }
230+
231+ fn id(&self, path: &str) -> u32 {
232+ self.sent.get(path).copied().unwrap_or(0)
233+ }
234+
235+ fn place(
236+ &mut self,
237+ out: &mut impl Write,
238+ placement: &Placement,
239+ cell: (u16, u16),
240+ ) -> io::Result<()> {
241+ let Some(id) = self.transmit(out, &placement.path)? else {
242+ return Ok(());
243+ };
244+ let area = placement.area;
245+ if area.w == 0 || area.h == 0 {
246+ return Ok(());
247+ }
248+ // The cursor is where a placement lands, so it goes there first — and
249+ // `C=1` leaves it there rather than letting the picture move it, which
250+ // would put the next thing this writes somewhere else entirely.
251+ write!(out, "\x1b[{};{}H", area.y + 1, area.x + 1)?;
252+ let mut keys = format!(
253+ "a=p,i={id},p={},c={},r={},C=1,q=2",
254+ placement.node, area.w, area.h
255+ );
256+ if placement.crop_top > 0 || placement.crop_bottom > 0 {
257+ // In pixels of the source, which is what the protocol crops in.
258+ let top = placement.crop_top as u32 * cell.1 as u32;
259+ let rows = area.h as u32 * cell.1 as u32;
260+ keys.push_str(&format!(",y={top},h={rows}"));
261+ }
262+ write!(out, "\x1b_G{keys}\x1b\\")
263+ }
264+
265+ /// Hand the file over, once. Answers the id it went under, or `None` for a
266+ /// file that is not there any more or is not a PNG — a fetch that failed
267+ /// leaves a link in the message, which is the right thing to be left with.
268+ fn transmit(&mut self, out: &mut impl Write, path: &str) -> io::Result<Option<u32>> {
269+ if let Some(id) = self.sent.get(path) {
270+ return Ok(Some(*id));
271+ }
272+ let Ok(bytes) = std::fs::read(path) else {
273+ return Ok(None);
274+ };
275+ if png_size(&bytes).is_none() {
276+ return Ok(None);
277+ }
278+ self.next_id += 1;
279+ let id = self.next_id;
280+ // Base64 in chunks the protocol's own size, each saying whether more
281+ // is coming. The first carries the keys; the rest carry only `m`.
282+ let encoded = base64(&bytes);
283+ let mut chunks = encoded.as_bytes().chunks(4096).peekable();
284+ let mut first = true;
285+ while let Some(chunk) = chunks.next() {
286+ let more = u8::from(chunks.peek().is_some());
287+ if first {
288+ write!(out, "\x1b_Ga=t,i={id},f=100,t=d,m={more},q=2;")?;
289+ first = false;
290+ } else {
291+ write!(out, "\x1b_Gm={more},q=2;")?;
292+ }
293+ out.write_all(chunk)?;
294+ write!(out, "\x1b\\")?;
295+ }
296+ self.sent.insert(path.to_owned(), id);
297+ Ok(Some(id))
298+ }
299+}
300+
301+/// Standard base64, which is what the protocol's payload is written in.
302+fn base64(bytes: &[u8]) -> String {
303+ const ALPHABET: &[u8; 64] =
304+ b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
305+ let mut out = String::with_capacity(bytes.len().div_ceil(3) * 4);
306+ for group in bytes.chunks(3) {
307+ let b = [group[0], *group.get(1).unwrap_or(&0), *group.get(2).unwrap_or(&0)];
308+ let n = u32::from_be_bytes([0, b[0], b[1], b[2]]);
309+ let mut quad = [0u8; 4];
310+ for (i, slot) in quad.iter_mut().enumerate() {
311+ *slot = ALPHABET[(n >> (18 - 6 * i) & 0x3F) as usize];
312+ }
313+ for (i, ch) in quad.iter().enumerate() {
314+ // A group short of three bytes pads: two characters carry one
315+ // byte, three carry two.
316+ out.push(if i > group.len() { '=' } else { *ch as char });
317+ }
318+ }
319+ out
320+}
321+
322+#[cfg(test)]
323+mod tests {
324+ use super::*;
325+
326+ #[test]
327+ fn base64_pads_the_way_the_alphabet_says() {
328+ assert_eq!(base64(b""), "");
329+ assert_eq!(base64(b"f"), "Zg==");
330+ assert_eq!(base64(b"fo"), "Zm8=");
331+ assert_eq!(base64(b"foo"), "Zm9v");
332+ assert_eq!(base64(b"foob"), "Zm9vYg==");
333+ assert_eq!(base64(b"any carnal pleasure."), "YW55IGNhcm5hbCBwbGVhc3VyZS4=");
334+ }
335+
336+ #[test]
337+ fn a_png_header_gives_up_its_size_and_anything_else_gives_up_nothing() {
338+ let mut png = vec![0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A];
339+ png.extend_from_slice(&13u32.to_be_bytes());
340+ png.extend_from_slice(b"IHDR");
341+ png.extend_from_slice(&640u32.to_be_bytes());
342+ png.extend_from_slice(&480u32.to_be_bytes());
343+ assert_eq!(png_size(&png), Some((640, 480)));
344+ assert_eq!(png_size(b"not a picture at all"), None);
345+ assert_eq!(png_size(&png[..20]), None);
346+ }
347+
348+ #[test]
349+ fn a_cell_is_the_screen_divided_by_the_grid_or_a_sensible_guess() {
350+ assert_eq!(cell_pixels(80, 24, 640, 384), (8, 16));
351+ assert_eq!(cell_pixels(80, 24, 0, 0), (8, 16));
352+ assert_eq!(cell_pixels(0, 0, 640, 384), (8, 16));
353+ }
354+}
new file mode 100644
@@ -0,0 +1,354 @@
1+//! Pictures in a terminal, over the Kitty graphics protocol.
2+//!
3+//! A cell grid has no pixels, so an `:image` is not painted like everything
4+//! else here: the painter reserves the cells and records where they are, and
5+//! this turns that into the escape sequences that put a picture there. The
6+//! terminal draws it over the blank cells the grid left behind.
7+//!
8+//! PNG only, and deliberately: the file is handed over as it is, with `f=100`,
9+//! so nothing here decodes an image. That is the same bargain the window
10+//! backend makes — `libvidya` decodes PNG and nothing else, and frq only ever
11+//! caches PNG — and it is what keeps a terminal backend free of an image
12+//! library.
13+//!
14+//! Not every terminal has the protocol, and one that has not would print the
15+//! escape as text: a screenful of base64 where a conversation was. So this is
16+//! off unless the terminal is one that is known to answer — `$TERM`,
17+//! `$TERM_PROGRAM` and kitty's own variable say so — and `JOLT_TUI_GRAPHICS`
18+//! forces it either way for a terminal this does not know about yet.
19+
20+use std::collections::HashMap;
21+use std::io::{self, Write};
22+
23+use crate::screen::Rect;
24+
25+/// Where one picture goes, as the painter left it.
26+#[derive(Clone, Debug, PartialEq, Eq)]
27+pub struct Placement {
28+ /// The node it belongs to, which is what makes it the same picture across
29+ /// frames while it scrolls.
30+ pub node: u32,
31+ /// The file, which is what makes it the same *picture* — two nodes showing
32+ /// one file are transmitted once.
33+ pub path: String,
34+ /// The cells it was given.
35+ pub area: Rect,
36+ /// Rows of the picture cut off the top and the bottom by whatever it is
37+ /// scrolling inside. A partly-visible picture is placed partly, rather
38+ /// than whole and over its neighbours, or not at all and flickering.
39+ pub crop_top: u16,
40+ pub crop_bottom: u16,
41+}
42+
43+/// A picture's size in pixels, read out of the PNG header.
44+///
45+/// The eight-byte signature, then the first chunk, which the format says is
46+/// `IHDR`: length, type, width, height. Sixteen bytes of a file, so this is a
47+/// read of the head rather than a decode of the whole.
48+pub fn png_size(bytes: &[u8]) -> Option<(u32, u32)> {
49+ const SIGNATURE: [u8; 8] = [0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A];
50+ if bytes.len() < 24 || bytes[..8] != SIGNATURE || &bytes[12..16] != b"IHDR" {
51+ return None;
52+ }
53+ let read = |at: usize| {
54+ u32::from_be_bytes([bytes[at], bytes[at + 1], bytes[at + 2], bytes[at + 3]])
55+ };
56+ let (w, h) = (read(16), read(20));
57+ (w > 0 && h > 0).then_some((w, h))
58+}
59+
60+/// Whether this terminal draws pictures.
61+///
62+/// A name rather than a question asked over the wire: the protocol's query is
63+/// an escape whose answer comes back as input, and the answer arrives while
64+/// the first frame is being painted — too late for the layout that has to
65+/// decide how many rows a picture takes. Names are what a terminal is
66+/// identified by everywhere else here, and `JOLT_TUI_GRAPHICS=1` is the way in
67+/// for one this list has not learned yet.
68+pub fn supported() -> bool {
69+ #[cfg(test)]
70+ if let Some(forced) = FORCED.with(|f| f.get()) {
71+ return forced;
72+ }
73+ match std::env::var("JOLT_TUI_GRAPHICS").as_deref() {
74+ Ok("1") | Ok("true") => return true,
75+ Ok("0") | Ok("false") => return false,
76+ _ => {}
77+ }
78+ if std::env::var_os("KITTY_WINDOW_ID").is_some()
79+ || std::env::var_os("GHOSTTY_RESOURCES_DIR").is_some()
80+ {
81+ return true;
82+ }
83+ let known = |name: String| {
84+ let name = name.to_ascii_lowercase();
85+ name.contains("kitty") || name.contains("ghostty") || name.contains("wezterm")
86+ };
87+ std::env::var("TERM").map(known).unwrap_or(false)
88+ || std::env::var("TERM_PROGRAM").map(known).unwrap_or(false)
89+}
90+
91+/// One cell in pixels. The terminal is asked; where it will not say — a
92+/// multiplexer in the way, a terminal that answers zero — the usual size of a
93+/// cell in a terminal font stands in, which is wrong by a little rather than
94+/// by an order of magnitude.
95+pub fn cell_pixels(columns: u16, rows: u16, width: u16, height: u16) -> (u16, u16) {
96+ let w = if columns > 0 && width > 0 {
97+ width / columns
98+ } else {
99+ 0
100+ };
101+ let h = if rows > 0 && height > 0 {
102+ height / rows
103+ } else {
104+ 0
105+ };
106+ (if w > 0 { w } else { 8 }, if h > 0 { h } else { 16 })
107+}
108+
109+thread_local! {
110+ /// An answer for [`supported`] that does not come from the environment —
111+ /// what a test says, so one asserting on a screen with a picture in it
112+ /// does not depend on the terminal it happens to be run from.
113+ #[cfg(test)]
114+ static FORCED: std::cell::Cell<Option<bool>> = const { std::cell::Cell::new(None) };
115+ /// What a cell measures, and the size of every picture on screen follows
116+ /// from it. The terminal is asked once when it opens and again when it is
117+ /// resized; a headless session keeps the default, which is what makes a
118+ /// test of a layout with a picture in it repeatable.
119+ static CELL: std::cell::Cell<(u16, u16)> = const { std::cell::Cell::new((8, 16)) };
120+ /// Sizes read out of PNG headers, by path. A layout asks for every picture
121+ /// on screen every frame, and a file's header does not change under it —
122+ /// frq writes a cached picture once, under a name of its own.
123+ static SIZES: std::cell::RefCell<HashMap<String, Option<(u32, u32)>>> =
124+ std::cell::RefCell::new(HashMap::new());
125+}
126+
127+/// Say yes or no for this thread, whatever the terminal is. `None` gives the
128+/// question back to the environment.
129+#[cfg(test)]
130+pub fn force(answer: Option<bool>) {
131+ FORCED.with(|f| f.set(answer));
132+}
133+
134+pub fn set_cell(cell: (u16, u16)) {
135+ CELL.with(|c| c.set(cell));
136+}
137+
138+pub fn cell() -> (u16, u16) {
139+ CELL.with(|c| c.get())
140+}
141+
142+/// The size of the picture at `path`, remembered.
143+pub fn size_of(path: &str) -> Option<(u32, u32)> {
144+ SIZES.with(|sizes| {
145+ if let Some(known) = sizes.borrow().get(path) {
146+ return *known;
147+ }
148+ let size = std::fs::read(path).ok().and_then(|bytes| png_size(&bytes));
149+ sizes.borrow_mut().insert(path.to_owned(), size);
150+ size
151+ })
152+}
153+
154+/// How many cells a picture takes, given what it may not exceed.
155+///
156+/// The caller's `:max-width` and `:max-height` are the bounds — frq measures
157+/// those against the window, and against a terminal they arrive as cells — and
158+/// the picture is fitted inside them with its shape kept. A picture whose size
159+/// cannot be read at all is given a modest box: something is there, and the
160+/// line above it says what.
161+pub fn cells_for(path: &str, max_cols: u16, max_rows: u16) -> (u16, u16) {
162+ let (max_cols, max_rows) = (max_cols.max(1), max_rows.max(1));
163+ let Some((pw, ph)) = size_of(path) else {
164+ return (max_cols.min(24), max_rows.min(3));
165+ };
166+ let (cw, ch) = cell();
167+ let natural_cols = (pw as f64 / cw as f64).ceil().max(1.0);
168+ let natural_rows = (ph as f64 / ch as f64).ceil().max(1.0);
169+ // Whichever bound it meets first, and never scaled up: a small picture is
170+ // small, the way it is in the window.
171+ let scale = (max_cols as f64 / natural_cols)
172+ .min(max_rows as f64 / natural_rows)
173+ .min(1.0);
174+ (
175+ ((natural_cols * scale).round() as u16).max(1),
176+ ((natural_rows * scale).round() as u16).max(1),
177+ )
178+}
179+
180+/// The pictures on screen, and what the terminal has been told about them.
181+#[derive(Default)]
182+pub struct Graphics {
183+ /// File -> the id it was transmitted under. A picture crosses the wire
184+ /// once, however many frames it is on screen for and wherever it scrolls.
185+ sent: HashMap<String, u32>,
186+ /// Node -> what it was last placed as. A placement that has not moved is
187+ /// left alone; the wire is quiet while a reader reads.
188+ placed: HashMap<u32, Placement>,
189+ next_id: u32,
190+}
191+
192+impl Graphics {
193+ /// Bring the terminal's idea of what is on screen into line with `now`.
194+ ///
195+ /// Deletions first: a picture that has moved is deleted and placed again,
196+ /// and doing it in that order means the cells it used to be over are the
197+ /// terminal's own to redraw rather than a hole under the new placement.
198+ pub fn sync(
199+ &mut self,
200+ out: &mut impl Write,
201+ now: &[Placement],
202+ cell: (u16, u16),
203+ ) -> io::Result<()> {
204+ for (node, was) in std::mem::take(&mut self.placed) {
205+ let still = now.iter().any(|p| p.node == node && *p == was);
206+ if still {
207+ self.placed.insert(node, was);
208+ } else {
209+ write!(out, "\x1b_Ga=d,d=i,i={},p={},q=2\x1b\\", self.id(&was.path), node)?;
210+ }
211+ }
212+ for placement in now {
213+ if self.placed.get(&placement.node) == Some(placement) {
214+ continue;
215+ }
216+ self.place(out, placement, cell)?;
217+ self.placed.insert(placement.node, placement.clone());
218+ }
219+ Ok(())
220+ }
221+
222+ /// Every picture forgotten, and the terminal told to drop them all. What
223+ /// closing a session, or leaving the alternate screen, has to do: a
224+ /// placement outlives the frame it was made in.
225+ pub fn clear(&mut self, out: &mut impl Write) -> io::Result<()> {
226+ self.sent.clear();
227+ self.placed.clear();
228+ write!(out, "\x1b_Ga=d,d=A,q=2\x1b\\")
229+ }
230+
231+ fn id(&self, path: &str) -> u32 {
232+ self.sent.get(path).copied().unwrap_or(0)
233+ }
234+
235+ fn place(
236+ &mut self,
237+ out: &mut impl Write,
238+ placement: &Placement,
239+ cell: (u16, u16),
240+ ) -> io::Result<()> {
241+ let Some(id) = self.transmit(out, &placement.path)? else {
242+ return Ok(());
243+ };
244+ let area = placement.area;
245+ if area.w == 0 || area.h == 0 {
246+ return Ok(());
247+ }
248+ // The cursor is where a placement lands, so it goes there first — and
249+ // `C=1` leaves it there rather than letting the picture move it, which
250+ // would put the next thing this writes somewhere else entirely.
251+ write!(out, "\x1b[{};{}H", area.y + 1, area.x + 1)?;
252+ let mut keys = format!(
253+ "a=p,i={id},p={},c={},r={},C=1,q=2",
254+ placement.node, area.w, area.h
255+ );
256+ if placement.crop_top > 0 || placement.crop_bottom > 0 {
257+ // In pixels of the source, which is what the protocol crops in.
258+ let top = placement.crop_top as u32 * cell.1 as u32;
259+ let rows = area.h as u32 * cell.1 as u32;
260+ keys.push_str(&format!(",y={top},h={rows}"));
261+ }
262+ write!(out, "\x1b_G{keys}\x1b\\")
263+ }
264+
265+ /// Hand the file over, once. Answers the id it went under, or `None` for a
266+ /// file that is not there any more or is not a PNG — a fetch that failed
267+ /// leaves a link in the message, which is the right thing to be left with.
268+ fn transmit(&mut self, out: &mut impl Write, path: &str) -> io::Result<Option<u32>> {
269+ if let Some(id) = self.sent.get(path) {
270+ return Ok(Some(*id));
271+ }
272+ let Ok(bytes) = std::fs::read(path) else {
273+ return Ok(None);
274+ };
275+ if png_size(&bytes).is_none() {
276+ return Ok(None);
277+ }
278+ self.next_id += 1;
279+ let id = self.next_id;
280+ // Base64 in chunks the protocol's own size, each saying whether more
281+ // is coming. The first carries the keys; the rest carry only `m`.
282+ let encoded = base64(&bytes);
283+ let mut chunks = encoded.as_bytes().chunks(4096).peekable();
284+ let mut first = true;
285+ while let Some(chunk) = chunks.next() {
286+ let more = u8::from(chunks.peek().is_some());
287+ if first {
288+ write!(out, "\x1b_Ga=t,i={id},f=100,t=d,m={more},q=2;")?;
289+ first = false;
290+ } else {
291+ write!(out, "\x1b_Gm={more},q=2;")?;
292+ }
293+ out.write_all(chunk)?;
294+ write!(out, "\x1b\\")?;
295+ }
296+ self.sent.insert(path.to_owned(), id);
297+ Ok(Some(id))
298+ }
299+}
300+
301+/// Standard base64, which is what the protocol's payload is written in.
302+fn base64(bytes: &[u8]) -> String {
303+ const ALPHABET: &[u8; 64] =
304+ b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
305+ let mut out = String::with_capacity(bytes.len().div_ceil(3) * 4);
306+ for group in bytes.chunks(3) {
307+ let b = [group[0], *group.get(1).unwrap_or(&0), *group.get(2).unwrap_or(&0)];
308+ let n = u32::from_be_bytes([0, b[0], b[1], b[2]]);
309+ let mut quad = [0u8; 4];
310+ for (i, slot) in quad.iter_mut().enumerate() {
311+ *slot = ALPHABET[(n >> (18 - 6 * i) & 0x3F) as usize];
312+ }
313+ for (i, ch) in quad.iter().enumerate() {
314+ // A group short of three bytes pads: two characters carry one
315+ // byte, three carry two.
316+ out.push(if i > group.len() { '=' } else { *ch as char });
317+ }
318+ }
319+ out
320+}
321+
322+#[cfg(test)]
323+mod tests {
324+ use super::*;
325+
326+ #[test]
327+ fn base64_pads_the_way_the_alphabet_says() {
328+ assert_eq!(base64(b""), "");
329+ assert_eq!(base64(b"f"), "Zg==");
330+ assert_eq!(base64(b"fo"), "Zm8=");
331+ assert_eq!(base64(b"foo"), "Zm9v");
332+ assert_eq!(base64(b"foob"), "Zm9vYg==");
333+ assert_eq!(base64(b"any carnal pleasure."), "YW55IGNhcm5hbCBwbGVhc3VyZS4=");
334+ }
335+
336+ #[test]
337+ fn a_png_header_gives_up_its_size_and_anything_else_gives_up_nothing() {
338+ let mut png = vec![0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A];
339+ png.extend_from_slice(&13u32.to_be_bytes());
340+ png.extend_from_slice(b"IHDR");
341+ png.extend_from_slice(&640u32.to_be_bytes());
342+ png.extend_from_slice(&480u32.to_be_bytes());
343+ assert_eq!(png_size(&png), Some((640, 480)));
344+ assert_eq!(png_size(b"not a picture at all"), None);
345+ assert_eq!(png_size(&png[..20]), None);
346+ }
347+
348+ #[test]
349+ fn a_cell_is_the_screen_divided_by_the_grid_or_a_sensible_guess() {
350+ assert_eq!(cell_pixels(80, 24, 640, 384), (8, 16));
351+ assert_eq!(cell_pixels(80, 24, 0, 0), (8, 16));
352+ assert_eq!(cell_pixels(0, 0, 640, 384), (8, 16));
353+ }
354+}
modified crates/jolt-tui/src/layout.rs +31 -0
@@ -10,6 +10,7 @@
1010 //! Nothing here touches a terminal or the screen grid: sizes are arithmetic on
1111 //! the tree, which is why the layout tests below need no TTY.
1212
13+use crate::graphics;
1314 use crate::screen::{glyph_cols, glyphs, text_cols, Rect};
1415 use crate::tree::{Props, Tag, Tree};
1516
@@ -129,6 +130,34 @@ fn longest_word(text: &str) -> u16 {
129130 text.split([' ', '\n']).map(text_cols).max().unwrap_or(0)
130131 }
131132
133+/// The cells a picture is given: its own shape, inside the caller's bounds and
134+/// inside the room the column has.
135+///
136+/// Without the protocol to draw one there is no picture, only the note that
137+/// says there was — so the box is a line, and the link in the message above it
138+/// is what the reader is left with either way.
139+pub fn image_cells(props: &Props, avail: u16) -> (u16, u16) {
140+ let path = props.str("src");
141+ if path.is_empty() {
142+ return (0, 0);
143+ }
144+ if !graphics::supported() {
145+ return (text_cols(PICTURE), 1);
146+ }
147+ let max_cols = match props.cells("max-width", 0) {
148+ 0 => avail,
149+ want => want.min(avail),
150+ };
151+ let max_rows = match props.cells("max-height", 0) {
152+ 0 => 16,
153+ want => want,
154+ };
155+ graphics::cells_for(path, max_cols, max_rows)
156+}
157+
158+/// What stands in for a picture where one cannot be drawn.
159+pub const PICTURE: &str = "[ picture ]";
160+
132161 /// The text an entry shows: its own, or its placeholder when it has none.
133162 pub fn entry_text(props: &Props) -> String {
134163 let text = props.str("text");
@@ -180,6 +209,7 @@ fn intrinsic_width(tree: &Tree, id: u32, minimum: bool) -> u16 {
180209 Tag::Separator => 1,
181210 Tag::Spacer => props.cells("size", 1),
182211 Tag::Emoji => text_cols(props.str("emoji")),
212+ Tag::Image => image_cells(&props, u16::MAX).0,
183213 Tag::Reaction => text_cols(&pill_text(&props)),
184214 Tag::Progress => {
185215 if minimum {
@@ -269,6 +299,7 @@ pub fn height_for_width(tree: &Tree, id: u32, avail: u16) -> u16 {
269299 | Tag::Spinner
270300 | Tag::Reaction
271301 | Tag::Emoji => 1,
302+ Tag::Image => image_cells(&props, inner).1,
272303 Tag::Entry => props.cells("rows", 1).max(1),
273304 Tag::Spacer => props.cells("size", 1),
274305 Tag::Listbox => tree.child_count(id) as u16,
@@ -10,6 +10,7 @@
10 //! Nothing here touches a terminal or the screen grid: sizes are arithmetic on10 //! Nothing here touches a terminal or the screen grid: sizes are arithmetic on
11 //! the tree, which is why the layout tests below need no TTY.11 //! the tree, which is why the layout tests below need no TTY.
12 12
13+use crate::graphics;
13 use crate::screen::{glyph_cols, glyphs, text_cols, Rect};14 use crate::screen::{glyph_cols, glyphs, text_cols, Rect};
14 use crate::tree::{Props, Tag, Tree};15 use crate::tree::{Props, Tag, Tree};
15 16
@@ -129,6 +130,34 @@ fn longest_word(text: &str) -> u16 {
129 text.split([' ', '\n']).map(text_cols).max().unwrap_or(0)130 text.split([' ', '\n']).map(text_cols).max().unwrap_or(0)
130 }131 }
131 132
133+/// The cells a picture is given: its own shape, inside the caller's bounds and
134+/// inside the room the column has.
135+///
136+/// Without the protocol to draw one there is no picture, only the note that
137+/// says there was — so the box is a line, and the link in the message above it
138+/// is what the reader is left with either way.
139+pub fn image_cells(props: &Props, avail: u16) -> (u16, u16) {
140+ let path = props.str("src");
141+ if path.is_empty() {
142+ return (0, 0);
143+ }
144+ if !graphics::supported() {
145+ return (text_cols(PICTURE), 1);
146+ }
147+ let max_cols = match props.cells("max-width", 0) {
148+ 0 => avail,
149+ want => want.min(avail),
150+ };
151+ let max_rows = match props.cells("max-height", 0) {
152+ 0 => 16,
153+ want => want,
154+ };
155+ graphics::cells_for(path, max_cols, max_rows)
156+}
157+
158+/// What stands in for a picture where one cannot be drawn.
159+pub const PICTURE: &str = "[ picture ]";
160+
132 /// The text an entry shows: its own, or its placeholder when it has none.161 /// The text an entry shows: its own, or its placeholder when it has none.
133 pub fn entry_text(props: &Props) -> String {162 pub fn entry_text(props: &Props) -> String {
134 let text = props.str("text");163 let text = props.str("text");
@@ -180,6 +209,7 @@ fn intrinsic_width(tree: &Tree, id: u32, minimum: bool) -> u16 {
180 Tag::Separator => 1,209 Tag::Separator => 1,
181 Tag::Spacer => props.cells("size", 1),210 Tag::Spacer => props.cells("size", 1),
182 Tag::Emoji => text_cols(props.str("emoji")),211 Tag::Emoji => text_cols(props.str("emoji")),
212+ Tag::Image => image_cells(&props, u16::MAX).0,
183 Tag::Reaction => text_cols(&pill_text(&props)),213 Tag::Reaction => text_cols(&pill_text(&props)),
184 Tag::Progress => {214 Tag::Progress => {
185 if minimum {215 if minimum {
@@ -269,6 +299,7 @@ pub fn height_for_width(tree: &Tree, id: u32, avail: u16) -> u16 {
269 | Tag::Spinner299 | Tag::Spinner
270 | Tag::Reaction300 | Tag::Reaction
271 | Tag::Emoji => 1,301 | Tag::Emoji => 1,
302+ Tag::Image => image_cells(&props, inner).1,
272 Tag::Entry => props.cells("rows", 1).max(1),303 Tag::Entry => props.cells("rows", 1).max(1),
273 Tag::Spacer => props.cells("size", 1),304 Tag::Spacer => props.cells("size", 1),
274 Tag::Listbox => tree.child_count(id) as u16,305 Tag::Listbox => tree.child_count(id) as u16,
modified crates/jolt-tui/src/lib.rs +3 -1
@@ -31,6 +31,7 @@
3131 // vocabulary, not dead code, so a headless build does not warn about them.
3232 #![cfg_attr(not(feature = "terminal"), allow(dead_code))]
3333
34+mod graphics;
3435 mod keys;
3536 mod layout;
3637 mod paint;
@@ -225,7 +226,8 @@ pub extern "C" fn tui_frame() {
225226 #[cfg(feature = "terminal")]
226227 if let Some(term) = session.term.as_mut() {
227228 let cursor = session.ui.cursor();
228- if let Err(e) = term.flush(&session.ui.screen, cursor) {
229+ let images = session.ui.images().to_vec();
230+ if let Err(e) = term.flush(&session.ui.screen, cursor, &images) {
229231 log::error!("jolt-tui: could not write a frame: {e}");
230232 }
231233 }
@@ -31,6 +31,7 @@
31 // vocabulary, not dead code, so a headless build does not warn about them.31 // vocabulary, not dead code, so a headless build does not warn about them.
32 #![cfg_attr(not(feature = "terminal"), allow(dead_code))]32 #![cfg_attr(not(feature = "terminal"), allow(dead_code))]
33 33
34+mod graphics;
34 mod keys;35 mod keys;
35 mod layout;36 mod layout;
36 mod paint;37 mod paint;
@@ -225,7 +226,8 @@ pub extern "C" fn tui_frame() {
225 #[cfg(feature = "terminal")]226 #[cfg(feature = "terminal")]
226 if let Some(term) = session.term.as_mut() {227 if let Some(term) = session.term.as_mut() {
227 let cursor = session.ui.cursor();228 let cursor = session.ui.cursor();
228- if let Err(e) = term.flush(&session.ui.screen, cursor) {229+ let images = session.ui.images().to_vec();
230+ if let Err(e) = term.flush(&session.ui.screen, cursor, &images) {
229 log::error!("jolt-tui: could not write a frame: {e}");231 log::error!("jolt-tui: could not write a frame: {e}");
230 }232 }
231 }233 }
modified crates/jolt-tui/src/paint.rs +62 -0
@@ -10,6 +10,7 @@
1010 //! asked for, in the middle.
1111
1212 use crate::layout::{self, wrap, Align};
13+use crate::graphics;
1314 use crate::screen::{self, attr, Color, Rect, Screen, Style};
1415 use crate::tree::{Props, Tag, Tree};
1516
@@ -32,6 +33,10 @@ pub struct Painted {
3233 pub scrolled: Vec<(u32, u16, u16, Rect)>,
3334 /// Where the cursor should sit — the focused entry's caret, if any.
3435 pub cursor: Option<(u16, u16)>,
36+ /// The pictures this frame wants on screen, in the cells they were given.
37+ /// Nothing was painted for them: the grid has no pixels, and the terminal
38+ /// is what draws one — see [`crate::graphics`].
39+ pub images: Vec<graphics::Placement>,
3540 }
3641
3742 struct Painter<'a> {
@@ -155,6 +160,7 @@ impl Painter<'_> {
155160 let ch = SPINNER[(self.tick as usize) % SPINNER.len()];
156161 self.screen.set(inner.x, inner.y, ch, style);
157162 }
163+ Tag::Image => self.image(id, inner, &props, style),
158164 Tag::Reaction => self.reaction(id, inner, &props, style),
159165 // The same glyph with nothing around it: a character in a line,
160166 // and the line is what says anything about it.
@@ -219,6 +225,39 @@ impl Painter<'_> {
219225 self.screen.text(area.x, area.y, area.w, &label, style);
220226 }
221227
228+ /// A picture: the cells it was given, and a note of where they are.
229+ ///
230+ /// Nothing goes in them. The terminal draws the picture over the blank
231+ /// cells when the frame is flushed, which is the only way pixels reach a
232+ /// grid; where there is no protocol for that, the cells carry the note
233+ /// that says a picture is here, and the link above it is the way to it.
234+ fn image(&mut self, id: u32, area: Rect, props: &Props, style: Style) {
235+ let path = props.str("src");
236+ if path.is_empty() || area.is_empty() {
237+ return;
238+ }
239+ if !graphics::supported() {
240+ self.screen
241+ .text(area.x, area.y, area.w, layout::PICTURE, style.with(attr::DIM));
242+ return;
243+ }
244+ // The column hands a child its whole width; a picture takes only what
245+ // its shape asks for out of that, so the placement is the picture and
246+ // not the room around it.
247+ let (cols, rows) = layout::image_cells(props, area.w);
248+ let area = Rect::new(area.x, area.y, cols.min(area.w), rows.min(area.h));
249+ if area.is_empty() {
250+ return;
251+ }
252+ self.out.images.push(graphics::Placement {
253+ node: id,
254+ path: path.to_owned(),
255+ area,
256+ crop_top: 0,
257+ crop_bottom: 0,
258+ });
259+ }
260+
222261 /// A reaction pill: the glyph, the tally where there is one, and whether
223262 /// you are on it.
224263 ///
@@ -425,6 +464,29 @@ impl Painter<'_> {
425464 // and drop the ones the viewport is not showing. A wheel over a nested
426465 // list has to land on the list under the pointer, and a rect left in
427466 // the wrong space is a wheel aimed at whatever happens to be there.
467+ // A picture inside a scroll moves with it, and is cut off by the
468+ // viewport rather than painted over what is above or below: the
469+ // protocol crops from the source, so a backlog scrolls past a picture
470+ // a row at a time instead of losing it whole at the edge.
471+ let bottom = offset.saturating_add(area.h);
472+ for mut placement in learned.images {
473+ let top = placement.area.y;
474+ let foot = top.saturating_add(placement.area.h);
475+ let seen_top = top.max(offset);
476+ let seen_foot = foot.min(bottom);
477+ if seen_foot <= seen_top {
478+ continue;
479+ }
480+ placement.crop_top += seen_top - top;
481+ placement.crop_bottom += foot - seen_foot;
482+ placement.area = Rect::new(
483+ area.x + placement.area.x,
484+ area.y + seen_top - offset,
485+ placement.area.w,
486+ seen_foot - seen_top,
487+ );
488+ self.out.images.push(placement);
489+ }
428490 for (node, inner_offset, max, rect) in learned.scrolled {
429491 if rect.y >= offset && rect.y < offset.saturating_add(area.h) {
430492 self.out.scrolled.push((
@@ -10,6 +10,7 @@
10 //! asked for, in the middle.10 //! asked for, in the middle.
11 11
12 use crate::layout::{self, wrap, Align};12 use crate::layout::{self, wrap, Align};
13+use crate::graphics;
13 use crate::screen::{self, attr, Color, Rect, Screen, Style};14 use crate::screen::{self, attr, Color, Rect, Screen, Style};
14 use crate::tree::{Props, Tag, Tree};15 use crate::tree::{Props, Tag, Tree};
15 16
@@ -32,6 +33,10 @@ pub struct Painted {
32 pub scrolled: Vec<(u32, u16, u16, Rect)>,33 pub scrolled: Vec<(u32, u16, u16, Rect)>,
33 /// Where the cursor should sit — the focused entry's caret, if any.34 /// Where the cursor should sit — the focused entry's caret, if any.
34 pub cursor: Option<(u16, u16)>,35 pub cursor: Option<(u16, u16)>,
36+ /// The pictures this frame wants on screen, in the cells they were given.
37+ /// Nothing was painted for them: the grid has no pixels, and the terminal
38+ /// is what draws one — see [`crate::graphics`].
39+ pub images: Vec<graphics::Placement>,
35 }40 }
36 41
37 struct Painter<'a> {42 struct Painter<'a> {
@@ -155,6 +160,7 @@ impl Painter<'_> {
155 let ch = SPINNER[(self.tick as usize) % SPINNER.len()];160 let ch = SPINNER[(self.tick as usize) % SPINNER.len()];
156 self.screen.set(inner.x, inner.y, ch, style);161 self.screen.set(inner.x, inner.y, ch, style);
157 }162 }
163+ Tag::Image => self.image(id, inner, &props, style),
158 Tag::Reaction => self.reaction(id, inner, &props, style),164 Tag::Reaction => self.reaction(id, inner, &props, style),
159 // The same glyph with nothing around it: a character in a line,165 // The same glyph with nothing around it: a character in a line,
160 // and the line is what says anything about it.166 // and the line is what says anything about it.
@@ -219,6 +225,39 @@ impl Painter<'_> {
219 self.screen.text(area.x, area.y, area.w, &label, style);225 self.screen.text(area.x, area.y, area.w, &label, style);
220 }226 }
221 227
228+ /// A picture: the cells it was given, and a note of where they are.
229+ ///
230+ /// Nothing goes in them. The terminal draws the picture over the blank
231+ /// cells when the frame is flushed, which is the only way pixels reach a
232+ /// grid; where there is no protocol for that, the cells carry the note
233+ /// that says a picture is here, and the link above it is the way to it.
234+ fn image(&mut self, id: u32, area: Rect, props: &Props, style: Style) {
235+ let path = props.str("src");
236+ if path.is_empty() || area.is_empty() {
237+ return;
238+ }
239+ if !graphics::supported() {
240+ self.screen
241+ .text(area.x, area.y, area.w, layout::PICTURE, style.with(attr::DIM));
242+ return;
243+ }
244+ // The column hands a child its whole width; a picture takes only what
245+ // its shape asks for out of that, so the placement is the picture and
246+ // not the room around it.
247+ let (cols, rows) = layout::image_cells(props, area.w);
248+ let area = Rect::new(area.x, area.y, cols.min(area.w), rows.min(area.h));
249+ if area.is_empty() {
250+ return;
251+ }
252+ self.out.images.push(graphics::Placement {
253+ node: id,
254+ path: path.to_owned(),
255+ area,
256+ crop_top: 0,
257+ crop_bottom: 0,
258+ });
259+ }
260+
222 /// A reaction pill: the glyph, the tally where there is one, and whether261 /// A reaction pill: the glyph, the tally where there is one, and whether
223 /// you are on it.262 /// you are on it.
224 ///263 ///
@@ -425,6 +464,29 @@ impl Painter<'_> {
425 // and drop the ones the viewport is not showing. A wheel over a nested464 // and drop the ones the viewport is not showing. A wheel over a nested
426 // list has to land on the list under the pointer, and a rect left in465 // list has to land on the list under the pointer, and a rect left in
427 // the wrong space is a wheel aimed at whatever happens to be there.466 // the wrong space is a wheel aimed at whatever happens to be there.
467+ // A picture inside a scroll moves with it, and is cut off by the
468+ // viewport rather than painted over what is above or below: the
469+ // protocol crops from the source, so a backlog scrolls past a picture
470+ // a row at a time instead of losing it whole at the edge.
471+ let bottom = offset.saturating_add(area.h);
472+ for mut placement in learned.images {
473+ let top = placement.area.y;
474+ let foot = top.saturating_add(placement.area.h);
475+ let seen_top = top.max(offset);
476+ let seen_foot = foot.min(bottom);
477+ if seen_foot <= seen_top {
478+ continue;
479+ }
480+ placement.crop_top += seen_top - top;
481+ placement.crop_bottom += foot - seen_foot;
482+ placement.area = Rect::new(
483+ area.x + placement.area.x,
484+ area.y + seen_top - offset,
485+ placement.area.w,
486+ seen_foot - seen_top,
487+ );
488+ self.out.images.push(placement);
489+ }
428 for (node, inner_offset, max, rect) in learned.scrolled {490 for (node, inner_offset, max, rect) in learned.scrolled {
429 if rect.y >= offset && rect.y < offset.saturating_add(area.h) {491 if rect.y >= offset && rect.y < offset.saturating_add(area.h) {
430 self.out.scrolled.push((492 self.out.scrolled.push((
modified crates/jolt-tui/src/term.rs +32 -1
@@ -17,6 +17,7 @@ use crossterm::terminal::{
1717 use crossterm::{cursor, execute, queue, style};
1818
1919 use crate::keys;
20+use crate::graphics::{self, Graphics, Placement};
2021 use crate::screen::{self, attr, Color, Screen, Style};
2122
2223 /// How far one notch of the wheel moves a list, in rows.
@@ -36,6 +37,8 @@ pub struct Term {
3637 /// What is on the screen now, so a frame only sends what changed.
3738 last: Screen,
3839 mouse: bool,
40+ /// The pictures the terminal has been given, and where they are.
41+ graphics: Graphics,
3942 }
4043
4144 impl Term {
@@ -49,10 +52,12 @@ impl Term {
4952 if mouse {
5053 execute!(out, EnableMouseCapture)?;
5154 }
55+ graphics::set_cell(measure_cell());
5256 Ok(Self {
5357 out,
5458 last: Screen::new(w, h),
5559 mouse,
60+ graphics: Graphics::default(),
5661 })
5762 }
5863
@@ -64,6 +69,10 @@ impl Term {
6469 /// hook — leaving a shell in raw mode with no cursor is the one failure a
6570 /// TUI must not have.
6671 pub fn close(&mut self) {
72+ // The pictures first: a placement is the terminal's, not the screen
73+ // buffer's, and one left behind outlives the alternate screen it was
74+ // made on.
75+ let _ = self.graphics.clear(&mut self.out);
6776 if self.mouse {
6877 let _ = execute!(self.out, DisableMouseCapture);
6978 }
@@ -122,9 +131,18 @@ impl Term {
122131 }
123132
124133 /// Send whatever differs between `screen` and what is already up there.
125- pub fn flush(&mut self, screen: &Screen, cursor: Option<(u16, u16)>) -> io::Result<()> {
134+ pub fn flush(
135+ &mut self,
136+ screen: &Screen,
137+ cursor: Option<(u16, u16)>,
138+ images: &[Placement],
139+ ) -> io::Result<()> {
126140 if (self.last.width(), self.last.height()) != (screen.width(), screen.height()) {
127141 self.last = Screen::new(screen.width(), screen.height());
142+ // A resize is a new cell size as often as not — a font change, a
143+ // window dragged to another screen — and every picture's size is
144+ // measured in cells.
145+ graphics::set_cell(measure_cell());
128146 queue!(
129147 self.out,
130148 crossterm::terminal::Clear(crossterm::terminal::ClearType::All)
@@ -167,6 +185,11 @@ impl Term {
167185 at = Some((x + screen::glyph_cols(&glyph), y));
168186 }
169187 queue!(self.out, style::ResetColor)?;
188+ // The pictures after the text, and before the cursor is put back: a
189+ // placement is drawn where the cursor is, so it moves the cursor, and
190+ // whatever the frame decided about the caret has to be the last word.
191+ let cell = graphics::cell();
192+ self.graphics.sync(&mut self.out, images, cell)?;
170193 match cursor {
171194 Some((x, y)) => queue!(self.out, cursor::MoveTo(x, y), cursor::Show)?,
172195 None => queue!(self.out, cursor::Hide)?,
@@ -214,3 +237,11 @@ fn write_style(out: &mut Stdout, style: Style) -> io::Result<()> {
214237 style::SetBackgroundColor(convert(style.bg))
215238 )
216239 }
240+
241+/// What one cell measures, asked of the terminal.
242+fn measure_cell() -> (u16, u16) {
243+ match crossterm::terminal::window_size() {
244+ Ok(size) => graphics::cell_pixels(size.columns, size.rows, size.width, size.height),
245+ Err(_) => graphics::cell_pixels(0, 0, 0, 0),
246+ }
247+}
@@ -17,6 +17,7 @@ use crossterm::terminal::{
17 use crossterm::{cursor, execute, queue, style};17 use crossterm::{cursor, execute, queue, style};
18 18
19 use crate::keys;19 use crate::keys;
20+use crate::graphics::{self, Graphics, Placement};
20 use crate::screen::{self, attr, Color, Screen, Style};21 use crate::screen::{self, attr, Color, Screen, Style};
21 22
22 /// How far one notch of the wheel moves a list, in rows.23 /// How far one notch of the wheel moves a list, in rows.
@@ -36,6 +37,8 @@ pub struct Term {
36 /// What is on the screen now, so a frame only sends what changed.37 /// What is on the screen now, so a frame only sends what changed.
37 last: Screen,38 last: Screen,
38 mouse: bool,39 mouse: bool,
40+ /// The pictures the terminal has been given, and where they are.
41+ graphics: Graphics,
39 }42 }
40 43
41 impl Term {44 impl Term {
@@ -49,10 +52,12 @@ impl Term {
49 if mouse {52 if mouse {
50 execute!(out, EnableMouseCapture)?;53 execute!(out, EnableMouseCapture)?;
51 }54 }
55+ graphics::set_cell(measure_cell());
52 Ok(Self {56 Ok(Self {
53 out,57 out,
54 last: Screen::new(w, h),58 last: Screen::new(w, h),
55 mouse,59 mouse,
60+ graphics: Graphics::default(),
56 })61 })
57 }62 }
58 63
@@ -64,6 +69,10 @@ impl Term {
64 /// hook — leaving a shell in raw mode with no cursor is the one failure a69 /// hook — leaving a shell in raw mode with no cursor is the one failure a
65 /// TUI must not have.70 /// TUI must not have.
66 pub fn close(&mut self) {71 pub fn close(&mut self) {
72+ // The pictures first: a placement is the terminal's, not the screen
73+ // buffer's, and one left behind outlives the alternate screen it was
74+ // made on.
75+ let _ = self.graphics.clear(&mut self.out);
67 if self.mouse {76 if self.mouse {
68 let _ = execute!(self.out, DisableMouseCapture);77 let _ = execute!(self.out, DisableMouseCapture);
69 }78 }
@@ -122,9 +131,18 @@ impl Term {
122 }131 }
123 132
124 /// Send whatever differs between `screen` and what is already up there.133 /// Send whatever differs between `screen` and what is already up there.
125- pub fn flush(&mut self, screen: &Screen, cursor: Option<(u16, u16)>) -> io::Result<()> {134+ pub fn flush(
135+ &mut self,
136+ screen: &Screen,
137+ cursor: Option<(u16, u16)>,
138+ images: &[Placement],
139+ ) -> io::Result<()> {
126 if (self.last.width(), self.last.height()) != (screen.width(), screen.height()) {140 if (self.last.width(), self.last.height()) != (screen.width(), screen.height()) {
127 self.last = Screen::new(screen.width(), screen.height());141 self.last = Screen::new(screen.width(), screen.height());
142+ // A resize is a new cell size as often as not — a font change, a
143+ // window dragged to another screen — and every picture's size is
144+ // measured in cells.
145+ graphics::set_cell(measure_cell());
128 queue!(146 queue!(
129 self.out,147 self.out,
130 crossterm::terminal::Clear(crossterm::terminal::ClearType::All)148 crossterm::terminal::Clear(crossterm::terminal::ClearType::All)
@@ -167,6 +185,11 @@ impl Term {
167 at = Some((x + screen::glyph_cols(&glyph), y));185 at = Some((x + screen::glyph_cols(&glyph), y));
168 }186 }
169 queue!(self.out, style::ResetColor)?;187 queue!(self.out, style::ResetColor)?;
188+ // The pictures after the text, and before the cursor is put back: a
189+ // placement is drawn where the cursor is, so it moves the cursor, and
190+ // whatever the frame decided about the caret has to be the last word.
191+ let cell = graphics::cell();
192+ self.graphics.sync(&mut self.out, images, cell)?;
170 match cursor {193 match cursor {
171 Some((x, y)) => queue!(self.out, cursor::MoveTo(x, y), cursor::Show)?,194 Some((x, y)) => queue!(self.out, cursor::MoveTo(x, y), cursor::Show)?,
172 None => queue!(self.out, cursor::Hide)?,195 None => queue!(self.out, cursor::Hide)?,
@@ -214,3 +237,11 @@ fn write_style(out: &mut Stdout, style: Style) -> io::Result<()> {
214 style::SetBackgroundColor(convert(style.bg))237 style::SetBackgroundColor(convert(style.bg))
215 )238 )
216 }239 }
240+
241+/// What one cell measures, asked of the terminal.
242+fn measure_cell() -> (u16, u16) {
243+ match crossterm::terminal::window_size() {
244+ Ok(size) => graphics::cell_pixels(size.columns, size.rows, size.width, size.height),
245+ Err(_) => graphics::cell_pixels(0, 0, 0, 0),
246+ }
247+}
modified crates/jolt-tui/src/tests.rs +80 -0
@@ -459,6 +459,86 @@ fn an_emoji_is_two_columns_wide_and_what_follows_it_knows_that() {
459459 );
460460 }
461461
462+/// A PNG that is only a header: this backend reads the size out of one and
463+/// hands the file to the terminal, so a header is all a test of the layout
464+/// needs.
465+fn png_file(name: &str, w: u32, h: u32) -> String {
466+ let mut bytes = vec![0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A];
467+ bytes.extend_from_slice(&13u32.to_be_bytes());
468+ bytes.extend_from_slice(b"IHDR");
469+ bytes.extend_from_slice(&w.to_be_bytes());
470+ bytes.extend_from_slice(&h.to_be_bytes());
471+ let path = std::env::temp_dir().join(name);
472+ std::fs::write(&path, &bytes).expect("wrote the picture");
473+ path.to_string_lossy().into_owned()
474+}
475+
476+#[test]
477+fn a_picture_takes_the_cells_its_shape_asks_for_and_says_where_it_is() {
478+ crate::graphics::force(Some(true));
479+ crate::graphics::set_cell((8, 16));
480+ let path = png_file("jolt-tui-wide.png", 320, 160);
481+
482+ let mut ui = Ui::new(60, 20);
483+ let root = ui.tree.root();
484+ node(&mut ui, root, "label", &[("label", "a picture:")]);
485+ let image = node(&mut ui, root, "image", &[("src", &path)]);
486+ ui.tree.set(image, "max-height", Value::Num(6.0));
487+ ui.frame();
488+
489+ // 320x160 pixels over 8x16 cells is 40 by 10, held to the six rows the
490+ // caller allowed — and the width comes down with it, not separately.
491+ let placed = ui.images().first().cloned().expect("a placement");
492+ assert_eq!((placed.area.w, placed.area.h), (24, 6));
493+ assert_eq!((placed.area.x, placed.area.y), (0, 1));
494+ assert_eq!(placed.path, path);
495+ // The cells themselves stay blank: the terminal draws over them.
496+ assert_eq!(ui.screen.line(1), "");
497+ crate::graphics::force(None);
498+}
499+
500+#[test]
501+fn a_picture_scrolling_past_the_edge_is_cropped_rather_than_lost() {
502+ crate::graphics::force(Some(true));
503+ crate::graphics::set_cell((8, 16));
504+ let path = png_file("jolt-tui-tall.png", 160, 320);
505+
506+ let mut ui = Ui::new(40, 4);
507+ let root = ui.tree.root();
508+ let scroll = node(&mut ui, root, "scroll", &[("scroll-key", "backlog")]);
509+ let image = node(&mut ui, scroll, "image", &[("src", &path)]);
510+ ui.tree.set(image, "max-height", Value::Num(8.0));
511+ node(&mut ui, scroll, "label", &[("label", "under it")]);
512+ ui.frame();
513+ let placed = ui.images().first().cloned().expect("a placement");
514+ assert_eq!((placed.area.y, placed.area.h), (0, 4), "as much as fits");
515+ assert_eq!((placed.crop_top, placed.crop_bottom), (0, 4));
516+
517+ ui.wheel(1, 1, 2);
518+ ui.frame();
519+ let placed = ui.images().first().cloned().expect("still placed");
520+ assert_eq!((placed.area.y, placed.area.h), (0, 4));
521+ assert_eq!(
522+ (placed.crop_top, placed.crop_bottom),
523+ (2, 2),
524+ "two rows of the picture have gone off the top"
525+ );
526+ crate::graphics::force(None);
527+}
528+
529+#[test]
530+fn a_terminal_that_draws_no_pictures_says_so_where_one_would_be() {
531+ crate::graphics::force(Some(false));
532+ let path = png_file("jolt-tui-note.png", 320, 160);
533+ let mut ui = Ui::new(40, 4);
534+ let root = ui.tree.root();
535+ node(&mut ui, root, "image", &[("src", &path)]);
536+ ui.frame();
537+ assert_eq!(ui.screen.line(0), "[ picture ]");
538+ assert!(ui.images().is_empty());
539+ crate::graphics::force(None);
540+}
541+
462542 #[test]
463543 fn an_overlay_floats_in_the_middle_over_whatever_was_under_it() {
464544 let mut ui = Ui::new(14, 5);
@@ -459,6 +459,86 @@ fn an_emoji_is_two_columns_wide_and_what_follows_it_knows_that() {
459 );459 );
460 }460 }
461 461
462+/// A PNG that is only a header: this backend reads the size out of one and
463+/// hands the file to the terminal, so a header is all a test of the layout
464+/// needs.
465+fn png_file(name: &str, w: u32, h: u32) -> String {
466+ let mut bytes = vec![0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A];
467+ bytes.extend_from_slice(&13u32.to_be_bytes());
468+ bytes.extend_from_slice(b"IHDR");
469+ bytes.extend_from_slice(&w.to_be_bytes());
470+ bytes.extend_from_slice(&h.to_be_bytes());
471+ let path = std::env::temp_dir().join(name);
472+ std::fs::write(&path, &bytes).expect("wrote the picture");
473+ path.to_string_lossy().into_owned()
474+}
475+
476+#[test]
477+fn a_picture_takes_the_cells_its_shape_asks_for_and_says_where_it_is() {
478+ crate::graphics::force(Some(true));
479+ crate::graphics::set_cell((8, 16));
480+ let path = png_file("jolt-tui-wide.png", 320, 160);
481+
482+ let mut ui = Ui::new(60, 20);
483+ let root = ui.tree.root();
484+ node(&mut ui, root, "label", &[("label", "a picture:")]);
485+ let image = node(&mut ui, root, "image", &[("src", &path)]);
486+ ui.tree.set(image, "max-height", Value::Num(6.0));
487+ ui.frame();
488+
489+ // 320x160 pixels over 8x16 cells is 40 by 10, held to the six rows the
490+ // caller allowed — and the width comes down with it, not separately.
491+ let placed = ui.images().first().cloned().expect("a placement");
492+ assert_eq!((placed.area.w, placed.area.h), (24, 6));
493+ assert_eq!((placed.area.x, placed.area.y), (0, 1));
494+ assert_eq!(placed.path, path);
495+ // The cells themselves stay blank: the terminal draws over them.
496+ assert_eq!(ui.screen.line(1), "");
497+ crate::graphics::force(None);
498+}
499+
500+#[test]
501+fn a_picture_scrolling_past_the_edge_is_cropped_rather_than_lost() {
502+ crate::graphics::force(Some(true));
503+ crate::graphics::set_cell((8, 16));
504+ let path = png_file("jolt-tui-tall.png", 160, 320);
505+
506+ let mut ui = Ui::new(40, 4);
507+ let root = ui.tree.root();
508+ let scroll = node(&mut ui, root, "scroll", &[("scroll-key", "backlog")]);
509+ let image = node(&mut ui, scroll, "image", &[("src", &path)]);
510+ ui.tree.set(image, "max-height", Value::Num(8.0));
511+ node(&mut ui, scroll, "label", &[("label", "under it")]);
512+ ui.frame();
513+ let placed = ui.images().first().cloned().expect("a placement");
514+ assert_eq!((placed.area.y, placed.area.h), (0, 4), "as much as fits");
515+ assert_eq!((placed.crop_top, placed.crop_bottom), (0, 4));
516+
517+ ui.wheel(1, 1, 2);
518+ ui.frame();
519+ let placed = ui.images().first().cloned().expect("still placed");
520+ assert_eq!((placed.area.y, placed.area.h), (0, 4));
521+ assert_eq!(
522+ (placed.crop_top, placed.crop_bottom),
523+ (2, 2),
524+ "two rows of the picture have gone off the top"
525+ );
526+ crate::graphics::force(None);
527+}
528+
529+#[test]
530+fn a_terminal_that_draws_no_pictures_says_so_where_one_would_be() {
531+ crate::graphics::force(Some(false));
532+ let path = png_file("jolt-tui-note.png", 320, 160);
533+ let mut ui = Ui::new(40, 4);
534+ let root = ui.tree.root();
535+ node(&mut ui, root, "image", &[("src", &path)]);
536+ ui.frame();
537+ assert_eq!(ui.screen.line(0), "[ picture ]");
538+ assert!(ui.images().is_empty());
539+ crate::graphics::force(None);
540+}
541+
462 #[test]542 #[test]
463 fn an_overlay_floats_in_the_middle_over_whatever_was_under_it() {543 fn an_overlay_floats_in_the_middle_over_whatever_was_under_it() {
464 let mut ui = Ui::new(14, 5);544 let mut ui = Ui::new(14, 5);
modified crates/jolt-tui/src/tree.rs +6 -0
@@ -45,6 +45,7 @@ pub enum Tag {
4545 Spinner,
4646 Reaction,
4747 Emoji,
48+ Image,
4849 Unknown(String),
4950 }
5051
@@ -80,6 +81,10 @@ impl Tag {
8081 // label, which is why an unknown tag painted neither.
8182 "reaction" => Self::Reaction,
8283 "emoji" => Self::Emoji,
84+ // A picture, which a cell grid cannot hold: the painter reserves
85+ // the cells and the terminal draws it over them, where it has the
86+ // protocol for that. See `crate::graphics`.
87+ "image" => Self::Image,
8388 other => Self::Unknown(other.to_owned()),
8489 }
8590 }
@@ -106,6 +111,7 @@ impl Tag {
106111 Self::Spinner => "spinner",
107112 Self::Reaction => "reaction",
108113 Self::Emoji => "emoji",
114+ Self::Image => "image",
109115 Self::Unknown(name) => name,
110116 }
111117 }
@@ -45,6 +45,7 @@ pub enum Tag {
45 Spinner,45 Spinner,
46 Reaction,46 Reaction,
47 Emoji,47 Emoji,
48+ Image,
48 Unknown(String),49 Unknown(String),
49 }50 }
50 51
@@ -80,6 +81,10 @@ impl Tag {
80 // label, which is why an unknown tag painted neither.81 // label, which is why an unknown tag painted neither.
81 "reaction" => Self::Reaction,82 "reaction" => Self::Reaction,
82 "emoji" => Self::Emoji,83 "emoji" => Self::Emoji,
84+ // A picture, which a cell grid cannot hold: the painter reserves
85+ // the cells and the terminal draws it over them, where it has the
86+ // protocol for that. See `crate::graphics`.
87+ "image" => Self::Image,
83 other => Self::Unknown(other.to_owned()),88 other => Self::Unknown(other.to_owned()),
84 }89 }
85 }90 }
@@ -106,6 +111,7 @@ impl Tag {
106 Self::Spinner => "spinner",111 Self::Spinner => "spinner",
107 Self::Reaction => "reaction",112 Self::Reaction => "reaction",
108 Self::Emoji => "emoji",113 Self::Emoji => "emoji",
114+ Self::Image => "image",
109 Self::Unknown(name) => name,115 Self::Unknown(name) => name,
110 }116 }
111 }117 }
modified crates/jolt-tui/src/ui.rs +5 -0
@@ -81,6 +81,11 @@ impl Ui {
8181 self.painted.cursor
8282 }
8383
84+ /// The pictures the last frame wants on screen, for whoever can draw one.
85+ pub fn images(&self) -> &[crate::graphics::Placement] {
86+ &self.painted.images
87+ }
88+
8489 /// Paint one frame, then settle the things painting decided: what is
8590 /// focusable now, and how far each scroll area really is.
8691 pub fn frame(&mut self) {
@@ -81,6 +81,11 @@ impl Ui {
81 self.painted.cursor81 self.painted.cursor
82 }82 }
83 83
84+ /// The pictures the last frame wants on screen, for whoever can draw one.
85+ pub fn images(&self) -> &[crate::graphics::Placement] {
86+ &self.painted.images
87+ }
88+
84 /// Paint one frame, then settle the things painting decided: what is89 /// Paint one frame, then settle the things painting decided: what is
85 /// focusable now, and how far each scroll area really is.90 /// focusable now, and how far each scroll area really is.
86 pub fn frame(&mut self) {91 pub fn frame(&mut self) {