feat(skills): add replace-aroll-take skill
Swap a stretch of the editing A-roll for a re-recorded take: transcribe it with ElevenLabs, re-place motions, sounds and zooms on the new delivery (retiming composition cues through a piecewise map when the pace changed), shift what follows, and save it as a single undoable history step.
5c16cb4 parent: 4e701a2 added
.claude/skills/replace-aroll-take/SKILL.md +142 -0 | new file mode 100644 | ||
| @@ -0,0 +1,142 @@ | ||
| 1 | +--- | |
| 2 | +name: replace-aroll-take | |
| 3 | +description: Replace a stretch of the editing timeline's A-roll (between two blade cuts, or up to the end) with a re-recorded take dropped in the project's resources, keeping everything else — B-roll, motions, sound design, zooms, texts — re-placed on the new delivery using word-level transcripts. Use when the user says they re-filmed a part ("j'ai refilmé…", "remplace de 11:35 à 13:44 par anonymat-partie6.mp4", "mets anonymat-outro à la place de la fin"). | |
| 4 | +--- | |
| 5 | + | |
| 6 | +# Replace an A-roll stretch with a new take | |
| 7 | + | |
| 8 | +The user re-recorded part of the video. They mark the stretch with blade cuts in the editing step | |
| 9 | +(`/p/<pid>/montage`) and drop the new file in the project's resources. Your job: swap the A-roll, | |
| 10 | +transcribe the new take, re-place every element that sat on the old delivery so it still lands on its | |
| 11 | +words, shift everything after, and report what no longer matches. The user decides what to cut or remove: | |
| 12 | +never trim their hesitations, never delete their motions without saying so. | |
| 13 | + | |
| 14 | +Talk to the user in French. Scripts, comments and config are English. | |
| 15 | + | |
| 16 | +## Setup | |
| 17 | + | |
| 18 | +- Project data: `data/projects/<pid>/` (`montage/timeline.json`, `montage/motion/<id>.html`, `ressources/`). | |
| 19 | + Timeline schema: docstring at the top of `backend/studio/modules/projects/montage.py`. | |
| 20 | +- Run scripts with the backend venv, from anywhere: | |
| 21 | + `PYTHONIOENCODING=utf-8 .venv/Scripts/python.exe .claude/skills/replace-aroll-take/scripts/<script>.py …` | |
| 22 | +- Scratch files (transcripts, configs) go to `%TEMP%/replace-aroll-take/<pid>/` (`common.work_dir`). | |
| 23 | + Never name a scratch script after a stdlib module (`inspect.py` broke the backend import once). | |
| 24 | + | |
| 25 | +## Steps | |
| 26 | + | |
| 27 | +### 1. Find the zone | |
| 28 | + | |
| 29 | +`zone.py <pid> clips` lists the A-roll clips with their timeline start/end. The user's times are | |
| 30 | +approximate: the zone is the clip run between the two blade cuts nearest to them (a cut between two | |
| 31 | +contiguous source clips, `out` of one == `in` of the next). "Jusqu'à la fin" means up to the last clip. | |
| 32 | + | |
| 33 | +- The zone can contain clips inserted from other media (a meme answering a line, `src` set). Keep them | |
| 34 | + and re-insert them at the same line in the new take. If the user had cut the line the gag answers, | |
| 35 | + cut it in the new take too (that's their earlier decision, not yours). Say so in the report. | |
| 36 | +- Elements placed just before the first cut (a part title, its whoosh) stay where they are; elements just | |
| 37 | + before the end cut that announce the next part follow the end (the `tail` setting, 1 s by default). | |
| 38 | + | |
| 39 | +### 2. Transcribe and compare | |
| 40 | + | |
| 41 | +- `transcribe.py <pid> ressources/<take>.mp4 [more takes]` — ElevenLabs Scribe v2 (key in | |
| 42 | + `data/transcription.json`), several files in parallel. Prints the phrases with times in the file. | |
| 43 | +- `zone.py <pid> old <a> <b>` — the old words on the timeline, and every element touching the zone | |
| 44 | + (motions with their idea, sounds with their `motion`, zooms with their `beat`). | |
| 45 | +- `zone.py <pid> new <raw.json> <a> [<in>]` — the new take's words in timeline time. | |
| 46 | +- Loudness: `ffmpeg -ss <src_in> -t <len> -i <old media> -vn -af ebur128 -f null -` vs the new file; | |
| 47 | + the new clip's `gain_db` = old LUFS − new LUFS (keep the old clips' own gain in mind). | |
| 48 | + | |
| 49 | +### 3. Decide where everything goes | |
| 50 | + | |
| 51 | +The new take is usually rewritten: lines appear, disappear, change order. Work element by element, with | |
| 52 | +the words in front of you. For every element on the zone, pick one: | |
| 53 | + | |
| 54 | +- **Moved as is** (`groups`): its words are said the same way. Anchor it at the same distance from the | |
| 55 | + same word. Sounds tied to a motion (`"motion": id`) follow automatically. | |
| 56 | +- **Retimed inside** (`motions`): a motion whose internal cues (comments like `// « chercheurs » (1.4)`) | |
| 57 | + no longer match the new pace. Give knots `(old_t, new_t)` relative to the motion start: one per cue | |
| 58 | + that must land on a word (new_t = new word time − new start − the same lead as before, usually 0.2 s), | |
| 59 | + plus the exit and the end. The new `dur` usually changes. Sounds tied to it, and untied ones you list in | |
| 60 | + `sounds` (a pop just before the entry, a ding on a counter), go through the same map. Helper calls | |
| 61 | + (`lever("#i1", 2.56, 3.76)`, `panel(".p1", 8.45)`, `step(2, 1.06, …)`, `light(…, 2.6)`) need a regex | |
| 62 | + in `helpers`; tween positions `}, t)` and the composition's `<script id="sfx">` cues are handled. | |
| 63 | +- **Trimmed** (`durs`): only to avoid an overlap, cutting the tail of a motion whose end matters least. | |
| 64 | +- **Fixed** (`fixed`): a single sound or zoom set to an absolute time. | |
| 65 | +- **Dropped** (`drop`): only for things that are plainly obsolete with the old take (a handwritten note | |
| 66 | + about the old take's sound problem). Always report it. | |
| 67 | +- Content the new take no longer says (a figure, a study): keep the motion at the closest meaningful | |
| 68 | + place, trimmed if needed, and list it under « à trier ». | |
| 69 | + | |
| 70 | +Constraints to check: motions must not overlap; a motion can't cover an inserted A-roll clip (the gag | |
| 71 | +must be seen); a video inside a composition (`<video data-duration>`) must be long enough if you lengthen | |
| 72 | +the motion (fix it with `edits`). | |
| 73 | + | |
| 74 | +### 4. Cut the new take | |
| 75 | + | |
| 76 | +- `in`: keep the same lead before the first word as the old cut had (≈0.15–0.4 s). | |
| 77 | +- `out`: same tail after the last word; if the next clip has a transition, leave at least half its | |
| 78 | + duration of media after the out point (the FCPXML transition eats it); never past the file's end. | |
| 79 | +- An internal transition of the old stretch (a punch-in `hblur` at a pivot line) is kept by splitting the | |
| 80 | + new take in two clips at the matching line (cut in the silence before it). | |
| 81 | + | |
| 82 | +### 5. Write the config, dry-run, apply | |
| 83 | + | |
| 84 | +```python | |
| 85 | +# %TEMP%/replace-aroll-take/<pid>/zone1.py | |
| 86 | +SRC = "ressources/anonymat-partie6.mp4" | |
| 87 | +CFG = { | |
| 88 | + "pid": "f728f85ffb", | |
| 89 | + "clips": ["a10"], # the old clip run, in order | |
| 90 | + "old": (694.4, 824.456), # its timeline span (safety check: aborts if the timeline moved) | |
| 91 | + "aroll": [ # the replacement, in order; an id string reuses a clip of the run as is | |
| 92 | + {"src": SRC, "in": 0.0, "out": 86.82, "gain_db": -1.2}, | |
| 93 | + "a7", # the meme kept between the two halves | |
| 94 | + {"src": SRC, "in": 88.87, "out": 174.56, "gain_db": -1.2}, | |
| 95 | + # {"src": SRC, "in": …, "out": …, "transition": {"fx": "hblur", "dur": 0.5}, "gain_db": …}, | |
| 96 | + ], | |
| 97 | + "raw": r"C:\Users\…\Temp\replace-aroll-take\f728f85ffb\anonymat-partie6_raw.json", | |
| 98 | + "motions": { # retimed inside | |
| 99 | + "m48": {"at": 721.64, "dur": 21.36, "sounds": ["s57"], | |
| 100 | + "knots": [(0, 0), (1.56, 1.56), (2.56, 2.64), (3.76, 3.82), (11.26, 20.96), (11.66, 21.36)], | |
| 101 | + "helpers": [r'lever\("#i\d", (?P<t1>-?\d+(?:\.\d+)?), (?P<t2>-?\d+(?:\.\d+)?)\)']}, | |
| 102 | + }, | |
| 103 | + "groups": [(794.84, ["m49"]), (830.17, ["z24", "z26", "z25"])], # (new time of the first id, ids moved together) | |
| 104 | + "fixed": {"s88": 326.45}, | |
| 105 | + "durs": {"m58": 4.95}, | |
| 106 | + "drop": [], | |
| 107 | + "edits": {"m70.html": [('data-duration="5.01"', 'data-duration="7"')]}, # exact one-off replacements | |
| 108 | +} | |
| 109 | +``` | |
| 110 | + | |
| 111 | +`replace.py <config> --dry` first: it prints the new end, the delta applied to everything after, the | |
| 112 | +motion spans around the zone with `OVERLAP` marks, and how many instants each retimed composition gets. | |
| 113 | +It aborts if an element in the zone has no decision. Then run it without `--dry`. It: | |
| 114 | + | |
| 115 | +1. saves the timeline through `montage.save` — one history step that holds the timeline **and every | |
| 116 | + composition before the retiming**, plus the storyboard beat shift, so a single Ctrl+Z in the UI undoes | |
| 117 | + the whole replacement (do the compositions only after this, never before); | |
| 118 | +2. moves the storyboard beats of re-placed elements with them; | |
| 119 | +3. rewrites the retimed compositions (keeping their CRLF line endings) and applies `edits`; | |
| 120 | +4. queues the motion renders and rewrites the FCPXML (`montage.after_change`); | |
| 121 | +5. writes `transcripts/<take>.srt/.txt` in the project. | |
| 122 | + | |
| 123 | +Several zones in one request: apply them one after the other, re-reading `zone.py clips` in between | |
| 124 | +(the first one shifts the second). | |
| 125 | + | |
| 126 | +### 6. Verify | |
| 127 | + | |
| 128 | +- `zone.py <pid> clips`: the new clips, the total length (old total + delta). | |
| 129 | +- `montage.build(pid)` problems: only the usual ones (non-dissolve transitions, texts as markers, | |
| 130 | + motions "pas encore rendu" while renders run). | |
| 131 | +- For retimed compositions, `diff --strip-trailing-cr` against `montage/.history/<last>/motion/<id>.html`: | |
| 132 | + only instants changed. Cue comments keep their old word times; that's fine. | |
| 133 | + | |
| 134 | +## Report (in French) | |
| 135 | + | |
| 136 | +- What was replaced (zone, file, gain, delta, final length), what was kept (gags) and cut (if you mirrored | |
| 137 | + an earlier cut of theirs). | |
| 138 | +- Each motion: moved, retimed (on which words), trimmed — and its new length when it changed. | |
| 139 | +- **À trier**: motions showing things no longer said, cues that land a bit early/late and why, anything | |
| 140 | + dropped, new passages with nothing on screen (with timeline ranges). | |
| 141 | +- Remind them to reload the editing tab, and that one Ctrl+Z (focus on the timeline, not a text field) | |
| 142 | + undoes the whole replacement. | |
| new file mode 100644 | |||
| @@ -0,0 +1,142 @@ | |||
| 1 | +--- | ||
| 2 | +name: replace-aroll-take | ||
| 3 | +description: Replace a stretch of the editing timeline's A-roll (between two blade cuts, or up to the end) with a re-recorded take dropped in the project's resources, keeping everything else — B-roll, motions, sound design, zooms, texts — re-placed on the new delivery using word-level transcripts. Use when the user says they re-filmed a part ("j'ai refilmé…", "remplace de 11:35 à 13:44 par anonymat-partie6.mp4", "mets anonymat-outro à la place de la fin"). | ||
| 4 | +--- | ||
| 5 | + | ||
| 6 | +# Replace an A-roll stretch with a new take | ||
| 7 | + | ||
| 8 | +The user re-recorded part of the video. They mark the stretch with blade cuts in the editing step | ||
| 9 | +(`/p/<pid>/montage`) and drop the new file in the project's resources. Your job: swap the A-roll, | ||
| 10 | +transcribe the new take, re-place every element that sat on the old delivery so it still lands on its | ||
| 11 | +words, shift everything after, and report what no longer matches. The user decides what to cut or remove: | ||
| 12 | +never trim their hesitations, never delete their motions without saying so. | ||
| 13 | + | ||
| 14 | +Talk to the user in French. Scripts, comments and config are English. | ||
| 15 | + | ||
| 16 | +## Setup | ||
| 17 | + | ||
| 18 | +- Project data: `data/projects/<pid>/` (`montage/timeline.json`, `montage/motion/<id>.html`, `ressources/`). | ||
| 19 | + Timeline schema: docstring at the top of `backend/studio/modules/projects/montage.py`. | ||
| 20 | +- Run scripts with the backend venv, from anywhere: | ||
| 21 | + `PYTHONIOENCODING=utf-8 .venv/Scripts/python.exe .claude/skills/replace-aroll-take/scripts/<script>.py …` | ||
| 22 | +- Scratch files (transcripts, configs) go to `%TEMP%/replace-aroll-take/<pid>/` (`common.work_dir`). | ||
| 23 | + Never name a scratch script after a stdlib module (`inspect.py` broke the backend import once). | ||
| 24 | + | ||
| 25 | +## Steps | ||
| 26 | + | ||
| 27 | +### 1. Find the zone | ||
| 28 | + | ||
| 29 | +`zone.py <pid> clips` lists the A-roll clips with their timeline start/end. The user's times are | ||
| 30 | +approximate: the zone is the clip run between the two blade cuts nearest to them (a cut between two | ||
| 31 | +contiguous source clips, `out` of one == `in` of the next). "Jusqu'à la fin" means up to the last clip. | ||
| 32 | + | ||
| 33 | +- The zone can contain clips inserted from other media (a meme answering a line, `src` set). Keep them | ||
| 34 | + and re-insert them at the same line in the new take. If the user had cut the line the gag answers, | ||
| 35 | + cut it in the new take too (that's their earlier decision, not yours). Say so in the report. | ||
| 36 | +- Elements placed just before the first cut (a part title, its whoosh) stay where they are; elements just | ||
| 37 | + before the end cut that announce the next part follow the end (the `tail` setting, 1 s by default). | ||
| 38 | + | ||
| 39 | +### 2. Transcribe and compare | ||
| 40 | + | ||
| 41 | +- `transcribe.py <pid> ressources/<take>.mp4 [more takes]` — ElevenLabs Scribe v2 (key in | ||
| 42 | + `data/transcription.json`), several files in parallel. Prints the phrases with times in the file. | ||
| 43 | +- `zone.py <pid> old <a> <b>` — the old words on the timeline, and every element touching the zone | ||
| 44 | + (motions with their idea, sounds with their `motion`, zooms with their `beat`). | ||
| 45 | +- `zone.py <pid> new <raw.json> <a> [<in>]` — the new take's words in timeline time. | ||
| 46 | +- Loudness: `ffmpeg -ss <src_in> -t <len> -i <old media> -vn -af ebur128 -f null -` vs the new file; | ||
| 47 | + the new clip's `gain_db` = old LUFS − new LUFS (keep the old clips' own gain in mind). | ||
| 48 | + | ||
| 49 | +### 3. Decide where everything goes | ||
| 50 | + | ||
| 51 | +The new take is usually rewritten: lines appear, disappear, change order. Work element by element, with | ||
| 52 | +the words in front of you. For every element on the zone, pick one: | ||
| 53 | + | ||
| 54 | +- **Moved as is** (`groups`): its words are said the same way. Anchor it at the same distance from the | ||
| 55 | + same word. Sounds tied to a motion (`"motion": id`) follow automatically. | ||
| 56 | +- **Retimed inside** (`motions`): a motion whose internal cues (comments like `// « chercheurs » (1.4)`) | ||
| 57 | + no longer match the new pace. Give knots `(old_t, new_t)` relative to the motion start: one per cue | ||
| 58 | + that must land on a word (new_t = new word time − new start − the same lead as before, usually 0.2 s), | ||
| 59 | + plus the exit and the end. The new `dur` usually changes. Sounds tied to it, and untied ones you list in | ||
| 60 | + `sounds` (a pop just before the entry, a ding on a counter), go through the same map. Helper calls | ||
| 61 | + (`lever("#i1", 2.56, 3.76)`, `panel(".p1", 8.45)`, `step(2, 1.06, …)`, `light(…, 2.6)`) need a regex | ||
| 62 | + in `helpers`; tween positions `}, t)` and the composition's `<script id="sfx">` cues are handled. | ||
| 63 | +- **Trimmed** (`durs`): only to avoid an overlap, cutting the tail of a motion whose end matters least. | ||
| 64 | +- **Fixed** (`fixed`): a single sound or zoom set to an absolute time. | ||
| 65 | +- **Dropped** (`drop`): only for things that are plainly obsolete with the old take (a handwritten note | ||
| 66 | + about the old take's sound problem). Always report it. | ||
| 67 | +- Content the new take no longer says (a figure, a study): keep the motion at the closest meaningful | ||
| 68 | + place, trimmed if needed, and list it under « à trier ». | ||
| 69 | + | ||
| 70 | +Constraints to check: motions must not overlap; a motion can't cover an inserted A-roll clip (the gag | ||
| 71 | +must be seen); a video inside a composition (`<video data-duration>`) must be long enough if you lengthen | ||
| 72 | +the motion (fix it with `edits`). | ||
| 73 | + | ||
| 74 | +### 4. Cut the new take | ||
| 75 | + | ||
| 76 | +- `in`: keep the same lead before the first word as the old cut had (≈0.15–0.4 s). | ||
| 77 | +- `out`: same tail after the last word; if the next clip has a transition, leave at least half its | ||
| 78 | + duration of media after the out point (the FCPXML transition eats it); never past the file's end. | ||
| 79 | +- An internal transition of the old stretch (a punch-in `hblur` at a pivot line) is kept by splitting the | ||
| 80 | + new take in two clips at the matching line (cut in the silence before it). | ||
| 81 | + | ||
| 82 | +### 5. Write the config, dry-run, apply | ||
| 83 | + | ||
| 84 | +```python | ||
| 85 | +# %TEMP%/replace-aroll-take/<pid>/zone1.py | ||
| 86 | +SRC = "ressources/anonymat-partie6.mp4" | ||
| 87 | +CFG = { | ||
| 88 | + "pid": "f728f85ffb", | ||
| 89 | + "clips": ["a10"], # the old clip run, in order | ||
| 90 | + "old": (694.4, 824.456), # its timeline span (safety check: aborts if the timeline moved) | ||
| 91 | + "aroll": [ # the replacement, in order; an id string reuses a clip of the run as is | ||
| 92 | + {"src": SRC, "in": 0.0, "out": 86.82, "gain_db": -1.2}, | ||
| 93 | + "a7", # the meme kept between the two halves | ||
| 94 | + {"src": SRC, "in": 88.87, "out": 174.56, "gain_db": -1.2}, | ||
| 95 | + # {"src": SRC, "in": …, "out": …, "transition": {"fx": "hblur", "dur": 0.5}, "gain_db": …}, | ||
| 96 | + ], | ||
| 97 | + "raw": r"C:\Users\…\Temp\replace-aroll-take\f728f85ffb\anonymat-partie6_raw.json", | ||
| 98 | + "motions": { # retimed inside | ||
| 99 | + "m48": {"at": 721.64, "dur": 21.36, "sounds": ["s57"], | ||
| 100 | + "knots": [(0, 0), (1.56, 1.56), (2.56, 2.64), (3.76, 3.82), (11.26, 20.96), (11.66, 21.36)], | ||
| 101 | + "helpers": [r'lever\("#i\d", (?P<t1>-?\d+(?:\.\d+)?), (?P<t2>-?\d+(?:\.\d+)?)\)']}, | ||
| 102 | + }, | ||
| 103 | + "groups": [(794.84, ["m49"]), (830.17, ["z24", "z26", "z25"])], # (new time of the first id, ids moved together) | ||
| 104 | + "fixed": {"s88": 326.45}, | ||
| 105 | + "durs": {"m58": 4.95}, | ||
| 106 | + "drop": [], | ||
| 107 | + "edits": {"m70.html": [('data-duration="5.01"', 'data-duration="7"')]}, # exact one-off replacements | ||
| 108 | +} | ||
| 109 | +``` | ||
| 110 | + | ||
| 111 | +`replace.py <config> --dry` first: it prints the new end, the delta applied to everything after, the | ||
| 112 | +motion spans around the zone with `OVERLAP` marks, and how many instants each retimed composition gets. | ||
| 113 | +It aborts if an element in the zone has no decision. Then run it without `--dry`. It: | ||
| 114 | + | ||
| 115 | +1. saves the timeline through `montage.save` — one history step that holds the timeline **and every | ||
| 116 | + composition before the retiming**, plus the storyboard beat shift, so a single Ctrl+Z in the UI undoes | ||
| 117 | + the whole replacement (do the compositions only after this, never before); | ||
| 118 | +2. moves the storyboard beats of re-placed elements with them; | ||
| 119 | +3. rewrites the retimed compositions (keeping their CRLF line endings) and applies `edits`; | ||
| 120 | +4. queues the motion renders and rewrites the FCPXML (`montage.after_change`); | ||
| 121 | +5. writes `transcripts/<take>.srt/.txt` in the project. | ||
| 122 | + | ||
| 123 | +Several zones in one request: apply them one after the other, re-reading `zone.py clips` in between | ||
| 124 | +(the first one shifts the second). | ||
| 125 | + | ||
| 126 | +### 6. Verify | ||
| 127 | + | ||
| 128 | +- `zone.py <pid> clips`: the new clips, the total length (old total + delta). | ||
| 129 | +- `montage.build(pid)` problems: only the usual ones (non-dissolve transitions, texts as markers, | ||
| 130 | + motions "pas encore rendu" while renders run). | ||
| 131 | +- For retimed compositions, `diff --strip-trailing-cr` against `montage/.history/<last>/motion/<id>.html`: | ||
| 132 | + only instants changed. Cue comments keep their old word times; that's fine. | ||
| 133 | + | ||
| 134 | +## Report (in French) | ||
| 135 | + | ||
| 136 | +- What was replaced (zone, file, gain, delta, final length), what was kept (gags) and cut (if you mirrored | ||
| 137 | + an earlier cut of theirs). | ||
| 138 | +- Each motion: moved, retimed (on which words), trimmed — and its new length when it changed. | ||
| 139 | +- **À trier**: motions showing things no longer said, cues that land a bit early/late and why, anything | ||
| 140 | + dropped, new passages with nothing on screen (with timeline ranges). | ||
| 141 | +- Remind them to reload the editing tab, and that one Ctrl+Z (focus on the timeline, not a text field) | ||
| 142 | + undoes the whole replacement. | ||
added
.claude/skills/replace-aroll-take/scripts/common.py +56 -0 | new file mode 100644 | ||
| @@ -0,0 +1,56 @@ | ||
| 1 | +"""Shared helpers: repo paths, backend imports, the source transcript, A-roll geometry.""" | |
| 2 | +import json | |
| 3 | +import sqlite3 | |
| 4 | +import sys | |
| 5 | +import tempfile | |
| 6 | +from pathlib import Path | |
| 7 | + | |
| 8 | +REPO = Path(__file__).resolve().parents[4] | |
| 9 | +DATA = REPO / "data" | |
| 10 | +sys.path.insert(0, str(REPO / "backend")) | |
| 11 | + | |
| 12 | + | |
| 13 | +def project_dir(pid: str) -> Path: | |
| 14 | + return DATA / "projects" / pid | |
| 15 | + | |
| 16 | + | |
| 17 | +def work_dir(pid: str) -> Path: | |
| 18 | + """Scratch space for transcripts and configs (outside the repo and outside data/).""" | |
| 19 | + d = Path(tempfile.gettempdir()) / "replace-aroll-take" / pid | |
| 20 | + d.mkdir(parents=True, exist_ok=True) | |
| 21 | + return d | |
| 22 | + | |
| 23 | + | |
| 24 | +def timeline(pid: str) -> dict: | |
| 25 | + return json.loads((project_dir(pid) / "montage" / "timeline.json").read_text(encoding="utf-8")) | |
| 26 | + | |
| 27 | + | |
| 28 | +def clip_starts(tl: dict) -> list[float]: | |
| 29 | + out, t = [], 0.0 | |
| 30 | + for c in tl["aroll"]: | |
| 31 | + out.append(round(t, 3)) | |
| 32 | + t += c["out"] - c["in"] | |
| 33 | + return out | |
| 34 | + | |
| 35 | + | |
| 36 | +def source_segments(pid: str) -> list[dict]: | |
| 37 | + """Phrases (with word timings) of the main source video, from its transcription job.""" | |
| 38 | + project = json.loads((project_dir(pid) / "project.json").read_text(encoding="utf-8")) | |
| 39 | + job = (project.get("source") or {}).get("transcript") | |
| 40 | + if not job: | |
| 41 | + return [] | |
| 42 | + db = sqlite3.connect(DATA / "studio.db") | |
| 43 | + row = db.execute("select state from jobs where id = ?", (job,)).fetchone() | |
| 44 | + return json.loads(row[0]).get("segments", []) if row else [] | |
| 45 | + | |
| 46 | + | |
| 47 | +def source_words_on_timeline(pid: str, tl: dict, a: float, b: float) -> list[tuple[float, str]]: | |
| 48 | + """Words of the main source as they sit on the timeline between a and b (clips with a `src` have none).""" | |
| 49 | + words = [(w[0], w[2].strip()) for s in source_segments(pid) for w in s["words"]] | |
| 50 | + out = [] | |
| 51 | + for c, start in zip(tl["aroll"], clip_starts(tl)): | |
| 52 | + end = start + c["out"] - c["in"] | |
| 53 | + if c.get("src") or end <= a or start >= b: | |
| 54 | + continue | |
| 55 | + out += [(round(start + t - c["in"], 2), w) for t, w in words if c["in"] <= t < c["out"]] | |
| 56 | + return sorted(x for x in out if a - 3 <= x[0] < b + 3) | |
| new file mode 100644 | |||
| @@ -0,0 +1,56 @@ | |||
| 1 | +"""Shared helpers: repo paths, backend imports, the source transcript, A-roll geometry.""" | ||
| 2 | +import json | ||
| 3 | +import sqlite3 | ||
| 4 | +import sys | ||
| 5 | +import tempfile | ||
| 6 | +from pathlib import Path | ||
| 7 | + | ||
| 8 | +REPO = Path(__file__).resolve().parents[4] | ||
| 9 | +DATA = REPO / "data" | ||
| 10 | +sys.path.insert(0, str(REPO / "backend")) | ||
| 11 | + | ||
| 12 | + | ||
| 13 | +def project_dir(pid: str) -> Path: | ||
| 14 | + return DATA / "projects" / pid | ||
| 15 | + | ||
| 16 | + | ||
| 17 | +def work_dir(pid: str) -> Path: | ||
| 18 | + """Scratch space for transcripts and configs (outside the repo and outside data/).""" | ||
| 19 | + d = Path(tempfile.gettempdir()) / "replace-aroll-take" / pid | ||
| 20 | + d.mkdir(parents=True, exist_ok=True) | ||
| 21 | + return d | ||
| 22 | + | ||
| 23 | + | ||
| 24 | +def timeline(pid: str) -> dict: | ||
| 25 | + return json.loads((project_dir(pid) / "montage" / "timeline.json").read_text(encoding="utf-8")) | ||
| 26 | + | ||
| 27 | + | ||
| 28 | +def clip_starts(tl: dict) -> list[float]: | ||
| 29 | + out, t = [], 0.0 | ||
| 30 | + for c in tl["aroll"]: | ||
| 31 | + out.append(round(t, 3)) | ||
| 32 | + t += c["out"] - c["in"] | ||
| 33 | + return out | ||
| 34 | + | ||
| 35 | + | ||
| 36 | +def source_segments(pid: str) -> list[dict]: | ||
| 37 | + """Phrases (with word timings) of the main source video, from its transcription job.""" | ||
| 38 | + project = json.loads((project_dir(pid) / "project.json").read_text(encoding="utf-8")) | ||
| 39 | + job = (project.get("source") or {}).get("transcript") | ||
| 40 | + if not job: | ||
| 41 | + return [] | ||
| 42 | + db = sqlite3.connect(DATA / "studio.db") | ||
| 43 | + row = db.execute("select state from jobs where id = ?", (job,)).fetchone() | ||
| 44 | + return json.loads(row[0]).get("segments", []) if row else [] | ||
| 45 | + | ||
| 46 | + | ||
| 47 | +def source_words_on_timeline(pid: str, tl: dict, a: float, b: float) -> list[tuple[float, str]]: | ||
| 48 | + """Words of the main source as they sit on the timeline between a and b (clips with a `src` have none).""" | ||
| 49 | + words = [(w[0], w[2].strip()) for s in source_segments(pid) for w in s["words"]] | ||
| 50 | + out = [] | ||
| 51 | + for c, start in zip(tl["aroll"], clip_starts(tl)): | ||
| 52 | + end = start + c["out"] - c["in"] | ||
| 53 | + if c.get("src") or end <= a or start >= b: | ||
| 54 | + continue | ||
| 55 | + out += [(round(start + t - c["in"], 2), w) for t, w in words if c["in"] <= t < c["out"]] | ||
| 56 | + return sorted(x for x in out if a - 3 <= x[0] < b + 3) | ||
added
.claude/skills/replace-aroll-take/scripts/remap.py +70 -0 | new file mode 100644 | ||
| @@ -0,0 +1,70 @@ | ||
| 1 | +"""Retime the inside of a motion composition: every instant goes through a piecewise-linear map.""" | |
| 2 | +import re | |
| 3 | +from bisect import bisect_right | |
| 4 | +from pathlib import Path | |
| 5 | + | |
| 6 | +NUM = r"-?\d+(?:\.\d+)?" | |
| 7 | + | |
| 8 | + | |
| 9 | +def mapper(knots): | |
| 10 | + """knots: [(old_t, new_t), ...] relative to the motion start; slope 1 outside the knots.""" | |
| 11 | + xs, ys = zip(*sorted(knots)) | |
| 12 | + assert all(b > a for a, b in zip(ys, ys[1:])), "knots must stay in order" | |
| 13 | + | |
| 14 | + def f(x): | |
| 15 | + if x <= xs[0]: | |
| 16 | + return ys[0] + (x - xs[0]) | |
| 17 | + if x >= xs[-1]: | |
| 18 | + return ys[-1] + (x - xs[-1]) | |
| 19 | + i = bisect_right(xs, x) - 1 | |
| 20 | + return ys[i] + (x - xs[i]) * (ys[i + 1] - ys[i]) / (xs[i + 1] - xs[i]) | |
| 21 | + return f | |
| 22 | + | |
| 23 | + | |
| 24 | +def fmt(x): | |
| 25 | + return f"{round(x, 2):.2f}".rstrip("0").rstrip(".") if abs(x) > 1e-9 else "0" | |
| 26 | + | |
| 27 | + | |
| 28 | +def remap(path: Path, knots, dur, helpers=(), dry=False): | |
| 29 | + """ | |
| 30 | + Rewrites in place: the root data-duration / data-hero, the final `tl.to({}, { duration: D }, 0)`, every tween | |
| 31 | + position (`}, t)`), helper calls matched by `helpers` (regexes whose named groups t1, t2… are instants, e.g. | |
| 32 | + r'lever\\("#i\\d", (?P<t1>…), (?P<t2>…)\\)'), and the "t" of each cue in <script id="sfx">. | |
| 33 | + Returns (map, number of instants rewritten). | |
| 34 | + """ | |
| 35 | + f = mapper(knots) | |
| 36 | + name = path.stem | |
| 37 | + s = path.read_text(encoding="utf-8") | |
| 38 | + n = 0 | |
| 39 | + | |
| 40 | + def sub(pattern, text, fn, flags=0): | |
| 41 | + nonlocal n | |
| 42 | + out, k = re.subn(pattern, fn, text, flags=flags) | |
| 43 | + n += k | |
| 44 | + return out | |
| 45 | + | |
| 46 | + s = sub(rf'(data-composition-id="{name}" data-duration=")({NUM})(")', s, lambda m: m[1] + fmt(dur) + m[3]) | |
| 47 | + s = sub(rf'(data-hero=")({NUM})(")', s, lambda m: m[1] + fmt(min(f(float(m[2])), dur - 0.2)) + m[3]) | |
| 48 | + s = sub(rf"(tl\.to\(\{{\}}, \{{ duration: )({NUM})( \}}, 0\))", s, lambda m: m[1] + fmt(dur) + m[3]) | |
| 49 | + s = sub(rf"(\}}, )({NUM})(\))", s, lambda m: m[1] + fmt(f(float(m[2]))) + m[3]) | |
| 50 | + for pattern in helpers: | |
| 51 | + def fix(m): | |
| 52 | + g = m.group(0) | |
| 53 | + for key in sorted((k for k in m.re.groupindex if k.startswith("t")), reverse=True): | |
| 54 | + a, b = m.start(key) - m.start(), m.end(key) - m.start() | |
| 55 | + g = g[:a] + fmt(f(float(m.group(key)))) + g[b:] | |
| 56 | + return g | |
| 57 | + s = sub(pattern, s, fix) | |
| 58 | + s = sub(r'<script type="application/json" id="sfx">.*?</script>', s, | |
| 59 | + lambda m: re.sub(rf'("t": )({NUM})', lambda k: k[1] + fmt(f(float(k[2]))), m[0]), flags=re.S) | |
| 60 | + if not dry: | |
| 61 | + write_like(path, s) | |
| 62 | + return f, n | |
| 63 | + | |
| 64 | + | |
| 65 | +def write_like(path: Path, text: str) -> None: | |
| 66 | + """Writes text keeping the file's own line endings (compositions are CRLF on this machine).""" | |
| 67 | + data = text.replace("\r\n", "\n").encode("utf-8") | |
| 68 | + if b"\r\n" in path.read_bytes(): | |
| 69 | + data = data.replace(b"\n", b"\r\n") | |
| 70 | + path.write_bytes(data) | |
| new file mode 100644 | |||
| @@ -0,0 +1,70 @@ | |||
| 1 | +"""Retime the inside of a motion composition: every instant goes through a piecewise-linear map.""" | ||
| 2 | +import re | ||
| 3 | +from bisect import bisect_right | ||
| 4 | +from pathlib import Path | ||
| 5 | + | ||
| 6 | +NUM = r"-?\d+(?:\.\d+)?" | ||
| 7 | + | ||
| 8 | + | ||
| 9 | +def mapper(knots): | ||
| 10 | + """knots: [(old_t, new_t), ...] relative to the motion start; slope 1 outside the knots.""" | ||
| 11 | + xs, ys = zip(*sorted(knots)) | ||
| 12 | + assert all(b > a for a, b in zip(ys, ys[1:])), "knots must stay in order" | ||
| 13 | + | ||
| 14 | + def f(x): | ||
| 15 | + if x <= xs[0]: | ||
| 16 | + return ys[0] + (x - xs[0]) | ||
| 17 | + if x >= xs[-1]: | ||
| 18 | + return ys[-1] + (x - xs[-1]) | ||
| 19 | + i = bisect_right(xs, x) - 1 | ||
| 20 | + return ys[i] + (x - xs[i]) * (ys[i + 1] - ys[i]) / (xs[i + 1] - xs[i]) | ||
| 21 | + return f | ||
| 22 | + | ||
| 23 | + | ||
| 24 | +def fmt(x): | ||
| 25 | + return f"{round(x, 2):.2f}".rstrip("0").rstrip(".") if abs(x) > 1e-9 else "0" | ||
| 26 | + | ||
| 27 | + | ||
| 28 | +def remap(path: Path, knots, dur, helpers=(), dry=False): | ||
| 29 | + """ | ||
| 30 | + Rewrites in place: the root data-duration / data-hero, the final `tl.to({}, { duration: D }, 0)`, every tween | ||
| 31 | + position (`}, t)`), helper calls matched by `helpers` (regexes whose named groups t1, t2… are instants, e.g. | ||
| 32 | + r'lever\\("#i\\d", (?P<t1>…), (?P<t2>…)\\)'), and the "t" of each cue in <script id="sfx">. | ||
| 33 | + Returns (map, number of instants rewritten). | ||
| 34 | + """ | ||
| 35 | + f = mapper(knots) | ||
| 36 | + name = path.stem | ||
| 37 | + s = path.read_text(encoding="utf-8") | ||
| 38 | + n = 0 | ||
| 39 | + | ||
| 40 | + def sub(pattern, text, fn, flags=0): | ||
| 41 | + nonlocal n | ||
| 42 | + out, k = re.subn(pattern, fn, text, flags=flags) | ||
| 43 | + n += k | ||
| 44 | + return out | ||
| 45 | + | ||
| 46 | + s = sub(rf'(data-composition-id="{name}" data-duration=")({NUM})(")', s, lambda m: m[1] + fmt(dur) + m[3]) | ||
| 47 | + s = sub(rf'(data-hero=")({NUM})(")', s, lambda m: m[1] + fmt(min(f(float(m[2])), dur - 0.2)) + m[3]) | ||
| 48 | + s = sub(rf"(tl\.to\(\{{\}}, \{{ duration: )({NUM})( \}}, 0\))", s, lambda m: m[1] + fmt(dur) + m[3]) | ||
| 49 | + s = sub(rf"(\}}, )({NUM})(\))", s, lambda m: m[1] + fmt(f(float(m[2]))) + m[3]) | ||
| 50 | + for pattern in helpers: | ||
| 51 | + def fix(m): | ||
| 52 | + g = m.group(0) | ||
| 53 | + for key in sorted((k for k in m.re.groupindex if k.startswith("t")), reverse=True): | ||
| 54 | + a, b = m.start(key) - m.start(), m.end(key) - m.start() | ||
| 55 | + g = g[:a] + fmt(f(float(m.group(key)))) + g[b:] | ||
| 56 | + return g | ||
| 57 | + s = sub(pattern, s, fix) | ||
| 58 | + s = sub(r'<script type="application/json" id="sfx">.*?</script>', s, | ||
| 59 | + lambda m: re.sub(rf'("t": )({NUM})', lambda k: k[1] + fmt(f(float(k[2]))), m[0]), flags=re.S) | ||
| 60 | + if not dry: | ||
| 61 | + write_like(path, s) | ||
| 62 | + return f, n | ||
| 63 | + | ||
| 64 | + | ||
| 65 | +def write_like(path: Path, text: str) -> None: | ||
| 66 | + """Writes text keeping the file's own line endings (compositions are CRLF on this machine).""" | ||
| 67 | + data = text.replace("\r\n", "\n").encode("utf-8") | ||
| 68 | + if b"\r\n" in path.read_bytes(): | ||
| 69 | + data = data.replace(b"\n", b"\r\n") | ||
| 70 | + path.write_bytes(data) | ||
added
.claude/skills/replace-aroll-take/scripts/replace.py +129 -0 | new file mode 100644 | ||
| @@ -0,0 +1,129 @@ | ||
| 1 | +""" | |
| 2 | +Replace a stretch of the A-roll with a new take and re-place everything that sits on it. | |
| 3 | + | |
| 4 | + python replace.py <config.py> [--dry] | |
| 5 | + | |
| 6 | +The config is a Python file defining CFG (see SKILL.md). --dry prints the plan (new end, delta, motion spans with | |
| 7 | +overlaps, number of instants retimed per composition) without writing anything. | |
| 8 | +""" | |
| 9 | +import json | |
| 10 | +import sys | |
| 11 | +from pathlib import Path | |
| 12 | + | |
| 13 | +from common import project_dir | |
| 14 | +import remap | |
| 15 | + | |
| 16 | +from studio.modules.projects import montage, storyboard # noqa: E402 (path set by common) | |
| 17 | +from studio.modules.transcript import cloud, cues # noqa: E402 | |
| 18 | + | |
| 19 | +TRACKS = ("zooms", "broll", "motion", "audio", "texts") | |
| 20 | + | |
| 21 | + | |
| 22 | +def run(cfg: dict, dry: bool) -> None: | |
| 23 | + pid = cfg["pid"] | |
| 24 | + tl = montage.load(pid) | |
| 25 | + ids = [c["id"] for c in tl["aroll"]] | |
| 26 | + starts, t = {}, 0.0 | |
| 27 | + for c in tl["aroll"]: | |
| 28 | + starts[c["id"]] = round(t, 3) | |
| 29 | + t += c["out"] - c["in"] | |
| 30 | + first, last = ids.index(cfg["clips"][0]), ids.index(cfg["clips"][-1]) | |
| 31 | + assert ids[first:last + 1] == cfg["clips"], f"clips are not contiguous: {ids[first:last + 1]}" | |
| 32 | + old_a = starts[cfg["clips"][0]] | |
| 33 | + old_b = starts[cfg["clips"][-1]] + tl["aroll"][last]["out"] - tl["aroll"][last]["in"] | |
| 34 | + assert abs(old_a - cfg["old"][0]) < 0.01 and abs(old_b - cfg["old"][1]) < 0.01, \ | |
| 35 | + f"zone moved since the config was written: {old_a}–{old_b}" | |
| 36 | + # inserted clips listed by id (a gag kept in the middle) are reused as they are | |
| 37 | + keep = {c["id"]: c for c in tl["aroll"][first:last + 1]} | |
| 38 | + new_clips = [dict(keep[c]) if isinstance(c, str) else c for c in cfg["aroll"]] | |
| 39 | + new_b = old_a + sum(c["out"] - c["in"] for c in new_clips) | |
| 40 | + delta = round(new_b - old_b, 3) | |
| 41 | + | |
| 42 | + items = {x["id"]: x for tr in TRACKS for x in tl[tr]} | |
| 43 | + old = {i: dict(x) for i, x in items.items()} | |
| 44 | + moved = set() | |
| 45 | + for mid, m in cfg.get("motions", {}).items(): | |
| 46 | + f = remap.mapper(m["knots"]) | |
| 47 | + s_old = old[mid]["at"] | |
| 48 | + items[mid]["at"], items[mid]["dur"] = m["at"], m["dur"] | |
| 49 | + moved.add(mid) | |
| 50 | + for sid in [x["id"] for x in tl["audio"] if x.get("motion") == mid] + m.get("sounds", []): | |
| 51 | + items[sid]["at"] = round(m["at"] + f(old[sid]["at"] - s_old), 3) | |
| 52 | + moved.add(sid) | |
| 53 | + for at, group in cfg.get("groups", []): | |
| 54 | + off = at - old[group[0]]["at"] | |
| 55 | + group = group + [x["id"] for x in tl["audio"] if x.get("motion") in group and x["id"] not in group] | |
| 56 | + for i in group: | |
| 57 | + items[i]["at"] = round(old[i]["at"] + off, 3) | |
| 58 | + moved.add(i) | |
| 59 | + for i, at in cfg.get("fixed", {}).items(): | |
| 60 | + items[i]["at"] = at | |
| 61 | + moved.add(i) | |
| 62 | + for i, dur in cfg.get("durs", {}).items(): | |
| 63 | + items[i]["dur"] = dur | |
| 64 | + drop = set(cfg.get("drop", ())) | |
| 65 | + for tr in TRACKS: | |
| 66 | + tl[tr] = [x for x in tl[tr] if x["id"] not in drop] | |
| 67 | + for x in tl[tr]: | |
| 68 | + if x["id"] in moved: | |
| 69 | + continue | |
| 70 | + o = old[x["id"]]["at"] | |
| 71 | + if o >= old_b - cfg.get("tail", 1.0): | |
| 72 | + x["at"] = round(x["at"] + delta, 3) | |
| 73 | + elif o >= old_a - 1e-3: | |
| 74 | + raise SystemExit(f"element left in the zone without a decision: {x['id']} at {o}") | |
| 75 | + for k in tl["markers"]: | |
| 76 | + if k["t"] >= old_b: | |
| 77 | + k["t"] = round(k["t"] + delta, 3) | |
| 78 | + used = {x["id"] for tr in montage.TRACKS for x in tl[tr]} - set(keep) | |
| 79 | + fresh = iter(f"a{n}" for n in range(1, 1000) if f"a{n}" not in used and f"a{n}" not in keep) | |
| 80 | + tl["aroll"][first:last + 1] = [c if c.get("id") in keep else {"id": next(fresh), "transition": None, **c} | |
| 81 | + for c in new_clips] | |
| 82 | + | |
| 83 | + print(f"zone {old_a:.3f}–{old_b:.3f} → new end {new_b:.3f}, delta {delta:+.3f}") | |
| 84 | + spans = sorted((x["at"], round(x["at"] + x["dur"], 2), x["id"]) for x in tl["motion"] | |
| 85 | + if old_a - 10 < x["at"] < new_b + 10) | |
| 86 | + for a, b in zip(spans, spans[1:] + [None]): | |
| 87 | + print(" ", a, "OVERLAP" if b and b[0] < a[1] - 1e-3 else "") | |
| 88 | + motion_dir = project_dir(pid) / "montage" / "motion" | |
| 89 | + for mid, m in cfg.get("motions", {}).items(): | |
| 90 | + _, n = remap.remap(motion_dir / f"{mid}.html", m["knots"], m["dur"], m.get("helpers", ()), dry=True) | |
| 91 | + print(f" {mid}: {n} instants retimed") | |
| 92 | + if dry: | |
| 93 | + return | |
| 94 | + | |
| 95 | + # history snapshot (timeline + every composition) happens here, before the compositions are touched | |
| 96 | + saved = montage.save(pid, tl, keep_history=True, shifts=[{"at": old_b, "length": delta}]) | |
| 97 | + board = storyboard.load(pid) | |
| 98 | + if board: | |
| 99 | + beat_at = {x["beat"]: x["at"] for tr in ("zooms", "broll", "motion") for x in saved[tr] | |
| 100 | + if x.get("beat") and x["id"] in moved} | |
| 101 | + for b in board["beats"]: | |
| 102 | + if b["id"] in beat_at and old_a <= b.get("at", 0) < old_b: | |
| 103 | + b["at"] = beat_at[b["id"]] | |
| 104 | + storyboard.save(pid, board) | |
| 105 | + for mid, m in cfg.get("motions", {}).items(): | |
| 106 | + remap.remap(motion_dir / f"{mid}.html", m["knots"], m["dur"], m.get("helpers", ())) | |
| 107 | + for name, pairs in cfg.get("edits", {}).items(): | |
| 108 | + p = motion_dir / name | |
| 109 | + s = p.read_text(encoding="utf-8") | |
| 110 | + for a, b in pairs: | |
| 111 | + assert s.count(a) == 1, (name, a) | |
| 112 | + s = s.replace(a, b) | |
| 113 | + remap.write_like(p, s) | |
| 114 | + montage.after_change(pid) # queues the renders, rewrites the FCPXML | |
| 115 | + | |
| 116 | + raw = json.loads(Path(cfg["raw"]).read_text(encoding="utf-8")) | |
| 117 | + segs = cloud.group([(w["start"], w["end"], w["text"]) for w in raw["words"] if w["type"] == "word"]) | |
| 118 | + built = cues.build_cues(segs, "line") | |
| 119 | + stem = Path(cfg["raw"]).name.removesuffix("_raw.json") | |
| 120 | + out = project_dir(pid) / "transcripts" | |
| 121 | + (out / f"{stem}.srt").write_text(cues.to_srt(built), encoding="utf-8") | |
| 122 | + (out / f"{stem}.txt").write_text(cues.to_txt(built, "line"), encoding="utf-8") | |
| 123 | + print("saved") | |
| 124 | + | |
| 125 | + | |
| 126 | +if __name__ == "__main__": | |
| 127 | + scope: dict = {} | |
| 128 | + exec(Path(sys.argv[1]).read_text(encoding="utf-8"), scope) | |
| 129 | + run(scope["CFG"], "--dry" in sys.argv) | |
| new file mode 100644 | |||
| @@ -0,0 +1,129 @@ | |||
| 1 | +""" | ||
| 2 | +Replace a stretch of the A-roll with a new take and re-place everything that sits on it. | ||
| 3 | + | ||
| 4 | + python replace.py <config.py> [--dry] | ||
| 5 | + | ||
| 6 | +The config is a Python file defining CFG (see SKILL.md). --dry prints the plan (new end, delta, motion spans with | ||
| 7 | +overlaps, number of instants retimed per composition) without writing anything. | ||
| 8 | +""" | ||
| 9 | +import json | ||
| 10 | +import sys | ||
| 11 | +from pathlib import Path | ||
| 12 | + | ||
| 13 | +from common import project_dir | ||
| 14 | +import remap | ||
| 15 | + | ||
| 16 | +from studio.modules.projects import montage, storyboard # noqa: E402 (path set by common) | ||
| 17 | +from studio.modules.transcript import cloud, cues # noqa: E402 | ||
| 18 | + | ||
| 19 | +TRACKS = ("zooms", "broll", "motion", "audio", "texts") | ||
| 20 | + | ||
| 21 | + | ||
| 22 | +def run(cfg: dict, dry: bool) -> None: | ||
| 23 | + pid = cfg["pid"] | ||
| 24 | + tl = montage.load(pid) | ||
| 25 | + ids = [c["id"] for c in tl["aroll"]] | ||
| 26 | + starts, t = {}, 0.0 | ||
| 27 | + for c in tl["aroll"]: | ||
| 28 | + starts[c["id"]] = round(t, 3) | ||
| 29 | + t += c["out"] - c["in"] | ||
| 30 | + first, last = ids.index(cfg["clips"][0]), ids.index(cfg["clips"][-1]) | ||
| 31 | + assert ids[first:last + 1] == cfg["clips"], f"clips are not contiguous: {ids[first:last + 1]}" | ||
| 32 | + old_a = starts[cfg["clips"][0]] | ||
| 33 | + old_b = starts[cfg["clips"][-1]] + tl["aroll"][last]["out"] - tl["aroll"][last]["in"] | ||
| 34 | + assert abs(old_a - cfg["old"][0]) < 0.01 and abs(old_b - cfg["old"][1]) < 0.01, \ | ||
| 35 | + f"zone moved since the config was written: {old_a}–{old_b}" | ||
| 36 | + # inserted clips listed by id (a gag kept in the middle) are reused as they are | ||
| 37 | + keep = {c["id"]: c for c in tl["aroll"][first:last + 1]} | ||
| 38 | + new_clips = [dict(keep[c]) if isinstance(c, str) else c for c in cfg["aroll"]] | ||
| 39 | + new_b = old_a + sum(c["out"] - c["in"] for c in new_clips) | ||
| 40 | + delta = round(new_b - old_b, 3) | ||
| 41 | + | ||
| 42 | + items = {x["id"]: x for tr in TRACKS for x in tl[tr]} | ||
| 43 | + old = {i: dict(x) for i, x in items.items()} | ||
| 44 | + moved = set() | ||
| 45 | + for mid, m in cfg.get("motions", {}).items(): | ||
| 46 | + f = remap.mapper(m["knots"]) | ||
| 47 | + s_old = old[mid]["at"] | ||
| 48 | + items[mid]["at"], items[mid]["dur"] = m["at"], m["dur"] | ||
| 49 | + moved.add(mid) | ||
| 50 | + for sid in [x["id"] for x in tl["audio"] if x.get("motion") == mid] + m.get("sounds", []): | ||
| 51 | + items[sid]["at"] = round(m["at"] + f(old[sid]["at"] - s_old), 3) | ||
| 52 | + moved.add(sid) | ||
| 53 | + for at, group in cfg.get("groups", []): | ||
| 54 | + off = at - old[group[0]]["at"] | ||
| 55 | + group = group + [x["id"] for x in tl["audio"] if x.get("motion") in group and x["id"] not in group] | ||
| 56 | + for i in group: | ||
| 57 | + items[i]["at"] = round(old[i]["at"] + off, 3) | ||
| 58 | + moved.add(i) | ||
| 59 | + for i, at in cfg.get("fixed", {}).items(): | ||
| 60 | + items[i]["at"] = at | ||
| 61 | + moved.add(i) | ||
| 62 | + for i, dur in cfg.get("durs", {}).items(): | ||
| 63 | + items[i]["dur"] = dur | ||
| 64 | + drop = set(cfg.get("drop", ())) | ||
| 65 | + for tr in TRACKS: | ||
| 66 | + tl[tr] = [x for x in tl[tr] if x["id"] not in drop] | ||
| 67 | + for x in tl[tr]: | ||
| 68 | + if x["id"] in moved: | ||
| 69 | + continue | ||
| 70 | + o = old[x["id"]]["at"] | ||
| 71 | + if o >= old_b - cfg.get("tail", 1.0): | ||
| 72 | + x["at"] = round(x["at"] + delta, 3) | ||
| 73 | + elif o >= old_a - 1e-3: | ||
| 74 | + raise SystemExit(f"element left in the zone without a decision: {x['id']} at {o}") | ||
| 75 | + for k in tl["markers"]: | ||
| 76 | + if k["t"] >= old_b: | ||
| 77 | + k["t"] = round(k["t"] + delta, 3) | ||
| 78 | + used = {x["id"] for tr in montage.TRACKS for x in tl[tr]} - set(keep) | ||
| 79 | + fresh = iter(f"a{n}" for n in range(1, 1000) if f"a{n}" not in used and f"a{n}" not in keep) | ||
| 80 | + tl["aroll"][first:last + 1] = [c if c.get("id") in keep else {"id": next(fresh), "transition": None, **c} | ||
| 81 | + for c in new_clips] | ||
| 82 | + | ||
| 83 | + print(f"zone {old_a:.3f}–{old_b:.3f} → new end {new_b:.3f}, delta {delta:+.3f}") | ||
| 84 | + spans = sorted((x["at"], round(x["at"] + x["dur"], 2), x["id"]) for x in tl["motion"] | ||
| 85 | + if old_a - 10 < x["at"] < new_b + 10) | ||
| 86 | + for a, b in zip(spans, spans[1:] + [None]): | ||
| 87 | + print(" ", a, "OVERLAP" if b and b[0] < a[1] - 1e-3 else "") | ||
| 88 | + motion_dir = project_dir(pid) / "montage" / "motion" | ||
| 89 | + for mid, m in cfg.get("motions", {}).items(): | ||
| 90 | + _, n = remap.remap(motion_dir / f"{mid}.html", m["knots"], m["dur"], m.get("helpers", ()), dry=True) | ||
| 91 | + print(f" {mid}: {n} instants retimed") | ||
| 92 | + if dry: | ||
| 93 | + return | ||
| 94 | + | ||
| 95 | + # history snapshot (timeline + every composition) happens here, before the compositions are touched | ||
| 96 | + saved = montage.save(pid, tl, keep_history=True, shifts=[{"at": old_b, "length": delta}]) | ||
| 97 | + board = storyboard.load(pid) | ||
| 98 | + if board: | ||
| 99 | + beat_at = {x["beat"]: x["at"] for tr in ("zooms", "broll", "motion") for x in saved[tr] | ||
| 100 | + if x.get("beat") and x["id"] in moved} | ||
| 101 | + for b in board["beats"]: | ||
| 102 | + if b["id"] in beat_at and old_a <= b.get("at", 0) < old_b: | ||
| 103 | + b["at"] = beat_at[b["id"]] | ||
| 104 | + storyboard.save(pid, board) | ||
| 105 | + for mid, m in cfg.get("motions", {}).items(): | ||
| 106 | + remap.remap(motion_dir / f"{mid}.html", m["knots"], m["dur"], m.get("helpers", ())) | ||
| 107 | + for name, pairs in cfg.get("edits", {}).items(): | ||
| 108 | + p = motion_dir / name | ||
| 109 | + s = p.read_text(encoding="utf-8") | ||
| 110 | + for a, b in pairs: | ||
| 111 | + assert s.count(a) == 1, (name, a) | ||
| 112 | + s = s.replace(a, b) | ||
| 113 | + remap.write_like(p, s) | ||
| 114 | + montage.after_change(pid) # queues the renders, rewrites the FCPXML | ||
| 115 | + | ||
| 116 | + raw = json.loads(Path(cfg["raw"]).read_text(encoding="utf-8")) | ||
| 117 | + segs = cloud.group([(w["start"], w["end"], w["text"]) for w in raw["words"] if w["type"] == "word"]) | ||
| 118 | + built = cues.build_cues(segs, "line") | ||
| 119 | + stem = Path(cfg["raw"]).name.removesuffix("_raw.json") | ||
| 120 | + out = project_dir(pid) / "transcripts" | ||
| 121 | + (out / f"{stem}.srt").write_text(cues.to_srt(built), encoding="utf-8") | ||
| 122 | + (out / f"{stem}.txt").write_text(cues.to_txt(built, "line"), encoding="utf-8") | ||
| 123 | + print("saved") | ||
| 124 | + | ||
| 125 | + | ||
| 126 | +if __name__ == "__main__": | ||
| 127 | + scope: dict = {} | ||
| 128 | + exec(Path(sys.argv[1]).read_text(encoding="utf-8"), scope) | ||
| 129 | + run(scope["CFG"], "--dry" in sys.argv) | ||
added
.claude/skills/replace-aroll-take/scripts/transcribe.py +55 -0 | new file mode 100644 | ||
| @@ -0,0 +1,55 @@ | ||
| 1 | +""" | |
| 2 | +Transcribe new takes with ElevenLabs Scribe (word timings), in parallel. | |
| 3 | + | |
| 4 | + python transcribe.py <pid> <file> [<file> ...] # files relative to the project (ressources/x.mp4) | |
| 5 | + | |
| 6 | +Writes <work>/<stem>_raw.json and <stem>.txt (one line per phrase, times in the file) and prints the text. | |
| 7 | +""" | |
| 8 | +import json | |
| 9 | +import subprocess | |
| 10 | +import sys | |
| 11 | +from concurrent.futures import ThreadPoolExecutor | |
| 12 | +from pathlib import Path | |
| 13 | + | |
| 14 | +import httpx | |
| 15 | + | |
| 16 | +from common import DATA, project_dir, work_dir | |
| 17 | + | |
| 18 | +URL = "https://api.elevenlabs.io/v1/speech-to-text" | |
| 19 | + | |
| 20 | + | |
| 21 | +def run(pid: str, rel: str, key: str) -> Path: | |
| 22 | + src = project_dir(pid) / rel | |
| 23 | + out = work_dir(pid) | |
| 24 | + mp3 = out / f"{src.stem}.mp3" | |
| 25 | + subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", str(src), "-vn", "-ac", "1", "-b:a", "96k", str(mp3)], check=True) | |
| 26 | + with mp3.open("rb") as f: | |
| 27 | + r = httpx.post(URL, headers={"xi-api-key": key}, timeout=900, | |
| 28 | + data={"model_id": "scribe_v2", "timestamps_granularity": "word", "tag_audio_events": "false", | |
| 29 | + "diarize": "false", "language_code": "fr"}, | |
| 30 | + files={"file": (mp3.name, f, "audio/mpeg")}) | |
| 31 | + r.raise_for_status() | |
| 32 | + mp3.unlink(missing_ok=True) | |
| 33 | + raw = r.json() | |
| 34 | + (out / f"{src.stem}_raw.json").write_text(json.dumps(raw, ensure_ascii=False), encoding="utf-8") | |
| 35 | + words = [w for w in raw["words"] if w["type"] == "word"] | |
| 36 | + lines, line = [], [] | |
| 37 | + for w in words: | |
| 38 | + if line and (w["start"] - line[-1]["end"] > 0.5 or line[-1]["text"][-1:] in ".?!"): | |
| 39 | + lines.append(line) | |
| 40 | + line = [] | |
| 41 | + line.append(w) | |
| 42 | + if line: | |
| 43 | + lines.append(line) | |
| 44 | + text = "\n".join(f"{l[0]['start']:.2f}-{l[-1]['end']:.2f} {' '.join(x['text'] for x in l)}" for l in lines) | |
| 45 | + (out / f"{src.stem}.txt").write_text(text, encoding="utf-8") | |
| 46 | + return out / f"{src.stem}.txt" | |
| 47 | + | |
| 48 | + | |
| 49 | +if __name__ == "__main__": | |
| 50 | + pid, files = sys.argv[1], sys.argv[2:] | |
| 51 | + key = json.loads((DATA / "transcription.json").read_text(encoding="utf-8"))["keys"]["elevenlabs"] | |
| 52 | + with ThreadPoolExecutor(len(files)) as ex: | |
| 53 | + for txt in ex.map(lambda f: run(pid, f, key), files): | |
| 54 | + print(f"===== {txt}") | |
| 55 | + print(txt.read_text(encoding="utf-8")) | |
| new file mode 100644 | |||
| @@ -0,0 +1,55 @@ | |||
| 1 | +""" | ||
| 2 | +Transcribe new takes with ElevenLabs Scribe (word timings), in parallel. | ||
| 3 | + | ||
| 4 | + python transcribe.py <pid> <file> [<file> ...] # files relative to the project (ressources/x.mp4) | ||
| 5 | + | ||
| 6 | +Writes <work>/<stem>_raw.json and <stem>.txt (one line per phrase, times in the file) and prints the text. | ||
| 7 | +""" | ||
| 8 | +import json | ||
| 9 | +import subprocess | ||
| 10 | +import sys | ||
| 11 | +from concurrent.futures import ThreadPoolExecutor | ||
| 12 | +from pathlib import Path | ||
| 13 | + | ||
| 14 | +import httpx | ||
| 15 | + | ||
| 16 | +from common import DATA, project_dir, work_dir | ||
| 17 | + | ||
| 18 | +URL = "https://api.elevenlabs.io/v1/speech-to-text" | ||
| 19 | + | ||
| 20 | + | ||
| 21 | +def run(pid: str, rel: str, key: str) -> Path: | ||
| 22 | + src = project_dir(pid) / rel | ||
| 23 | + out = work_dir(pid) | ||
| 24 | + mp3 = out / f"{src.stem}.mp3" | ||
| 25 | + subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", str(src), "-vn", "-ac", "1", "-b:a", "96k", str(mp3)], check=True) | ||
| 26 | + with mp3.open("rb") as f: | ||
| 27 | + r = httpx.post(URL, headers={"xi-api-key": key}, timeout=900, | ||
| 28 | + data={"model_id": "scribe_v2", "timestamps_granularity": "word", "tag_audio_events": "false", | ||
| 29 | + "diarize": "false", "language_code": "fr"}, | ||
| 30 | + files={"file": (mp3.name, f, "audio/mpeg")}) | ||
| 31 | + r.raise_for_status() | ||
| 32 | + mp3.unlink(missing_ok=True) | ||
| 33 | + raw = r.json() | ||
| 34 | + (out / f"{src.stem}_raw.json").write_text(json.dumps(raw, ensure_ascii=False), encoding="utf-8") | ||
| 35 | + words = [w for w in raw["words"] if w["type"] == "word"] | ||
| 36 | + lines, line = [], [] | ||
| 37 | + for w in words: | ||
| 38 | + if line and (w["start"] - line[-1]["end"] > 0.5 or line[-1]["text"][-1:] in ".?!"): | ||
| 39 | + lines.append(line) | ||
| 40 | + line = [] | ||
| 41 | + line.append(w) | ||
| 42 | + if line: | ||
| 43 | + lines.append(line) | ||
| 44 | + text = "\n".join(f"{l[0]['start']:.2f}-{l[-1]['end']:.2f} {' '.join(x['text'] for x in l)}" for l in lines) | ||
| 45 | + (out / f"{src.stem}.txt").write_text(text, encoding="utf-8") | ||
| 46 | + return out / f"{src.stem}.txt" | ||
| 47 | + | ||
| 48 | + | ||
| 49 | +if __name__ == "__main__": | ||
| 50 | + pid, files = sys.argv[1], sys.argv[2:] | ||
| 51 | + key = json.loads((DATA / "transcription.json").read_text(encoding="utf-8"))["keys"]["elevenlabs"] | ||
| 52 | + with ThreadPoolExecutor(len(files)) as ex: | ||
| 53 | + for txt in ex.map(lambda f: run(pid, f, key), files): | ||
| 54 | + print(f"===== {txt}") | ||
| 55 | + print(txt.read_text(encoding="utf-8")) | ||
added
.claude/skills/replace-aroll-take/scripts/zone.py +58 -0 | new file mode 100644 | ||
| @@ -0,0 +1,58 @@ | ||
| 1 | +""" | |
| 2 | +Look at the timeline before replacing a zone. | |
| 3 | + | |
| 4 | + python zone.py <pid> clips # A-roll clips with their timeline start/end | |
| 5 | + python zone.py <pid> old <a> <b> # old spoken text + every element placed on [a, b] | |
| 6 | + python zone.py <pid> new <raw.json> <at> [<in>] # new take's words in timeline time, if it starts at <at> | |
| 7 | +""" | |
| 8 | +import json | |
| 9 | +import sys | |
| 10 | + | |
| 11 | +from common import clip_starts, source_words_on_timeline, timeline | |
| 12 | + | |
| 13 | + | |
| 14 | +def clips(pid): | |
| 15 | + tl = timeline(pid) | |
| 16 | + for c, s in zip(tl["aroll"], clip_starts(tl)): | |
| 17 | + print(f"{s:9.3f} {s + c['out'] - c['in']:9.3f} {c['id']:5} {c.get('src', 'source')[-50:]:50} " | |
| 18 | + f"in={c['in']} out={c['out']} tr={c.get('transition')} gain={c.get('gain_db')}") | |
| 19 | + | |
| 20 | + | |
| 21 | +def old(pid, a, b): | |
| 22 | + tl = timeline(pid) | |
| 23 | + print("=== old words (timeline time)") | |
| 24 | + words = source_words_on_timeline(pid, tl, a, b) | |
| 25 | + line = [] | |
| 26 | + for t, w in words: | |
| 27 | + line.append(f"{t:.2f}:{w}") | |
| 28 | + if w[-1:] in ".?!": | |
| 29 | + print(" ".join(line)) | |
| 30 | + line = [] | |
| 31 | + print(" ".join(line)) | |
| 32 | + print("=== elements touching the zone") | |
| 33 | + for tr in ("zooms", "broll", "motion", "audio", "texts", "markers"): | |
| 34 | + for x in tl[tr]: | |
| 35 | + t = x.get("at", x.get("t")) | |
| 36 | + end = t + (x["dur"] if "dur" in x else (x["out"] - x["in"] if "out" in x else 0)) | |
| 37 | + if end > a - 2 and t < b + 3: | |
| 38 | + if tr == "motion": | |
| 39 | + info = f"dur={x['dur']} beat={x.get('beat')} {x.get('idea', '')[:240]!r}" | |
| 40 | + else: | |
| 41 | + info = json.dumps({k: v for k, v in x.items() if k not in ("id", "at", "in", "role")}, ensure_ascii=False)[:160] | |
| 42 | + print(f"{tr:6} {x['id']:8} {t:9.2f} {end:9.2f} {info}") | |
| 43 | + | |
| 44 | + | |
| 45 | +def new(raw_path, at, start_in=0.0): | |
| 46 | + raw = json.loads(open(raw_path, encoding="utf-8").read()) | |
| 47 | + print(" ".join(f"{at + w['start'] - start_in:.2f}:{w['text']}" for w in raw["words"] | |
| 48 | + if w["type"] == "word" and w["start"] >= start_in)) | |
| 49 | + | |
| 50 | + | |
| 51 | +if __name__ == "__main__": | |
| 52 | + pid, cmd, *args = sys.argv[1:] | |
| 53 | + if cmd == "clips": | |
| 54 | + clips(pid) | |
| 55 | + elif cmd == "old": | |
| 56 | + old(pid, float(args[0]), float(args[1])) | |
| 57 | + else: | |
| 58 | + new(args[0], float(args[1]), float(args[2]) if len(args) > 2 else 0.0) | |
| new file mode 100644 | |||
| @@ -0,0 +1,58 @@ | |||
| 1 | +""" | ||
| 2 | +Look at the timeline before replacing a zone. | ||
| 3 | + | ||
| 4 | + python zone.py <pid> clips # A-roll clips with their timeline start/end | ||
| 5 | + python zone.py <pid> old <a> <b> # old spoken text + every element placed on [a, b] | ||
| 6 | + python zone.py <pid> new <raw.json> <at> [<in>] # new take's words in timeline time, if it starts at <at> | ||
| 7 | +""" | ||
| 8 | +import json | ||
| 9 | +import sys | ||
| 10 | + | ||
| 11 | +from common import clip_starts, source_words_on_timeline, timeline | ||
| 12 | + | ||
| 13 | + | ||
| 14 | +def clips(pid): | ||
| 15 | + tl = timeline(pid) | ||
| 16 | + for c, s in zip(tl["aroll"], clip_starts(tl)): | ||
| 17 | + print(f"{s:9.3f} {s + c['out'] - c['in']:9.3f} {c['id']:5} {c.get('src', 'source')[-50:]:50} " | ||
| 18 | + f"in={c['in']} out={c['out']} tr={c.get('transition')} gain={c.get('gain_db')}") | ||
| 19 | + | ||
| 20 | + | ||
| 21 | +def old(pid, a, b): | ||
| 22 | + tl = timeline(pid) | ||
| 23 | + print("=== old words (timeline time)") | ||
| 24 | + words = source_words_on_timeline(pid, tl, a, b) | ||
| 25 | + line = [] | ||
| 26 | + for t, w in words: | ||
| 27 | + line.append(f"{t:.2f}:{w}") | ||
| 28 | + if w[-1:] in ".?!": | ||
| 29 | + print(" ".join(line)) | ||
| 30 | + line = [] | ||
| 31 | + print(" ".join(line)) | ||
| 32 | + print("=== elements touching the zone") | ||
| 33 | + for tr in ("zooms", "broll", "motion", "audio", "texts", "markers"): | ||
| 34 | + for x in tl[tr]: | ||
| 35 | + t = x.get("at", x.get("t")) | ||
| 36 | + end = t + (x["dur"] if "dur" in x else (x["out"] - x["in"] if "out" in x else 0)) | ||
| 37 | + if end > a - 2 and t < b + 3: | ||
| 38 | + if tr == "motion": | ||
| 39 | + info = f"dur={x['dur']} beat={x.get('beat')} {x.get('idea', '')[:240]!r}" | ||
| 40 | + else: | ||
| 41 | + info = json.dumps({k: v for k, v in x.items() if k not in ("id", "at", "in", "role")}, ensure_ascii=False)[:160] | ||
| 42 | + print(f"{tr:6} {x['id']:8} {t:9.2f} {end:9.2f} {info}") | ||
| 43 | + | ||
| 44 | + | ||
| 45 | +def new(raw_path, at, start_in=0.0): | ||
| 46 | + raw = json.loads(open(raw_path, encoding="utf-8").read()) | ||
| 47 | + print(" ".join(f"{at + w['start'] - start_in:.2f}:{w['text']}" for w in raw["words"] | ||
| 48 | + if w["type"] == "word" and w["start"] >= start_in)) | ||
| 49 | + | ||
| 50 | + | ||
| 51 | +if __name__ == "__main__": | ||
| 52 | + pid, cmd, *args = sys.argv[1:] | ||
| 53 | + if cmd == "clips": | ||
| 54 | + clips(pid) | ||
| 55 | + elif cmd == "old": | ||
| 56 | + old(pid, float(args[0]), float(args[1])) | ||
| 57 | + else: | ||
| 58 | + new(args[0], float(args[1]), float(args[2]) if len(args) > 2 else 0.0) | ||