Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/desktop-gpui/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2074,7 +2074,7 @@ selected", with a "Select all") because the source's does.
| **zoom** | Amount slider, Auto/Manual mode tabs, the "How does it work?" reveal, and — in Manual — a `PositionPad` writing `mode.manual.{x,y}` | `ZoomMultiSegmentConfig`: shared Amount and Mode with a **Mixed** badge when the selection disagrees, one shared pad, and a removable row per segment |
| **text** | multi-line content box, Enabled, Layout, **Templates** (8 preset cards), Font family / weight / italic / size, alignment, line height, letter spacing, colour, opacity, shadow, position, and in/out animations with durations | one card per segment |
| **caption** | text box and Timing (Start / End `NumberField`s with a live duration) | one card per segment |
| **mask** | Sensitive / Highlight kind, Enabled, then either Effect (Blur / Pixelate) + amount, or Outside Darkness + Fade Duration | one card per segment |
| **mask** | Sensitive / Highlight kind, Enabled, then either Effect (Blur / Pixelate) + amount, or Outside Darkness + Fade Duration, then Tracking: Show tracking, Add / Remove Marker at the playhead, Clear, Track Motion and Track Points. Click four or more points on the target (a rectangle's corners, or points around a rounded or irregular shape), drag their numbered handles to adjust, then start tracking; Cancel and Undo remain available during placement. The mask is the box around the points, fitted per axis each frame so it can widen without growing taller, with sides that start on a straight even-coloured edge (a window border, a card's fill) kept on that edge. Show tracking hides masks and guides only in the editor preview, without changing export. Stop preserves partial tracking. Timing dots live on the timeline, not in the panel. | one card per segment |
| **scene** | Camera Layout (5 modes, two disabled without a camera), a description card, Transition In/Out, and — for Split Screen and Floating — Screen/Camera Zoom and two `PositionPad`s | header **alone**, which is the source's own unfinished state (`:1667-1678` renders the body only for a single selection) and is reproduced rather than invented over |
| **3D** | **Templates** (3 scenes, 5 angle presets, 8 motion templates), the Start/End pose cards with swap and flips, Camera (9 sliders), Blur (mode, bokeh, per-mode sliders), Advanced (transitions, motion style) | header alone, same as the source |
| **audio** | track card, name box, Volume, Fade In, Fade Out | one card per segment |
Expand Down
75 changes: 75 additions & 0 deletions apps/desktop-gpui/kernels/examples/mask_track_replay.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
//! Replays mask tracking over a real clip, without the editor, to see how
//! the trackers do on recorded footage.
//!
//! ffmpeg -i clip.mp4 -vf scale=960:540 -f rawvideo -pix_fmt rgba frames.rgba
//! cargo run --release -p cap-gpui-kernels --example mask_track_replay -- \
//! frames.rgba 960x540 30 "x,y x,y x,y x,y"
//!
//! Frames dumped by the editor (`CAP_GPUI_MASK_TRACK_DUMP`) are at the
//! tracking resolution fitted to the recording's aspect ratio, which the
//! editor logs when tracking starts.
//!
//! Points are normalized to the frame (0..1), taken on the first frame. Four
//! or more track points (covering both estimates where they disagree, as a
//! sensitive mask does); two (centre, size) track the whole box instead.
//! Prints the tracked box per frame as `time left top right bottom` in
//! pixels, then the spans in which the target was lost.

use cap_gpui_kernels::motion_track::{LumaFrame, TrackSession};
use cap_gpui_kernels::point_track::{Disagreement, PointTrackSession};

fn main() {
let args = std::env::args().collect::<Vec<_>>();
let [_, frames, size, fps, points] = &args[..] else {
eprintln!("usage: mask_track_replay <frames.rgba> <width>x<height> <fps> \"x,y x,y ...\"");
std::process::exit(2);
};
let (width, height) = size.split_once('x').expect("<width>x<height>");
let (width, height): (u32, u32) = (
width.parse().expect("width"),
height.parse().expect("height"),
);
let fps: f64 = fps.parse().expect("fps");
let points = points
.split_whitespace()
.map(|pair| {
let (x, y) = pair.split_once(',').expect("x,y");
(x.parse::<f64>().expect("x"), y.parse::<f64>().expect("y"))
})
.collect::<Vec<_>>();
let bytes = std::fs::read(frames).expect("frames");
let frame_len = (width * height * 4) as usize;
let mut frames = bytes
.chunks_exact(frame_len)
.map(|frame| LumaFrame::from_rgba(frame, width, height, (width * 4) as usize));
let first = frames.next().expect("at least one frame");
let (path, lost) = if points.len() >= 4 {
let mut session = PointTrackSession::new(&first, 0.0, &points, Disagreement::CoverBoth)
.expect("trackable points");
for (index, frame) in frames.enumerate() {
session.push((index + 1) as f64 / fps, frame);
}
session.finish()
} else {
let [center, size] = points[..] else {
panic!("give four or more points, or a centre and a size");
};
let mut session = TrackSession::new(&first, 0.0, center, size).expect("trackable box");
for (index, frame) in frames.enumerate() {
session.push((index + 1) as f64 / fps, frame);
}
session.finish()
};
for point in path {
let (w, h) = (f64::from(width), f64::from(height));
println!(
"{:.3} {:.1} {:.1} {:.1} {:.1}",
point.time,
(point.center.0 - point.size.0 / 2.0) * w,
(point.center.1 - point.size.1 / 2.0) * h,
(point.center.0 + point.size.0 / 2.0) * w,
(point.center.1 + point.size.1 / 2.0) * h,
);
}
eprintln!("lost: {lost:?}");
}
Loading