发表机构
HKUST; Adobe Research; HKUST(GZ); Nanyang Technological University(香港科技大学; Adobe 研究部; 港中大(深圳); 南洋理工大学)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
CraftTrace通过将视频转换为可塑的多层次结构,利用AI代理传播编辑,帮助用户理解视频并制定编辑意图,支持快速原型制作和后期制作。
AI 中文摘要
近期的生成式视频编辑模型支持视频内容修改(例如,更改角色),但主要针对短视频片段。将这些模型扩展到完整的多镜头视频需要繁琐的工作,包括在镜头间定位相关内容、将其分割为片段、为每个片段精心制作上下文感知的编辑提示,并反复阐述复杂的编辑意图。为解决此问题,我们探索了一种通过底层视频结构(如脚本、场景、角色、镜头及其关系)进行编辑的交互范式。我们提出了CraftTrace,一个交互式原型,将视频转换为可塑的多层次结构以支持生成式编辑。用户在任务中心的工作空间中修改元素或重塑关系,而AI代理则负责在视频中转换和传播更改。用户研究和专家评审表明,这种结构有助于用户理解视频、制定和细化编辑意图,并探索替代方案,支持早期探索阶段的快速原型制作和完整的视频后期制作。
英文摘要
Recent generative video editing models enable video content modification (e.g., changing a character) but target short clips. Extending them to full multi-shot videos requires tedious work to locate relevant content across shots, segment it into clips, craft context-aware editing prompts for each clip, and repeatedly articulate complex editing intent. To address this, we explore an interaction paradigm for editing through underlying video structures (e.g., scripts, scenes, characters, shots, and their relationships). We present CraftTrace, an interactive prototype that transforms a video into a malleable, multilevel structure for generative editing. Users work in task-centric workspaces to modify elements or reshape relationships, while an AI agent translates and propagates changes across the video. A user study and expert review show that this structure helps users understand videos, formulate and refine editing intent, and explore alternatives, supporting rapid prototyping during early-stage exploration and full video post-production.