arXiv 2512.128752025-12-14cs.CVcs.MMcs.SD

Schrodinger Audio-Visual Editor: 对象级音频视觉去除

Schrodinger Audio-Visual Editor: Object-Level Audiovisual Removal

MIT（麻省理工学院）；UC Berkeley（加州大学伯克利分校）；Berkeley AI Research（伯克利人工智能研究）；Sony AI（索尼人工智能）；Sony Group Corporation（索尼集团）

Weihan Xu, Kan Jen Cheng, Koichi Saito, Muhammad Jehanzeb Mirza, Tingle Li, Yisi Liu, Alexander H. Liu, Liming Wang, Masato Ishii, Takashi Shibuya, Yuki Mitsufu… 展开作者

Weihan Xu, Kan Jen Cheng, Koichi Saito, Muhammad Jehanzeb Mirza, Tingle Li, Yisi Liu, Alexander H. Liu, Liming Wang, Masato Ishii, Takashi Shibuya, Yuki Mitsufuji, Gopala Anumanchipalli, Paul Pu Liang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Schrodinger Audio-Visual Editor，通过联合编辑音频和视频实现对象级去除，提升时间同步性和语义对应性。

摘要：

音频和视觉内容的联合编辑对于精确且可控的内容创作至关重要。这项新任务面临挑战，因为目标编辑前后配对的音频视觉数据存在限制，且模态间存在异质性。为了解决联合音频视觉编辑中的数据和建模挑战，我们引入了SAVEBench，一个带有文本和掩码条件的配对音频视觉数据集，以实现基于对象的源到目标学习。借助SAVEBench，我们训练了Schrodinger Audio-Visual Editor（SAVE），一个端到端的流匹配模型，能够在并行编辑音频和视频的同时保持它们在整个处理过程中的对齐。SAVE集成了一个Schrodinger Bridge，学习从源到目标音频视觉混合的直接传输。我们的评估显示，所提出的SAVE模型能够去除音频和视觉内容中的目标对象，同时保留剩余内容，相比音频编辑器和视频编辑器的配对组合，具有更强的时间同步性和音频视觉语义对应性。

英文摘要：

Joint editing of audio and visual content is crucial for precise and controllable content creation. This new task poses challenges due to the limitations of paired audio-visual data before and after targeted edits, and the heterogeneity across modalities. To address the data and modeling challenges in joint audio-visual editing, we introduce SAVEBench, a paired audiovisual dataset with text and mask conditions to enable object-grounded source-to-target learning. With SAVEBench, we train the Schrodinger Audio-Visual Editor (SAVE), an end-to-end flow-matching model that edits audio and video in parallel while keeping them aligned throughout processing. SAVE incorporates a Schrodinger Bridge that learns a direct transport from source to target audiovisual mixtures. Our evaluation demonstrates that the proposed SAVE model is able to remove the target objects in audio and visual content while preserving the remaining content, with stronger temporal synchronization and audiovisual semantic correspondence compared with pairwise combinations of an audio editor and a video editor.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。