arXiv 2412.128772025-03-25cs.CV

PRIMEdit：面向实例感知多对象视频编辑的概率重分配与基准数据集

PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Dataset

KAIST（韩国科学技术院）；Adobe Research（奥多比研究院）；Chung-Ang University（中央大学）

Samuel Teodoro, Agus Gunawan, Soo Ye Kim, Jihyong Oh, Munchurl Kim

备注：: The first two authors contributed equally to this work. The last two authors are co-corresponding authors. Please visit our project page at https://kaist-viclab.github.io/primedit-site/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出零样本框架PRIMEdit，通过实例中心概率重分配和分离多实例采样，解决多对象视频编辑中的不忠实与泄漏问题，并引入MIVE数据集和CIA指标，显著优于现有方法。

摘要：

近年来，基于人工智能的视频编辑技术使用户能够通过简单的文本提示编辑视频，大大简化了编辑过程。然而，最近的零样本视频编辑技术主要侧重于全局或单对象编辑，这可能导致视频其他部分发生意外变化。当需要对多个对象进行局部编辑时，现有方法面临诸多挑战，如编辑不忠实、编辑泄漏以及缺乏合适的评估数据集和指标。为克服这些限制，我们提出了面向实例感知多对象视频编辑的概率重分配方法（PRIMEdit）。PRIMEdit是一个零样本框架，引入了两个关键模块：（i）实例中心概率重分配（IPR），以确保精确定位和忠实编辑；（ii）解缠的多实例采样（DMS），以防止编辑泄漏。此外，我们提出了新的MIVE数据集，涵盖多样化的视频场景，并引入了跨实例准确率（CIA）分数，以评估多实例视频编辑任务中的编辑泄漏。我们大量的定性、定量和用户研究评估表明，PRIMEdit在编辑忠实度、准确性和泄漏预防方面显著优于最新的最先进方法，为多实例视频编辑树立了新的基准。

英文摘要：

Recent AI-based video editing has enabled users to edit videos through simple text prompts, significantly simplifying the editing process. However, recent zero-shot video editing techniques primarily focus on global or single-object edits, which can lead to unintended changes in other parts of the video. When multiple objects require localized edits, existing methods face challenges, such as unfaithful editing, editing leakage, and lack of suitable evaluation datasets and metrics. To overcome these limitations, we propose $\textbf{P}$robability $\textbf{R}$edistribution for $\textbf{I}$nstance-aware $\textbf{M}$ulti-object Video $\textbf{Edit}$ing ($\textbf{PRIMEdit}$). PRIMEdit is a zero-shot framework that introduces two key modules: (i) Instance-centric Probability Redistribution (IPR) to ensure precise localization and faithful editing and (ii) Disentangled Multi-instance Sampling (DMS) to prevent editing leakage. Additionally, we present our new MIVE Dataset for video editing featuring diverse video scenarios, and introduce the Cross-Instance Accuracy (CIA) Score to evaluate editing leakage in multi-instance video editing tasks. Our extensive qualitative, quantitative, and user study evaluations demonstrate that PRIMEdit significantly outperforms recent state-of-the-art methods in terms of editing faithfulness, accuracy, and leakage prevention, setting a new benchmark for multi-instance video editing.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。