arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.14534cs.CVcs.AIcs.MM
收藏

PROVE:一种用于视觉媒体的感知移除一致性基准

PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media

MiLM Plus, Xiaomi Inc.(小米公司MiLM Plus)

Fuhao Li, Shaofeng You, Jiagao Hu, Yu Liu, Yuxuan Chen, Zepeng Wang, Fei Wang, Daiguo Zhou, Jian Luan

AI总结:

本文提出RC指标和PROVE-Bench基准,通过滑动窗口特征比较和分布跟踪提升图像和视频移除任务的评估精度,实现更符合人类感知的一致性衡量。

摘要:

评估图像和视频中的物体移除任务仍具挑战性,因为该任务本质上是单到多的,而现有指标常与人类感知相悖。全参考指标奖励复制粘贴行为而非真实移除;无参考指标受系统偏差影响,如偏好模糊结果;全局时间指标对编辑区域内的局部瑕疵不敏感。为解决这些限制,我们提出了RC(移除一致性),由RC-S和RC-T两指标组成:RC-S通过滑动窗口比较遮蔽区域与背景区域的特征测量空间一致性,RC-T通过在相邻帧共享恢复区域内的分布跟踪测量时间一致性。为验证RC并支持社区基准测试,我们进一步引入PROVE-Bench,一个包含PROVE-M(80个视频配对数据集含运动增强)和PROVE-H(100个视频挑战子集无真实标注)的双层现实基准。RC指标与PROVE-Bench共同构成PROVE(感知移除一致性)评估框架。在多样化的图像和视频基准上的实验表明,RC在人类判断上的对齐性显著优于现有评估协议。RC指标和PROVE-Bench的代码已公开在:https://github.com/xiaomi-research/prove/.

英文摘要:

Evaluating object removal in images and videos remains challenging because the task is inherently one-to-many, yet existing metrics frequently disagree with human perception. Full-reference metrics reward copy-paste behaviors over genuine erasure; no-reference metrics suffer from systematic biases such as favoring blurry results; and global temporal metrics are insensitive to localized artifacts within edited regions. To address these limitations, we propose RC (Removal Coherence), a pair of perception-aligned metrics: RC-S, which measures spatial coherence via sliding-window feature comparison between masked and background regions, and RC-T, which measures temporal consistency via distribution tracking within shared restored regions across adjacent frames. To validate RC and support community benchmarking, we further introduce PROVE-Bench, a two-tier real-world benchmark comprising PROVE-M, an 80-video paired dataset with motion augmentation, and PROVE-H, a 100-video challenging subset without ground truth. Together, RC metrics and PROVE-Bench form the PROVE (Perceptual RemOVal cohErence) evaluation framework for visual media. Experiments across diverse image and video benchmarks demonstrate that RC achieves substantially stronger alignment with human judgments than existing evaluation protocols. Project page: https://xiaomi-research.github.io/prove/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑