Change3D：从视频建模视角重新审视变化检测与描述
Change3D: Revisiting Change Detection and Captioning from A Video Modeling Perspective
AI总结：
针对现有变化检测与描述方法难以统一、特征编码关注变化区域不足的问题，提出基于视频建模的Change3D框架，通过插入可学习感知帧实现统一任务处理，以极低参数量和运算量取得更优性能。
摘要：
本文提出Change3D框架，该框架通过视频建模重新定义了变化检测与描述任务。现有方法将每对双时相图像视为独立帧，已取得显著成效：它们采用权值共享的图像编码器提取空间特征，再通过变化提取器捕捉两幅图像间的差异。但图像特征编码是任务无关的过程，无法有效关注变化区域；此外，针对不同变化检测与描述任务设计的各类变化提取器，也难以构建统一框架。为解决这些问题，Change3D将双时相图像视为类似微型视频的两帧，通过在双时相图像间插入可学习感知帧，让视频编码器驱动感知帧与图像直接交互并感知差异，从而无需复杂的变化提取器，为各类变化检测与描述任务提供统一框架。我们在8个标准基准上验证了Change3D在多类任务上的表现，涵盖变化检测（包括二值变化检测、语义变化检测和建筑损毁评估）与变化描述。无需额外技巧，这一简洁有效的框架仅用约为SOTA方法6%-13%的参数量、8%-34%的浮点运算量（FLOPs）的超轻量视频模型，就能取得更优性能。我们希望Change3D能成为基于2D的模型的替代方案，推动未来相关研究。
英文摘要：
In this paper, we present Change3D, a framework that reconceptualizes the change detection and captioning tasks through video modeling. Recent methods have achieved remarkable success by regarding each pair of bi-temporal images as separate frames. They employ a shared-weight image encoder to extract spatial features and then use a change extractor to capture differences between the two images. However, image feature encoding, being a task-agnostic process, cannot attend to changed regions effectively. Furthermore, different change extractors designed for various change detection and captioning tasks make it difficult to have a unified framework. To tackle these challenges, Change3D regards the bi-temporal images as comprising two frames akin to a tiny video. By integrating learnable perception frames between the bi-temporal images, a video encoder enables the perception frames to interact with the images directly and perceive their differences. Therefore, we can get rid of the intricate change extractors, providing a unified framework for different change detection and captioning tasks. We verify Change3D on multiple tasks, encompassing change detection (including binary change detection, semantic change detection, and building damage assessment) and change captioning, across eight standard benchmarks. Without bells and whistles, this simple yet effective framework can achieve superior performance with an ultra-light video model comprising only ~6%-13% of the parameters and ~8%-34% of the FLOPs compared to state-of-the-art methods. We hope that Change3D could be an alternative to 2D-based models and facilitate future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
