通过双模态因果干预实现鲁棒的自我参照视频目标分割
Robust Egocentric Referring Video Object Segmentation via Dual-Modal Causal Intervention
AI总结：
CERES通过双模态因果干预方法，在自我参照视频中实现鲁棒的目标分割，提升对快速运动和遮挡的鲁棒性。
摘要：
自我参照视频目标分割（Ego-RVOS）旨在在第一人称视频中对参与人类动作的特定对象进行分割，该对象由语言查询描述。这一任务对于理解自我参照人类行为至关重要。然而，由于自我参照视频中固有的模糊性和训练数据中的偏差，实现这种分割的鲁棒性具有挑战性。现有方法往往难以应对，因为它们从数据集中倾斜的对象-动作配对中学习到虚假的相关性，以及自我参照视角的基本视觉混淆因素，如快速运动和频繁遮挡。为了解决这些限制，我们引入了因果自我参照分割（CERES），一种可插拔的因果框架，将强大的、预训练的RVOS骨干网络适应到自我参照领域。CERES实现了双模态因果干预：应用后门调整原理来对抗从数据集统计中学习到的语言表示偏差，并利用前门调整概念来通过智能整合语义视觉特征与几何深度信息来解决视觉混淆，这些信息由因果原则引导，从而创建更鲁棒于自我参照扭曲的表示。广泛的实验表明，CERES在Ego-RVOS基准测试中实现了最先进的性能，突显了应用因果推理来构建更可靠的模型以实现更广泛的自我参照视频理解的潜力。
英文摘要：
Egocentric Referring Video Object Segmentation (Ego-RVOS) aims to segment the specific object actively involved in a human action, as described by a language query, within first-person videos. This task is critical for understanding egocentric human behavior. However, achieving such segmentation robustly is challenging due to ambiguities inherent in egocentric videos and biases present in training data. Consequently, existing methods often struggle, learning spurious correlations from skewed object-action pairings in datasets and fundamental visual confounding factors of the egocentric perspective, such as rapid motion and frequent occlusions. To address these limitations, we introduce Causal Ego-REferring Segmentation (CERES), a plug-in causal framework that adapts strong, pre-trained RVOS backbones to the egocentric domain. CERES implements dual-modal causal intervention: applying backdoor adjustment principles to counteract language representation biases learned from dataset statistics, and leveraging front-door adjustment concepts to address visual confounding by intelligently integrating semantic visual features with geometric depth information guided by causal principles, creating representations more robust to egocentric distortions. Extensive experiments demonstrate that CERES achieves state-of-the-art performance on Ego-RVOS benchmarks, highlighting the potential of applying causal reasoning to build more reliable models for broader egocentric video understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
