跨视角多模态分割 @ Ego-Exo4D 挑战赛 2025
Cross-View Multi-Modal Segmentation @ Ego-Exo4D Challenges 2025
AI总结：
本文针对 Ego-Exo4D 2025 目标对应任务，提出融合视觉掩码与文本描述的多模态条件融合模块及跨视角目标对齐模块，在排行榜上取得第二名。
摘要：
在本报告中，我们提出了一种跨视角多模态目标分割方法，用于 Ego-Exo4D Correspondence Challenges 2025 中的目标对应任务。给定来自一个视角（例如 ego 视角）的目标查询，目标是预测另一个视角（例如 exo 视角）中对应的目标掩码。为解决该任务，我们提出了一种多模态条件融合模块，通过同时利用视觉掩码和文本描述作为分割条件来增强目标定位。此外，为应对 ego 与 exo 视角之间的视觉域差距，我们引入了一种跨视角目标对齐模块，该模块强制跨视角的目标级一致性，从而提高模型对视角变化的鲁棒性。我们提出的方法在大规模 Ego-Exo4D 目标对应基准排行榜上排名第二。代码将在 https://github.com/lovelyqian/ObjectRelator 公开。
英文摘要：
In this report, we present a cross-view multi-modal object segmentation approach for the object correspondence task in the Ego-Exo4D Correspondence Challenges 2025. Given object queries from one perspective (e.g., ego view), the goal is to predict the corresponding object masks in another perspective (e.g., exo view). To tackle this task, we propose a multimodal condition fusion module that enhances object localization by leveraging both visual masks and textual descriptions as segmentation conditions. Furthermore, to address the visual domain gap between ego and exo views, we introduce a cross-view object alignment module that enforces object-level consistency across perspectives, thereby improving the model's robustness to viewpoint changes. Our proposed method ranked second on the leaderboard of the large-scale Ego-Exo4D object correspondence benchmark. Code will be made available at https://github.com/lovelyqian/ObjectRelator.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
