arXiv 2602.135882026-02-14cs.CVcs.AI

双流交互式场景解析与几何视觉任务联合学习

Two-Stream Interactive Joint Learning of Scene Parsing and Geometric Vision Tasks

College of Electronic and Information Engineering, Tongji University（同济大学电子信息学院）；Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学智能自主系统研究所）；College of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；Department of Vehicle Control System and Software Development, NIO（蔚来汽车车辆控制系统与软件开发部门）；Department of Automotive Engineering, Jilin University（吉林大学汽车工程学院）；Key Laboratory of Embedded System and Service Computing (Ministry of Education), Tongji University（同济大学嵌入式系统与服务计算重点实验室）；College of Electronic and Information Engineering, Shanghai Institute of Intelligent Science and Technology（上海智能科学与技术研究院电子信息学院）；Shanghai Key Laboratory of Intelligent Autonomous Systems（上海智能自主系统重点实验室）

Guanfeng Tang, Hongbo Zhao, Ziwei Long, Jiayao Li, Bohong Xiao, Wei Ye, Hanli Wang, Rui Fan

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

TwInS通过双流交互式联合学习框架，实现场景解析与几何视觉任务的同步优化，采用跨任务适配器提升性能并减少对人工标注的依赖。

摘要：

受人类视觉系统启发，该系统通过两个平行但相互作用的流进行上下文和空间理解。本文提出了Two Interactive Streams（TwInS），一种新颖的生物启发式联合学习框架，能够同时执行场景解析和几何视觉任务。TwInS采用统一的通用架构，在场景解析流中的多级上下文特征被注入到几何视觉流中，以指导其迭代细化。在反向方向，解码的几何特征被投影到上下文特征空间中，通过新的跨任务适配器进行选择性异构特征融合，利用丰富的跨视角几何线索来增强场景解析。为消除对昂贵的人工标注对应关系地面真实值的依赖，TwInS进一步配备了定制的半监督训练策略，释放大规模多视角数据的潜力，并在不需要地面真实对应关系的情况下实现持续的自我进化。在三个公开数据集上进行的广泛实验验证了TwInS核心组件的有效性，并展示了其优于现有最先进方法的优越性能。源代码将在发表时公开。

英文摘要：

Inspired by the human visual system, which operates on two parallel yet interactive streams for contextual and spatial understanding, this article presents Two Interactive Streams (TwInS), a novel bio-inspired joint learning framework capable of simultaneously performing scene parsing and geometric vision tasks. TwInS adopts a unified, general-purpose architecture in which multi-level contextual features from the scene parsing stream are infused into the geometric vision stream to guide its iterative refinement. In the reverse direction, decoded geometric features are projected into the contextual feature space for selective heterogeneous feature fusion via a novel cross-task adapter, which leverages rich cross-view geometric cues to enhance scene parsing. To eliminate the dependence on costly human-annotated correspondence ground truth, TwInS is further equipped with a tailored semi-supervised training strategy, which unleashes the potential of large-scale multi-view data and enables continuous self-evolution without requiring ground-truth correspondences. Extensive experiments conducted on three public datasets validate the effectiveness of TwInS's core components and demonstrate its superior performance over existing state-of-the-art approaches. The source code will be made publicly available upon publication.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。