arXiv 2504.022642025-04-03cs.CV

MMTL-UniAD：辅助驾驶感知中的多模态与多任务学习统一框架

MMTL-UniAD: A Unified Framework for Multimodal and Multi-Task Learning in Assistive Driving Perception

Beijing Institute of Technology（北京理工大学）；Tsinghua University（清华大学）；HKUST(GZ)（香港科技大学（广州））；Beijing University of Chemical Technology（北京化工大学）；Beihang University（北京航空航天大学）

Wenzhuo Liu, Wenshuo Wang, Yicheng Qiao, Qiannan Guo, Jiayin Zhu, Pengfei Li, Zilong Chen, Huiming Yang, Zhiwei Li, Lening Wang, Tiao Tan, Huaping Liu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出统一多模态多任务学习框架 MMTL-UniAD，通过多轴区域注意力网络和双分支多模态嵌入解决任务间负迁移问题，在 AIDE 数据集上的四项辅助驾驶感知任务中均超越现有方法。

摘要：

高级驾驶辅助系统需要全面理解驾驶员的心理/生理状态和交通环境，但现有工作往往忽视了这些任务之间联合学习的潜在益处。本文提出 MMTL-UniAD，一个统一的多模态多任务学习框架，同时识别驾驶员行为（如环顾四周、交谈）、驾驶员情绪（如焦虑、快乐）、车辆行为（如停车、转弯）和交通环境（如交通拥堵、交通顺畅）。一个关键挑战是避免任务之间的负迁移，这会损害学习性能。为此，我们在框架中引入两个关键组件：一是多轴区域注意力网络，用于提取全局上下文敏感特征；二是双分支多模态嵌入，用于从任务共享和任务特定特征中学习多模态嵌入。前者利用多注意力机制提取任务相关特征，缓解由任务无关特征引起的负迁移；后者采用双分支结构自适应调整任务共享和任务特定参数，在减少任务冲突的同时增强跨任务知识迁移。我们在 AIDE 数据集上评估 MMTL-UniAD，通过一系列消融实验，证明其在所有四项任务上均优于现有最先进方法。代码可在 https://github.com/Wenzhuo-Liu/MMTL-UniAD 获取。

英文摘要：

Advanced driver assistance systems require a comprehensive understanding of the driver's mental/physical state and traffic context but existing works often neglect the potential benefits of joint learning between these tasks. This paper proposes MMTL-UniAD, a unified multi-modal multi-task learning framework that simultaneously recognizes driver behavior (e.g., looking around, talking), driver emotion (e.g., anxiety, happiness), vehicle behavior (e.g., parking, turning), and traffic context (e.g., traffic jam, traffic smooth). A key challenge is avoiding negative transfer between tasks, which can impair learning performance. To address this, we introduce two key components into the framework: one is the multi-axis region attention network to extract global context-sensitive features, and the other is the dual-branch multimodal embedding to learn multimodal embeddings from both task-shared and task-specific features. The former uses a multi-attention mechanism to extract task-relevant features, mitigating negative transfer caused by task-unrelated features. The latter employs a dual-branch structure to adaptively adjust task-shared and task-specific parameters, enhancing cross-task knowledge transfer while reducing task conflicts. We assess MMTL-UniAD on the AIDE dataset, using a series of ablation studies, and show that it outperforms state-of-the-art methods across all four tasks. The code is available on https://github.com/Wenzhuo-Liu/MMTL-UniAD.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。