arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.25448cs.CV
收藏

UBD-Bench:自动驾驶多模态感知中单模态分支退化的基准测试与分析

UBD-Bench: Benchmarking and Analyzing Unimodal Branch Degradation in Multimodal Perception for Autonomous Driving

Beihang University(北京航空航天大学);Xiaomi EV(小米汽车);University of Macau(澳门大学)

Ang Liu, Xiang Gu, Ruikai Li, Qiyue Wang, Yizhe Fan, Hao Shan, Ziyang Yan, Han Jiang, Zhenning Li, Xiaoshuai Hao, Yilong Ren, Zhiyong Cui, Haiyang Yu

AI总结:

UBD-Bench基准测试发现自动驾驶多模态感知中相机分支易退化,引入单模态到多模态蒸馏可改善相机分支及全模型性能,凸显分支表征质量的评估重要性。

摘要:

多模态感知通过融合互补的传感器信息提升自动驾驶性能,但更强的多模态预测并不一定意味着各模态特定分支的表征更强,这类分支层面的弱点可能在传统的输出级评估中被隐藏。我们提出UBD-Bench,这是一个用于研究多模态感知中单模态分支退化(Unimodal Branch Degradation,UBD)的系统性基准。UBD-Bench采用配对冻结探针(Paired Frozen-Branch Probe),在匹配的解码条件下,对比相同模态经独立单模态训练与多模态联合训练后的任务相关表征质量。在四项自动驾驶感知任务和10个模型上,我们发现相机(Camera)分支在多模态联合训练后持续退化,而激光雷达(LiDAR)分支的变化更弱且更依赖模型,揭示出显著的模态不对称性。分层探针进一步显示,相机分支的退化从视觉主干延伸至相机到鸟瞰图(Camera-to-BEV)通路,在融合前的相机BEV表征处最为严重。基于该诊断,我们在退化的表征处引入单模态到多模态的蒸馏(unimodal-to-multimodal distillation),该干预在所有四项任务中均同时提升了相机分支探针性能和全模型性能。这些结果表明,多模态性能提升可能与退化的单模态表征共存,凸显分支层面的表征质量是评估和改进多模态感知的重要维度。

英文摘要:

Multimodal perception improves autonomous-driving performance by combining complementary sensor information. However, stronger multimodal predictions do not necessarily imply stronger representations in each modality-specific branch. Such branch-level weaknesses can remain hidden in conventional output-level evaluation. We introduce UBD-Bench, a systematic benchmark for studying unimodal branch degradation (UBD) in multimodal perception. UBD-Bench uses a Paired Frozen-Branch Probe to compare the task-relevant representation quality of the same modality after independent unimodal training and multimodal joint training under matched decoding conditions. Across four autonomous-driving perception tasks and 10 models, we find that Camera branches consistently degrade after multimodal joint training, whereas LiDAR branches exhibit weaker and more model-dependent changes, revealing a pronounced modality asymmetry. Hierarchical probing further shows that Camera-branch degradation extends from the visual backbone through the Camera-to-BEV pathway and is most severe at the pre-fusion Camera BEV representation. Guided by this diagnosis, we introduce unimodal-to-multimodal distillation at the degraded representation. This intervention consistently improves both Camera-branch Probe performance and full-model performance across all four tasks. These results show that multimodal gains can coexist with degraded unimodal representations, highlighting branch-level representation quality as an important dimension for evaluating and improving multimodal perception.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑