深入探讨面向道路场景理解的多模态多任务基础模型：从学习范式的视角
Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives
AI总结：
本文综述了面向道路场景的多模态多任务视觉理解基础模型（MM-VUFMs），系统分析其学习范式、关键挑战与未来趋势，并提供了持续更新的资源库。
摘要：
基础模型确实对各个领域产生了深远的影响，成为显著塑造智能系统能力的关键组成部分。在智能车辆背景下，利用基础模型的力量已被证明具有变革性，为视觉理解带来了显著进步。具备多模态和多任务学习能力，多模态多任务视觉理解基础模型（MM-VUFMs）有效处理和融合来自不同模态的数据，同时以强大的适应性处理各种驾驶相关任务，有助于对周围场景进行更全面的理解。在本综述中，我们对专门针对道路场景的MM-VUFMs进行了系统分析。我们的目标不仅是提供常见实践的全面概述，包括任务特定模型、统一多模态模型、统一多任务模型和基础模型提示技术，还强调它们在多样化学习范式中的先进能力。这些范式包括开放世界理解、道路场景的高效迁移、持续学习、交互和生成能力。此外，我们提供了对关键挑战和未来趋势的见解，如闭环驾驶系统、可解释性、具身驾驶智能体和世界模型。为便于研究人员及时了解道路场景MM-VUFMs的最新进展，我们已在https://github.com/rolsheng/MM-VUFM4DS建立了一个持续更新的资源库。
英文摘要：
Foundation models have indeed made a profound impact on various fields, emerging as pivotal components that significantly shape the capabilities of intelligent systems. In the context of intelligent vehicles, leveraging the power of foundation models has proven to be transformative, offering notable advancements in visual understanding. Equipped with multi-modal and multi-task learning capabilities, multi-modal multi-task visual understanding foundation models (MM-VUFMs) effectively process and fuse data from diverse modalities and simultaneously handle various driving-related tasks with powerful adaptability, contributing to a more holistic understanding of the surrounding scene. In this survey, we present a systematic analysis of MM-VUFMs specifically designed for road scenes. Our objective is not only to provide a comprehensive overview of common practices, referring to task-specific models, unified multi-modal models, unified multi-task models, and foundation model prompting techniques, but also to highlight their advanced capabilities in diverse learning paradigms. These paradigms include open-world understanding, efficient transfer for road scenes, continual learning, interactive and generative capability. Moreover, we provide insights into key challenges and future trends, such as closed-loop driving systems, interpretability, embodied driving agents, and world models. To facilitate researchers in staying abreast of the latest developments in MM-VUFMs for road scenes, we have established a continuously updated repository at https://github.com/rolsheng/MM-VUFM4DS
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
