PKRD-CoT：面向自动驾驶中多模态大语言模型的统一思维链提示
PKRD-CoT: A Unified Chain-of-thought Prompting for Multi-Modal Large Language Models in Autonomous Driving
AI总结：
本文提出基于感知、知识、推理与决策的 PKRD-CoT 零样本思维链提示框架，使 GPT-4.0 等多模态大语言模型无需训练即可提升自动驾驶实时决策表现。
摘要：
人们越来越关注在自动驾驶场景中直接利用功能强大的多模态大语言模型（MLLMs）的能力。然而，设计和训练端到端自动驾驶模型的高昂成本与复杂性，使许多企业和研究机构难以应对。为解决这一问题，本研究探索将 MLLMs 无缝集成到自动驾驶系统中，提出了一种名为 PKRD-CoT 的零样本思维链（Zero-Shot-CoT）提示设计。PKRD-CoT 基于自动驾驶的四项基本能力：感知、知识、推理和决策。这使其特别适合通过逐步模仿人类思维过程来理解并响应动态驾驶环境，从而增强实时场景中的决策能力。我们的设计使 MLLMs 能够在没有先前经验的情况下解决问题，从而提高其在非结构化自动驾驶环境中的实用性。在实验中，我们展示了搭载 PKRD-CoT 的 GPT-4.0 在各项自动驾驶任务上的卓越性能，突出了其在自动驾驶场景中的有效性。此外，我们的基准分析揭示了 PKRD-CoT 对其他 MLLMs（如 Claude、LLava1.6 和 Qwen-VL-Plus）具有良好的应用可行性。总体而言，本研究为 GPT-4.0 和其他 MLLMs 在自动驾驶中的应用贡献了一种新颖且统一的提示设计框架，同时通过全面比较，严格评估了这些广受认可的 MLLMs 在自动驾驶领域的效能。
英文摘要：
There is growing interest in leveraging the capabilities of robust Multi-Modal Large Language Models (MLLMs) directly within autonomous driving contexts. However, the high costs and complexity of designing and training end-to-end autonomous driving models make them challenging for many enterprises and research entities. To address this, our study explores a seamless integration of MLLMs into autonomous driving systems by proposing a Zero-Shot Chain-of-Thought (Zero-Shot-CoT) prompt design named PKRD-CoT. PKRD-CoT is based on the four fundamental capabilities of autonomous driving: perception, knowledge, reasoning, and decision-making. This makes it particularly suitable for understanding and responding to dynamic driving environments by mimicking human thought processes step by step, thus enhancing decision-making in real-time scenarios. Our design enables MLLMs to tackle problems without prior experience, thereby increasing their utility within unstructured autonomous driving environments. In experiments, we demonstrate the exceptional performance of GPT-4.0 with PKRD-CoT across autonomous driving tasks, highlighting its effectiveness in autonomous driving scenarios. Additionally, our benchmark analysis reveals the promising viability of PKRD-CoT for other MLLMs, such as Claude, LLava1.6, and Qwen-VL-Plus. Overall, this study contributes a novel and unified prompt-design framework for GPT-4.0 and other MLLMs in autonomous driving, while also rigorously evaluating the efficacy of these widely recognized MLLMs in the autonomous driving domain through comprehensive comparisons.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
