通过任务特定提示与空间推理增强面向自动驾驶的视觉语言模型
Enhancing Vision-Language Models for Autonomous Driving through Task-Specific Prompting and Spatial Reasoning
AI总结：
针对IROS 2025 RoboSense挑战赛的自动驾驶场景理解任务，提出含四大组件的框架，基于Qwen2.5-VL-72B实现，在两阶段测试中分别取得70.87%和72.85%的平均准确率，有效提升VLM性能。
摘要：
本技术报告展示了我们针对IROS 2025 RoboSense挑战赛的解决方案，该赛事评估视觉语言模型（VLM）在自动驾驶场景理解中感知、预测、规划和损坏检测任务的表现。我们提出了一个由四大核心组件构成的系统性框架。第一，混合提示路由器对问题进行分类，并将其分派到任务特定的专家提示，消除不同问题类型间的干扰。第二，任务特定提示嵌入了显式坐标系、空间推理规则、角色扮演、思维链/思维树推理，以及为各任务定制的少样本示例。第三，视觉组装模块根据问题需求，将多视图图像与目标裁剪图、品红色标记及自适应历史帧进行组合。第四，我们为每个任务配置模型推理参数（温度、top-p、消息角色）以优化输出质量。我们的方法基于Qwen2.5-VL-72B实现，在第一阶段（干净数据）取得70.87%的平均准确率，在第二阶段（损坏数据）取得72.85%的平均准确率，证明结构化提示与空间接地可大幅提升VLM在安全关键型自动驾驶任务中的性能。代码与提示可在https://github.com/wuaodi/UCAS-CSU-phase2获取。
英文摘要：
This technical report presents our solution for the RoboSense Challenge at IROS 2025, which evaluates Vision-Language Models (VLMs) on autonomous driving scene understanding across perception, prediction, planning, and corruption detection tasks. We propose a systematic framework built on four core components. First, a Mixture-of-Prompts router classifies questions and dispatches them to task-specific expert prompts, eliminating interference across diverse question types. Second, task-specific prompts embed explicit coordinate systems, spatial reasoning rules, role-playing, Chain-of-Thought/Tree-of-Thought reasoning, and few-shot examples tailored to each task. Third, a visual assembly module composes multi-view images with object crops, magenta markers, and adaptive historical frames based on question requirements. Fourth, we configure model inference parameters (temperature, top-p, message roles) per task to optimize output quality. Implemented on Qwen2.5-VL-72B, our approach achieves 70.87% average accuracy on Phase-1 (clean data) and 72.85% on Phase-2 (corrupted data), demonstrating that structured prompting and spatial grounding substantially enhance VLM performance on safety-critical autonomous driving tasks. Code and prompt are available at https://github.com/wuaodi/UCAS-CSU-phase2.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
