Percept-WAM：感知增强的环境感知-行动模型用于鲁棒的端到端自动驾驶
Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Driving
AI总结：
Percept-WAM通过整合2D/3D场景理解能力，提升自动驾驶的感知与行动决策，实现端到端鲁棒性能。
摘要：
自动驾驶严重依赖于准确且鲁棒的空间感知。许多故障源于不准确和不稳定，尤其是在长尾场景和复杂交互中。然而，当前的视觉-语言模型在空间定位和理解方面表现薄弱，基于它们的VLA系统因此表现出有限的感知和定位能力。为了解决这些挑战，我们引入了Percept-WAM，一种感知增强的环境感知-行动模型，这是首个在单一视觉-语言模型（VLM）中隐式整合2D/3D场景理解能力的模型。不同于依赖问答式空间推理，Percept-WAM将2D/3D感知任务统一到World-PV和World-BEV标记中，这些标记编码了空间坐标和置信度。我们提出了一种基于网格的预测机制用于密集物体感知，结合了IoU感知评分和并行自回归解码，提高了在长尾、远距离和小物体场景中的稳定性。此外，Percept-WAM利用预训练的VLM参数来保留通用智能（例如逻辑推理），并且可以直接输出感知结果和轨迹控制输出。实验显示，Percept-WAM在下游感知基准上与经典检测器和分割器匹配或超越，实现了51.7/58.9 mAP在COCO 2D检测和nuScenes BEV 3D检测上。当与轨迹解码器结合时，它进一步提高了nuScenes和NAVSIM上的规划性能，例如在NAVSIM上PMDS上超越DiffusionDrive 2.1。定性结果进一步突显了其强大的开放词汇和长尾泛化能力。
英文摘要：
Autonomous driving heavily relies on accurate and robust spatial perception. Many failures arise from inaccuracies and instability, especially in long-tail scenarios and complex interactions. However, current vision-language models are weak at spatial grounding and understanding, and VLA systems built on them therefore show limited perception and localization ability. To address these challenges, we introduce Percept-WAM, a perception-enhanced World-Awareness-Action Model that is the first to implicitly integrate 2D/3D scene understanding abilities within a single vision-language model (VLM). Instead of relying on QA-style spatial reasoning, Percept-WAM unifies 2D/3D perception tasks into World-PV and World-BEV tokens, which encode both spatial coordinates and confidence. We propose a grid-conditioned prediction mechanism for dense object perception, incorporating IoU-aware scoring and parallel autoregressive decoding, improving stability in long-tail, far-range, and small-object scenarios. Additionally, Percept-WAM leverages pretrained VLM parameters to retain general intelligence (e.g., logical reasoning) and can output perception results and trajectory control outputs directly. Experiments show that Percept-WAM matches or surpasses classical detectors and segmenters on downstream perception benchmarks, achieving 51.7/58.9 mAP on COCO 2D detection and nuScenes BEV 3D detection. When integrated with trajectory decoders, it further improves planning performance on nuScenes and NAVSIM, e.g., surpassing DiffusionDrive by 2.1 in PMDS on NAVSIM. Qualitative results further highlight its strong open-vocabulary and long-tail generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
