VILTA：一种用于增强驾驶策略鲁棒性的视觉语言模型闭环对抗者
VILTA: A VLM-in-the-Loop Adversary for Enhancing Driving Policy Robustness
AI总结：
VILTA通过整合视觉语言模型到闭环训练中，提升自动驾驶策略在长尾事件中的安全性和鲁棒性。
摘要：
自动驾驶（AD）系统安全部署的根本障碍在于长尾问题，即罕见但关键的驾驶场景在现实数据中严重代表性不足。现有解决方案包括安全关键场景生成和闭环学习，通常依赖于基于规则的启发式方法、重采样方法和从离线数据集中学习的生成模型，限制了它们生成多样和新颖挑战的能力。尽管近期工作利用视觉语言模型（VLMs）生成场景描述，以指导独立的下游模型生成危险轨迹，但这种两阶段框架限制了VLMs的生成潜力，因为最终轨迹的多样性最终受限于下游算法的泛化天花板。为了克服这些限制，我们引入VILTA（视觉语言模型闭环轨迹对抗者），一种将VLM整合到AD代理闭环训练中的新框架。与以往工作不同，VILTA通过理解动态驾驶环境并直接编辑周围代理的未来轨迹，主动参与训练循环，战略性地生成具有挑战性的场景。这种直接编辑方法充分利用了VLM强大的泛化能力，创建了一个多样化且具有挑战性的场景课程，超越了传统方法的范围。我们证明，我们的方法显著提高了所生成AD策略的安全性和鲁棒性，特别是在其导航关键长尾事件的能力方面。
英文摘要：
The safe deployment of autonomous driving (AD) systems is fundamentally hindered by the long-tail problem, where rare yet critical driving scenarios are severely underrepresented in real-world data. Existing solutions including safety-critical scenario generation and closed-loop learning often rely on rule-based heuristics, resampling methods and generative models learned from offline datasets, limiting their ability to produce diverse and novel challenges. While recent works leverage Vision Language Models (VLMs) to produce scene descriptions that guide a separate, downstream model in generating hazardous trajectories for agents, such two-stage framework constrains the generative potential of VLMs, as the diversity of the final trajectories is ultimately limited by the generalization ceiling of the downstream algorithm. To overcome these limitations, we introduce VILTA (VLM-In-the-Loop Trajectory Adversary), a novel framework that integrates a VLM into the closed-loop training of AD agents. Unlike prior works, VILTA actively participates in the training loop by comprehending the dynamic driving environment and strategically generating challenging scenarios through direct, fine-grained editing of surrounding agents' future trajectories. This direct-editing approach fully leverages the VLM's powerful generalization capabilities to create a diverse curriculum of plausible yet challenging scenarios that extend beyond the scope of traditional methods. We demonstrate that our approach substantially enhances the safety and robustness of the resulting AD policy, particularly in its ability to navigate critical long-tail events.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
