arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.05337cs.CVcs.LGcs.RO
收藏

ACT-Bench:面向自动驾驶的动作可控世界模型研究

ACT-Bench: Towards Action Controllable World Models for Autonomous Driving

Turing Inc.(图灵公司)

Hidehisa Arai, Keishi Ishihara, Tsubasa Takahashi, Yu Yamaguchi

AI总结:

针对自动驾驶世界模型动作指令保真度评估不足、现有研究闭源难复现的问题,提出开源评估框架ACT-Bench及基线模型Terra,验证了现有SOTA模型动作遵循性不足,而Terra动作保真度更优,相关组件将开源。

摘要:

世界模型已成为自动驾驶领域颇具前景的神经模拟器,有望补充稀缺的真实世界数据并支持闭环评估。但当前研究主要依据视觉真实感或下游任务性能评估这类模型,对特定动作指令的保真度关注有限——而这是生成针对性仿真场景的关键属性。尽管部分研究涉及动作保真度,但其评估依赖闭源机制,可复现性受限。为填补这一空白,我们开发了开源评估框架ACT-Bench(动作可控基准)用于量化动作保真度,同时提出了基线世界模型Terra。我们的基准框架包含一个大规模数据集,将nuScenes的短上下文视频与对应的未来轨迹数据配对,为生成未来视频帧提供条件输入,且支持对已执行动作的动作保真度评估。此外,Terra在多个带轨迹标注的大规模数据集上训练,以提升动作保真度。借助该框架我们证实,当前最优模型无法完全遵循给定指令,而Terra实现了更高的动作保真度。我们基准框架的所有组件都将公开,以支撑未来研究。

英文摘要:

World models have emerged as promising neural simulators for autonomous driving, with the potential to supplement scarce real-world data and enable closed-loop evaluations. However, current research primarily evaluates these models based on visual realism or downstream task performance, with limited focus on fidelity to specific action instructions - a crucial property for generating targeted simulation scenes. Although some studies address action fidelity, their evaluations rely on closed-source mechanisms, limiting reproducibility. To address this gap, we develop an open-access evaluation framework, ACT-Bench, for quantifying action fidelity, along with a baseline world model, Terra. Our benchmarking framework includes a large-scale dataset pairing short context videos from nuScenes with corresponding future trajectory data, which provides conditional input for generating future video frames and enables evaluation of action fidelity for executed motions. Furthermore, Terra is trained on multiple large-scale trajectory-annotated datasets to enhance action fidelity. Leveraging this framework, we demonstrate that the state-of-the-art model does not fully adhere to given instructions, while Terra achieves improved action fidelity. All components of our benchmark framework will be made publicly available to support future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑