arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.04868cs.AI
收藏

K-Gen:一种多模态语言条件方法用于可解释的关键点引导轨迹生成

K-Gen: A Multimodal Language-Conditioned Approach for Interpretable Keypoint-Guided Trajectory Generation

Harbin Institute of Technology Chongqing Research Inst. of HIT(哈尔滨工业大学重庆研究院);Changan Automobile Co., Ltd Chongqing, China(长安汽车有限公司重庆)

Mingxuan Mu, Guo Yang, Lei Chen, Ping Wu, Jianxun Cui

AI总结:

K-Gen通过多模态语言模型结合关键点引导方法,提升自动驾驶轨迹生成的可解释性和准确性。

摘要:

生成逼真且多样的轨迹是自动驾驶仿真中的关键挑战。尽管大型语言模型(LLMs)显示出潜力,但现有方法往往依赖于结构化数据,如向量地图,这无法捕捉场景中丰富的非结构化视觉上下文。为此,我们提出了K-Gen,一种可解释的关键点引导多模态框架,利用多模态大语言模型(MLLMs)将栅格化BEV地图输入与文本场景描述统一起来。与直接预测完整轨迹不同,K-Gen生成可解释的关键点并伴有反映代理意图的推理,随后通过细化模块将其精炼为准确的轨迹。为进一步提高关键点生成,我们应用T-DAPO,一种具有轨迹意识的强化学习微调算法。在WOMD和nuPlan上的实验表明,K-Gen优于现有基线,凸显了将多模态推理与关键点引导轨迹生成结合的有效性。

英文摘要:

Generating realistic and diverse trajectories is a critical challenge in autonomous driving simulation. While Large Language Models (LLMs) show promise, existing methods often rely on structured data like vectorized maps, which fail to capture the rich, unstructured visual context of a scene. To address this, we propose K-Gen, an interpretable keypoint-guided multimodal framework that leverages Multimodal Large Language Models (MLLMs) to unify rasterized BEV map inputs with textual scene descriptions. Instead of directly predicting full trajectories, K-Gen generates interpretable keypoints along with reasoning that reflects agent intentions, which are subsequently refined into accurate trajectories by a refinement module. To further enhance keypoint generation, we apply T-DAPO, a trajectory-aware reinforcement fine-tuning algorithm. Experiments on WOMD and nuPlan demonstrate that K-Gen outperforms existing baselines, highlighting the effectiveness of combining multimodal reasoning with keypoint-guided trajectory generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑