从言辞到车轮：使用基础模型理解人类语言指令的基于视觉的自动驾驶
Words to Wheels: Vision-Based Autonomous Driving Understanding Human Language Instructions Using Foundation Models
AI总结：
本文提出一种基于基础模型的UGV导航方法，利用LLM将语言指令转化为认知路线描述，结合地形分割模型与MPPI规划器进行局部路径规划，实现新环境下的目标导航。
摘要：
本文介绍了基础模型的一项创新应用，使配备RGB-D相机的无人地面车辆（UGV）能够根据人类语言指令导航至指定目的地。与基于学习的方法不同，该方法无需预先训练，而是利用现有基础模型，从而促进对新环境的泛化。在接收到人类语言指令后，这些指令被大型语言模型（LLM）转换为“认知路线描述”——以人类语言表达的详细导航路线。车辆随后将该描述分解为地标和导航操作。车辆还通过在开放数据集上训练的地形分割模型GANav确定高程成本并识别不同区域的可通行性级别。考虑了高程和可通行性级别的语义高程成本被估计并提供给负责局部路径规划的模型预测路径积分（MPPI）规划器。同时，车辆使用基础模型（包括YOLO-World和EfficientViT-SAM）搜索目标地标。最终，车辆执行导航命令以到达指定目的地，即最终地标。我们的实验表明，该应用成功引导UGV在陌生地形或城市环境等新环境中遵循人类语言指令到达目的地。
英文摘要：
This paper introduces an innovative application of foundation models, enabling Unmanned Ground Vehicles (UGVs) equipped with an RGB-D camera to navigate to designated destinations based on human language instructions. Unlike learning-based methods, this approach does not require prior training but instead leverages existing foundation models, thus facilitating generalization to novel environments. Upon receiving human language instructions, these are transformed into a 'cognitive route description' using a large language model (LLM)-a detailed navigation route expressed in human language. The vehicle then decomposes this description into landmarks and navigation maneuvers. The vehicle also determines elevation costs and identifies navigability levels of different regions through a terrain segmentation model, GANav, trained on open datasets. Semantic elevation costs, which take both elevation and navigability levels into account, are estimated and provided to the Model Predictive Path Integral (MPPI) planner, responsible for local path planning. Concurrently, the vehicle searches for target landmarks using foundation models, including YOLO-World and EfficientViT-SAM. Ultimately, the vehicle executes the navigation commands to reach the designated destination, the final landmark. Our experiments demonstrate that this application successfully guides UGVs to their destinations following human language instructions in novel environments, such as unfamiliar terrain or urban settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
