LocoVLM：为适应多功能腿式运动策略而 grounding 视觉和语言
LocoVLM: Grounding Vision and Language for Adapting Versatile Legged Locomotion Policies
AI总结：
LocoVLM通过整合高层常识推理和视觉语言模型，实现了基于指令的实时腿式运动策略适应，准确率达87%。
摘要：
最近的腿式运动学习进展仍然受到几何环境表示的主导，限制了机器人对更高层次语义如人类指令的响应能力。为了解决这一限制，我们提出了一种新的方法，将基础模型的高层常识推理整合到腿式运动适应过程中。具体而言，我们的方法利用预训练的大语言模型来合成一个针对腿式机器人的指令 grounding 技能数据库。预训练的视觉语言模型用于提取高层环境语义，并将其 grounding 到技能数据库中，使机器人能够实时获得技能建议。为了实现多功能技能控制，我们训练了一个风格条件化的策略，能够生成多样且稳健的运动技能，并且与指定风格高度一致。据我们所知，这是首次展示利用环境语义和指令的高层推理进行实时腿式运动适应的工作，且在无需在线查询云基础模型的情况下，指令遵循准确率可达87%。
英文摘要：
Recent advances in legged locomotion learning are still dominated by the utilization of geometric representations of the environment, limiting the robot's capability to respond to higher-level semantics such as human instructions. To address this limitation, we propose a novel approach that integrates high-level commonsense reasoning from foundation models into the process of legged locomotion adaptation. Specifically, our method utilizes a pre-trained large language model to synthesize an instruction-grounded skill database tailored for legged robots. A pre-trained vision-language model is employed to extract high-level environmental semantics and ground them within the skill database, enabling real-time skill advisories for the robot. To facilitate versatile skill control, we train a style-conditioned policy capable of generating diverse and robust locomotion skills with high fidelity to specified styles. To the best of our knowledge, this is the first work to demonstrate real-time adaptation of legged locomotion using high-level reasoning from environmental semantics and instructions with instruction-following accuracy of up to 87% without the need for online query to on-the-cloud foundation models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
