arXiv 2401.043342024-01-09cs.ROcs.AI

面向机器人学的大语言模型：机遇、挑战与展望

Large Language Models for Robotics: Opportunities, Challenges, and Perspectives

Northwestern Polytechnical University（西北工业大学）；The University of Georgia（佐治亚大学）；Shaanxi Normal University（陕西师范大学）；Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）

Jiaqi Wang, Zihao Wu, Yiwei Li, Hanqi Jiang, Peng Shu, Enze Shi, Huawen Hu, Chong Ma, Yiheng Liu, Xuhui Wang, Yincheng Yao, Xuan Liu, Huaqin Zhao, Zhengliang Li… 展开作者

Jiaqi Wang, Zihao Wu, Yiwei Li, Hanqi Jiang, Peng Shu, Enze Shi, Huawen Hu, Chong Ma, Yiheng Liu, Xuhui Wang, Yincheng Yao, Xuan Liu, Huaqin Zhao, Zhengliang Liu, Haixing Dai, Lin Zhao, Bao Ge, Xiang Li, Tianming Liu, Shu Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究综述了LLMs在机器人任务规划中的应用，并提出利用多模态GPT-4V结合自然语言与视觉感知增强具身任务规划的框架，有效提升了机器人在具身任务中的性能。

摘要：

大语言模型（LLMs）经历了显著扩展，并日益融入各个领域。值得注意的是，在机器人任务规划领域，LLMs利用其先进的推理和语言理解能力，根据自然语言指令制定精确且高效的行动计划。然而，对于机器人与复杂环境交互的具身任务，纯文本LLMs由于缺乏与机器人视觉感知的兼容性，常常面临挑战。本研究全面概述了LLMs和多模态LLMs在各类机器人任务中的新兴融合。此外，我们提出了一个框架，利用多模态GPT-4V，通过结合自然语言指令和机器人视觉感知来增强具身任务规划。基于多样化数据集的结果表明，GPT-4V有效提升了机器人在具身任务中的性能。这项对LLMs和多模态LLMs在各类机器人任务中的广泛调查与评估，丰富了对以LLM为中心的具身智能的理解，并为弥合人-机-环境交互中的差距提供了前瞻性见解。

英文摘要：

Large language models (LLMs) have undergone significant expansion and have been increasingly integrated across various domains. Notably, in the realm of robot task planning, LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions. However, for embodied tasks, where robots interact with complex environments, text-only LLMs often face challenges due to a lack of compatibility with robotic visual perception. This study provides a comprehensive overview of the emerging integration of LLMs and multimodal LLMs into various robotic tasks. Additionally, we propose a framework that utilizes multimodal GPT-4V to enhance embodied task planning through the combination of natural language instructions and robot visual perceptions. Our results, based on diverse datasets, indicate that GPT-4V effectively enhances robot performance in embodied tasks. This extensive survey and evaluation of LLMs and multimodal LLMs across a variety of robotic tasks enriches the understanding of LLM-centric embodied intelligence and provides forward-looking insights toward bridging the gap in Human-Robot-Environment interaction.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。