arXiv 2510.021042025-10-02cs.RO

LangGrasp：利用微调后的大语言模型实现含模糊指令的语言交互式机器人抓取

LangGrasp: Leveraging Fine-Tuned LLMs for Language Interactive Robot Grasping with Ambiguous Instructions

School of Computer Science and Technology（计算机科学与技术学院）；Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System（湖北省智能信息处理与实时工业系统重点实验室）；Institute of Robotics and Intelligent Systems（机器人与智能系统研究院）；Wuhan University of Science and Technology（武汉理工大学）

Yunhan Lin, Wenqi Wu, Zhijie Zhang, Huasong Min

备注：: 8 pages, 6 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有语言驱动抓取方法难处理含隐含意图的模糊指令的问题，提出整合微调LLM与2D分割引导点云定位模块的LangGrasp框架，可解析隐含意图、实现物体到部件级的高精度抓取，提升机器人非结构化环境适应性与效率。

摘要：

现有的语言驱动抓取方法难以充分处理包含隐含意图的模糊指令。为应对这一挑战，我们提出LangGrasp，一种新型语言交互式机器人抓取框架。该框架整合了微调后的大语言模型（LLMs），利用其强大的常识理解与环境感知能力，从语言指令中推导隐含意图，明确任务要求与目标操作对象。此外，我们设计的点云定位模块在2D部件分割的引导下，可实现场景中的部分点云定位，从而将抓取操作从粗粒度的物体级拓展到细粒度的部件级操作。实验结果表明，LangGrasp框架能准确解析模糊指令中的隐含意图，识别出未明确说明但对任务完成至关重要的操作与目标信息；同时它还能整合环境信息，动态选择最优抓取姿态，实现从物体级到部件级操作的高精度抓取，显著提升机器人在非结构化环境中的适应性与任务执行效率。更多信息与代码可访问：https://github.com/wu467/LangGrasp。

英文摘要：

The existing language-driven grasping methods struggle to fully handle ambiguous instructions containing implicit intents. To tackle this challenge, we propose LangGrasp, a novel language-interactive robotic grasping framework. The framework integrates fine-tuned large language models (LLMs) to leverage their robust commonsense understanding and environmental perception capabilities, thereby deducing implicit intents from linguistic instructions and clarifying task requirements along with target manipulation objects. Furthermore, our designed point cloud localization module, guided by 2D part segmentation, enables partial point cloud localization in scenes, thereby extending grasping operations from coarse-grained object-level to fine-grained part-level manipulation. Experimental results show that the LangGrasp framework accurately resolves implicit intents in ambiguous instructions, identifying critical operations and target information that are unstated yet essential for task completion. Additionally, it dynamically selects optimal grasping poses by integrating environmental information. This enables high-precision grasping from object-level to part-level manipulation, significantly enhancing the adaptability and task execution efficiency of robots in unstructured environments. More information and code are available here: https://github.com/wu467/LangGrasp.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。