基于粗到细学习的语言引导抓取检测用于机器人操作
Language-Guided Grasp Detection with Coarse-to-Fine Learning for Robotic Manipulation
AI总结：
本文提出基于粗到细学习的语言引导抓取检测方法，通过跨模态融合和动态卷积头提升抓取精度与鲁棒性，实验证明其在复杂环境中的有效性。
摘要：
抓取是机器人操作中最基本的挑战性能力之一，尤其是在无结构、杂乱和语义多样的环境中。最近的研究越来越多地探索了语言引导的操作，其中机器人不仅感知场景，还能解读与任务相关的自然语言指令。然而，现有的语言条件抓取方法通常依赖于浅层融合策略，导致语义接地有限，语言意图与视觉抓取推理之间的对齐较弱。在本文中，我们提出了一种基于粗到细学习的Language-Guided Grasp Detection（LGGD）方法，用于机器人操作。LGGD利用基于CLIP的视觉和文本嵌入，在分层的跨模态融合管道中进行视觉和文本嵌入的融合，逐步将语言线索注入视觉特征重建过程中。这种设计实现了细粒度的视觉-语义对齐，并提高了预测抓取与任务指令的相关性。此外，我们引入了一个语言条件的动态卷积头（LDCH），该头基于句子级别的特征混合多个卷积专家，实现了指令自适应的粗略掩码和抓取预测。最后的细化模块进一步增强了复杂场景中的抓取一致性与鲁棒性。在OCID-VLG和Grasp-Anything++数据集上的实验表明，LGGD优于现有的语言引导抓取方法，表现出对未见物体和多样化语言查询的强大泛化能力。此外，在真实机器人平台上的部署展示了我们方法在执行准确、基于指令的抓取动作方面的实际有效性。代码将在接受后公开发布。
英文摘要：
Grasping is one of the most fundamental challenging capabilities in robotic manipulation, especially in unstructured, cluttered, and semantically diverse environments. Recent researches have increasingly explored language-guided manipulation, where robots not only perceive the scene but also interpret task-relevant natural language instructions. However, existing language-conditioned grasping methods typically rely on shallow fusion strategies, leading to limited semantic grounding and weak alignment between linguistic intent and visual grasp reasoning.In this work, we propose Language-Guided Grasp Detection (LGGD) with a coarse-to-fine learning paradigm for robotic manipulation. LGGD leverages CLIP-based visual and textual embeddings within a hierarchical cross-modal fusion pipeline, progressively injecting linguistic cues into the visual feature reconstruction process. This design enables fine-grained visual-semantic alignment and improves the feasibility of the predicted grasps with respect to task instructions. In addition, we introduce a language-conditioned dynamic convolution head (LDCH) that mixes multiple convolution experts based on sentence-level features, enabling instruction-adaptive coarse mask and grasp predictions. A final refinement module further enhances grasp consistency and robustness in complex scenes.Experiments on the OCID-VLG and Grasp-Anything++ datasets show that LGGD surpasses existing language-guided grasping methods, exhibiting strong generalization to unseen objects and diverse language queries. Moreover, deployment on a real robotic platform demonstrates the practical effectiveness of our approach in executing accurate, instruction-conditioned grasp actions. The code will be released publicly upon acceptance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
