GROW$^2$：为机器人工具使用确定哪个物体和哪个部位
GROW$^2$: Grounding Which and Where for Robot Tool Use
AI总结：
提出GROW$^2$方法，通过分层语义和几何接地实现开放世界工具选择与部位定位，在零样本泛化中优于基线。
摘要：
如果没有刀，机器人能用盘子切蛋糕吗？工具使用极大地扩展了机器人的能力，但要创造性地使用工具超出其预期功能，机器人面临$\textit{开放世界可负担性接地}$的挑战：选择一个开放类别的物体作为工具，并定位其具体作用区域。为此，我们引入GROW$^2$（确定哪个物体和哪个部位），它利用物体部件作为自然抽象，将接地过程分层为语义和几何级别，从而绕过了数据密集型端到端训练的需求。在语义层面，GROW$^2$利用视觉语言模型（VLM）的常识推理来解析自然语言任务指令，选择合适的物体作为工具，并在工具和目标物体上识别任务相关部件。在几何层面，视觉基础模型随后从单个RGB-D图像中将所选部件接地到精确的3D区域。在已建立基准上的实验表明，GROW$^2$在可负担性预测基准上优于最先进的基线。此外，它在开放类别物体上实现了零样本泛化，并在模拟和真实机器人工具使用实验中均优于基线。
英文摘要：
Can the robot use a plate to cut a cake if no knife is available? Tool use greatly expands robot capabilities, but to use tools creatively beyond their intended functions, the robot faces the challenge of $\textit{open-world affordance grounding}$: select an open-category object to act as a tool and localize its specific region of action. To this end, we introduce GROW$^2$ (GROunding Which and Where), which leverages object parts as a natural abstraction to split the grounding process hierarchically into semantic and geometric levels, thus bypassing the need for data-heavy, end-to-end training. Semantically, GROW$^2$ harnesses the commonsense reasoning of Vision-Language Models (VLMs) to parse a natural-language task instruction, select a suitable object as the tool, and identify task-relevant parts on the tool and the target object. Geometrically, vision foundation models then ground the selected parts into precise 3D regions from a single RGB-D image. Experiments on established benchmarks show that GROW$^2$ outperforms state-of-the-art baselines on affordance prediction benchmarks. Further, it achieves zero-shot generalization over open-category objects and outperforms baselines in both simulated and real-world robot tool use experiments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
