arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.07832cs.RO
收藏

DeliGrasp:利用 LLM 推断物体属性以实现自适应抓取策略

DeliGrasp: Inferring Object Properties with LLMs for Adaptive Grasp Policies

University of Colorado at Boulder(科罗拉多大学博尔德分校)

William Xie, Maria Valentini, Jensen Lavering, Nikolaus Correll

AI总结:

本文提出 DeliGrasp,利用 LLM 从语义描述推断质量、摩擦和刚度参数,并生成第一性原理自适应抓取策略,在 12 种易碎可变形物品上优于传统与直接 LLM 策略。

摘要:

大型语言模型(LLM)能够为世界上大多数物体提供丰富的物理描述,从而使机器人能够实现更知情、更强的抓取能力。我们利用 LLM 的常识物理推理和代码编写能力,从语义描述中推断物体的物理特征——质量 m、摩擦系数 μ 和弹簧常数 k——随后将这些特征转化为可执行的自适应抓取策略。我们使用内置深度相机、可通过限制电机电流来控制扭矩的两指夹爪,证明了由 LLM 参数化但基于第一性原理的抓取策略,在包含 12 种易碎和可变形物品(包括食品、农产品、玩具和其他日常用品)的自定义基准上,优于传统自适应抓取策略和直接的 LLM-as-code 策略;这些物品的质量和所需拾取力跨越两个数量级。随后,我们通过在基于属性的比较上进行模型微调,并借助思维链提示引出此类比较,改进了对尺寸可变物体的属性估计和抓取性能。我们还展示了 DeliGrasp 策略提供的柔顺反馈如何辅助下游任务,例如测量农产品成熟度。我们的代码和视频可在 https://deligrasp.github.io 获取。

英文摘要:

Large language models (LLMs) can provide rich physical descriptions of most worldly objects, allowing robots to achieve more informed and capable grasping. We leverage LLMs' common sense physical reasoning and code-writing abilities to infer an object's physical characteristics$\unicode{x2013}$mass $m$, friction coefficient $μ$, and spring constant $k$$\unicode{x2013}$from a semantic description, and then translate those characteristics into an executable adaptive grasp policy. Using a two-finger gripper with a built-in depth camera that can control its torque by limiting motor current, we demonstrate that LLM-parameterized but first-principles grasp policies outperform both traditional adaptive grasp policies and direct LLM-as-code policies on a custom benchmark of 12 delicate and deformable items including food, produce, toys, and other everyday items, spanning two orders of magnitude in mass and required pick-up force. We then improve property estimation and grasp performance on variable size objects with model finetuning on property-based comparisons and eliciting such comparisons via chain-of-thought prompting. We also demonstrate how compliance feedback from DeliGrasp policies can aid in downstream tasks such as measuring produce ripeness. Our code and videos are available at: https://deligrasp.github.io

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑