arXiv 2507.068222025-07-09cs.RO

多指手关节工具操作的分层强化学习

Hierarchical Reinforcement Learning for Articulated Tool Manipulation with Multifingered Hand

State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University（机械系统与振动国家重点实验室，机械工程学院，上海交通大学）；Shanghai Key Laboratory of Intelligent Robotics, Meta Robotics Institute, Shanghai Jiao Tong University（智能机器人上海重点实验室，元机器人研究院，上海交通大学）

Wei Xu, Yanchao Zhao, Weichao Guo, Xinjun Sheng

备注：: Accepted by 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025). copyright 2025 IEEE. Final version to appear in IEEE Xplore

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出分层目标条件强化学习框架，结合低级灵巧手策略与高级手臂策略，利用点云编码器估计工具可供性，实现多指手对镊子类关节工具的精确操纵，真实实验中抓取成功率达70.8%。

摘要：

在以往的研究中，很少探索使用镊子或剪刀等关节工具的操纵。与刚性工具不同，关节工具会动态改变其形状，这给灵巧机器人手带来了独特的挑战。在这项工作中，我们提出了一种分层、目标条件的强化学习（GCRL）框架，以提高拟人机器人手使用关节工具的操纵能力。我们的框架包括两个策略层：（1）一个低级策略，使灵巧手能够针对不同尺寸的物体将工具操纵到各种配置；（2）一个高级策略，定义工具的目标状态并控制机器人手臂进行物体拾取任务。我们采用一个在合成点云上训练的编码器，从输入点云估计工具的可供性状态——具体来说，不同的工具配置（例如镊子张开角度）如何使抓取不同尺寸的物体成为可能——从而实现精确的工具操纵。我们还利用一个特权信息启发式策略来生成回放缓冲区，提高高级策略的训练效率。我们通过真实世界实验验证了我们的方法，表明机器人能够有效操纵类似镊子的工具，以70.8%的成功率抓取各种形状和尺寸的物体。这项研究突显了强化学习在推进灵巧机器人操纵关节工具方面的潜力。

英文摘要：

Manipulating articulated tools, such as tweezers or scissors, has rarely been explored in previous research. Unlike rigid tools, articulated tools change their shape dynamically, creating unique challenges for dexterous robotic hands. In this work, we present a hierarchical, goal-conditioned reinforcement learning (GCRL) framework to improve the manipulation capabilities of anthropomorphic robotic hands using articulated tools. Our framework comprises two policy layers: (1) a low-level policy that enables the dexterous hand to manipulate the tool into various configurations for objects of different sizes, and (2) a high-level policy that defines the tool's goal state and controls the robotic arm for object-picking tasks. We employ an encoder, trained on synthetic pointclouds, to estimate the tool's affordance states--specifically, how different tool configurations (e.g., tweezer opening angles) enable grasping of objects of varying sizes--from input point clouds, thereby enabling precise tool manipulation. We also utilize a privilege-informed heuristic policy to generate replay buffer, improving the training efficiency of the high-level policy. We validate our approach through real-world experiments, showing that the robot can effectively manipulate a tweezer-like tool to grasp objects of diverse shapes and sizes with a 70.8 % success rate. This study highlights the potential of RL to advance dexterous robotic manipulation of articulated tools.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。