OmniDexGrasp：通过基础模型与力反馈实现通用灵巧抓取
OmniDexGrasp: Generalizable Dexterous Grasping via Foundation Model and Force Feedback
- School of Computer Science and Engineering, Sun Yat-sen University, China（计算机科学与工程学院，中山大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对现有灵巧抓取泛化性差的问题，提出OmniDexGrasp框架，结合基础模型生成人类抓取图像、迁移策略及力反馈自适应抓取，实现多样用户提示、灵巧手和任务的通用抓取，并在仿真和真实机器人上验证了有效性。
AI中文摘要：
使机器人能够基于人类指令灵巧地抓取和操作物体是机器人学中一个很有前景的方向。然而，由于语义灵巧抓取数据集的规模有限，现有方法难以在不同物体或任务间泛化。基础模型提供了一种增强泛化能力的新途径，但直接利用它们生成可行的机器人动作仍然具有挑战性，因为抽象模型知识与物理机器人执行之间存在差距。为解决这些挑战，我们提出了OmniDexGrasp，一个通过将基础模型与迁移和控制策略相结合，在用户提示、灵巧具身和抓取任务中实现全能力的通用框架。OmniDexGrasp集成了三个关键模块：（i）使用基础模型通过生成支持用户提示和任务全能力的人类抓取图像来增强泛化能力；（ii）一种从人类图像到机器人动作的迁移策略，将人类演示转换为可执行的机器人动作，实现全灵巧具身；（iii）力感知自适应抓取策略，确保稳健且稳定的抓取执行。在仿真和真实机器人上的实验验证了OmniDexGrasp在多样用户提示、抓取任务和灵巧手上的有效性，进一步的结果显示了其对灵巧操作任务的可扩展性。
英文摘要：
Enabling robots to dexterously grasp and manipulate objects based on human commands is a promising direction in robotics. However, existing approaches are challenging to generalize across diverse objects or tasks due to the limited scale of semantic dexterous grasp datasets. Foundation models offer a new way to enhance generalization, yet directly leveraging them to generate feasible robotic actions remains challenging due to the gap between abstract model knowledge and physical robot execution. To address these challenges, we propose OmniDexGrasp, a generalizable framework that achieves omni-capabilities in user prompting, dexterous embodiment, and grasping tasks by combining foundation models with the transfer and control strategies. OmniDexGrasp integrates three key modules: (i) foundation models are used to enhance generalization by generating human grasp images supporting omni-capability of user prompt and task; (ii) a human-image-to-robot-action transfer strategy converts human demonstrations into executable robot actions, enabling omni dexterous embodiment; (iii) force-aware adaptive grasp strategy ensures robust and stable grasp execution. Experiments in simulation and on real robots validate the effectiveness of OmniDexGrasp on diverse user prompts, grasp task and dexterous hands, and further results show its extensibility to dexterous manipulation tasks.
