FSAG: 通过扩散模型增强人对灵巧手手指特定 affordance 的 grounding
FSAG: Enhancing Human-to-Dexterous-Hand Finger-Specific Affordance Grounding via Diffusion Models
AI总结：
本文提出FSAG框架，通过扩散模型提取语义affordance，实现稳定多接触抓取，无需硬件特定数据集，且单模态深度信息即可实现高性能抓取合成。
摘要：
灵巧抓取合成必须同时满足功能意图和物理可行性，但现有流程往往将语义 grounding 与细化分离，导致在物体和姿态变化下产生不稳定或非功能性的接触。这一挑战被高维性和多指手的运动学多样性加剧，使得许多方法依赖于大量硬件特定的抓取数据集，这些数据集是在模拟中收集或通过昂贵的现实世界试验获得的。我们提出了一种数据高效的框架，通过利用预训练生成扩散模型中的对象中心语义先验，绕过了机器人抓取数据的收集。从原始人类视频演示中提取时间对齐且细粒度的抓取 affordance，并与深度图像中的3D场景几何融合，以推断语义 grounded 的接触目标。我们进一步将这些 affordance 区域纳入抓取细化目标中，明确指导每个指尖在优化过程中朝向其预测的区域。所得到的系统能够在常见物体和工具上产生稳定、符合人类直觉的多接触抓取，同时在类别中未见过的物体实例、姿态变化和多个手 embodiment 中表现出强大的泛化能力。本工作（i）引入了一种利用视觉-语言生成先验的语义 affordance 提取流程，用于灵巧抓取；（ii）展示了跨手泛化，无需构建硬件特定的抓取数据集；（iii）证明当与基础模型语义结合时，单一深度模态足以实现高性能抓取合成。我们的结果展示了通过人类演示和预训练生成模型驱动的可扩展、硬件无关的灵巧操作路径。
英文摘要：
Dexterous grasp synthesis must jointly satisfy functional intent and physical feasibility, yet existing pipelines often decouple semantic grounding from refinement, yielding unstable or non-functional contacts under object and pose variations. This challenge is exacerbated by the high dimensionality and kinematic diversity of multi-fingered hands, which makes many methods rely on large, hardware-specific grasp datasets collected in simulation or through costly real-world trials. We propose a data-efficient framework that bypasses robot grasp data collection by exploiting object-centric semantic priors in pretrained generative diffusion models. Temporally aligned and fine-grained grasp affordances are extracted from raw human video demonstrations and fused with 3D scene geometry from depth images to infer semantically grounded contact targets. We further incorporate these affordance regions into the grasp refinement objective, explicitly guiding each fingertip toward its predicted region during optimization. The resulting system produces stable, human-intuitive multi-contact grasps across common objects and tools, while exhibiting strong generalization to previously unseen object instances within a category, pose variations, and multiple hand embodiments.This work (i) introduces a semantic affordance extraction pipeline leveraging vision--language generative priors for dexterous grasping, (ii) demonstrates cross-hand generalization without constructing hardware-specific grasp datasets, and (iii) establishes that a single depth modality suffices for high-performance grasp synthesis when coupled with foundation-model semantics. Our results highlight a path toward scalable, hardware-agnostic dexterous manipulation driven by human demonstrations and pretrained generative models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
