GLOVER++:释放从人类行为中学习可供性的潜力以用于机器人操作
GLOVER++: Unleashing the Potential of Affordance Learning from Human Behaviors for Robotic Manipulation
AI总结:
该研究构建含50万图像的HOVA-500K可供性数据集与基准,并提出全局到局部的GLOVER++框架,实现人类演示知识向开放词汇推理和机器人操作任务的有效迁移。
摘要:
从人类演示视频中学习操作技能,为可泛化且可解释的机器人智能提供了一条颇具前景的路径,尤其是通过可行动可供性(actionable affordances)这一视角。然而,迁移此类知识仍具挑战,原因在于:1)缺乏带有精确可供性标注的大规模数据集;2)对多样化操作情境中的可供性探索不足。为弥补这些空白,我们推出 HOVA-500K,这是一个大规模、带可供性标注的数据集,包含覆盖 1,726 个物体类别和 675 种动作的 500,000 张图像。我们还发布了用于多模态可供性推理的标准化基准套件。基于 HOVA-500K,我们提出 GLOVER++,这是一个从全局到局部的可供性训练框架,可有效地将来自人类演示的可行动可供性知识迁移到下游开放词汇推理任务。GLOVER++ 在 HOVA-500K 基准上取得了最先进结果,并在多样化的下游机器人操作任务中展现出强大泛化能力。通过显式建模可行动可供性,GLOVER++ 促进了跨场景、跨模态和跨任务的稳健迁移。我们希望 HOVA-500K 与 GLOVER++ 框架能够成为弥合人类演示与机器人操作能力之间差距的宝贵资源。
英文摘要:
Learning manipulation skills from human demonstration videos offers a promising path toward generalizable and interpretable robotic intelligence-particularly through the lens of actionable affordances. However, transferring such knowledge remains challenging due to: 1) a lack of large-scale datasets with precise affordance annotations, and 2) insufficient exploration of affordances in diverse manipulation contexts. To address these gaps, we introduce HOVA-500K, a large-scale, affordance-annotated dataset comprising 500,000 images across 1,726 object categories and 675 actions. We also release a standardized benchmarking suite for multi-modal affordance reasoning. Built upon HOVA-500K, we present GLOVER++, a global-to-local affordance training framework that effectively transfers actionable affordance knowledge from human demonstrations to downstream open-vocabulary reasoning tasks. GLOVER++ achieves state-of-the-art results on the HOVA-500K benchmark and demonstrates strong generalization across diverse downstream robotic manipulation tasks. By explicitly modeling actionable affordances, GLOVER++ facilitates robust transfer across scenes, modalities, and tasks. We hope that HOVA-500K and the GLOVER++ framework will serve as valuable resources for bridging the gap between human demonstrations and robotic manipulation capabilities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。