发表机构
The Hong Kong University of Science and Technology (Guangzhou); The University of Hong Kong; Tsinghua University（香港科技大学（广州）; 香港大学; 清华大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出Project Kitchen游戏化数据采集平台和Game2Policy方法，通过VR游戏获取多样化操作数据并迁移至真实机器人，在少样本下显著提升策略成功率。
AI 中文摘要
学习可泛化的机器人操作策略需要大规模且多样化的交互数据，然而采集真实世界演示数据仍然成本高昂且难以扩展。现有的数据采集方法要么依赖于特定机器人硬件，限制了众包和可迁移性，要么存在标注不完整和行为多样性有限的问题。受游戏能够维持人类长期参与的启发，我们探索了一种替代范式，将数据采集转变为引人入胜的游戏体验，并将由此产生的人类操作经验迁移到真实机器人上。我们提出了Project Kitchen，一个基于VR的游戏化第一人称视角数据采集平台，能够引发多样化、目标导向的操作行为，同时不依赖于特定的机器人本体和硬件，使其适用于更广泛且可能大规模部署的场景。为弥合游戏与现实之间的差距，我们进一步引入了Game2Policy，该方法从游戏轨迹中提取与本体无关的可供性线索，包括接触点和子目标状态。一个可供性模型在游戏采集的数据上进行预训练，然后仅使用少量真实机器人演示与下游策略进行联合微调。实验表明，在少样本设置下，Game2Policy在仿真中将平均成功率提高了10.0个百分点，在真实机器人上提高了18.3个百分点。用户研究和定量分析进一步表明，Project Kitchen促进了多样化的操作行为，并提供了引人入胜的数据采集体验。这些结果证明了游戏化虚拟环境作为可扩展操作知识来源的潜力。平台和代码将在论文被接收后发布。
英文摘要
Learning generalizable robot manipulation policies requires large-scale and diverse interaction data, yet collecting real-world demonstrations remains costly and difficult to scale. Existing approaches to data collection are either dependent on specific robot hardware that limits crowdsourcing and transferability, or suffer from incomplete annotation and limited behavioral diversity. Inspired by how games sustain long-term human engagement, we explore an alternative paradigm that turns data collection into an engaging gameplay experience and transfers the resulting human manipulation experience to real robots. We present Project Kitchen, a VR-based gamified egocentric data collection platform that elicits diverse, goal-directed manipulation while remaining independent of specific robot embodiments and hardware, making it applicable to broader and potentially large-scale deployment. To bridge the game-to-real gap, we further introduce Game2Policy, which extracts embodiment-invariant affordance cues, including contact points and sub-goal states, from gameplay trajectories. An affordance model is pre-trained on game-collected data and then jointly fine-tuned with downstream policies using only a handful of real-robot demonstrations. Experiments show that Game2Policy improves average success rates by 10.0 points in simulation and 18.3 points on real robots in the few-shot setting. User studies and quantitative analyses further show that Project Kitchen promotes diverse manipulation behaviors and provides an engaging data collection experience. These results demonstrate the potential of gamified virtual environments as a scalable source of manipulation knowledge. The platform and code will be released upon acceptance.
Comments9 pages, 6 figures
