ComplexMimic: Human-Scene Interaction Imitation in Complex 3D Environments
College of Computer Science and Technology, Jilin University, China(吉林大学计算机科学与技术学院);Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, China(教育部符号计算与知识工程重点实验室)
Physics-based Human-Scene Interaction (HSI) imitation learning is crucial for embodied intelligence as it bridges the gap between kinematic 3D motions and real-world dynamics. However, most existing methods focus on simplified scene settings, leaving complex environments largely unexplored, which limits their applicability in real-world scenarios. In this paper, we focus on HSI mimicry in complex environments. Under this complex setting, we observe an inherent trade-off between successfully performing interaction and maintaining natural, physically plausible motions. To address this challenge, we propose ComplexMimic, a framework that reconstructs diverse HSI by interpreting imperfect MoCap data. First, we introduce a Dual Flow Strategy, which learns two complementary experts: an imitation expert for accurate motion tracking and an interaction expert for collision-aware adaptation in complex scenes. Second, naive multi-expert distillation, which treats all experts equally, often under-samples challenging behaviors, limiting effective learning. To mitigate this issue, we propose a difficulty-aware distillation strategy that adaptively weights supervision and prioritizes hard-yet-learnable trajectories guided by failure statistics and learning progress signals. Extensive experiments on three benchmark datasets demonstrate that our approach outperforms current state-of-the-art methods.
AI2-THOR: An Interactive 3D Environment for Visual AI
Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学);Stanford University(斯坦福大学);Carnegie Mellon University(卡内基梅隆大学)
我们介绍了交互之家(THE HOUSE OF INTERACTIONS,THOR),这是一个用于视觉AI研究的框架,可在http://ai2thor.allenai.org上获得。AI 2-THOR由近乎照片般逼真的3D室内场景组成,其中人工智能智能体可以在场景中导航并与对象交互以执行任务。AI 2-THOR支持许多不同领域的研究,包括但不限于深度强化学习、模仿学习、交互学习、规划、视觉问答、无监督表示学习、目标检测和分割以及认知学习模型。AI 2-THOR的目标是促进视觉智能模型的构建,并推动该领域的研究。