arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.02034cs.CV
收藏

ComplexMimic: 复杂3D环境中的人-场景交互模仿

ComplexMimic: Human-Scene Interaction Imitation in Complex 3D Environments

College of Computer Science and Technology, Jilin University, China(吉林大学计算机科学与技术学院);Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, China(教育部符号计算与知识工程重点实验室)

Lu Pan, Hongwei Zhao

AI总结:

针对复杂环境中物理人-场景交互模仿的挑战,提出ComplexMimic框架,通过双流策略和难度感知蒸馏,在保持自然运动的同时实现准确交互,在三个基准数据集上超越现有方法。

摘要:

基于物理的人-场景交互(HSI)模仿学习对于具身智能至关重要,因为它弥合了运动学3D运动与现实世界动力学之间的差距。然而,大多数现有方法侧重于简化的场景设置,复杂环境在很大程度上未被探索,这限制了它们在现实场景中的适用性。在本文中,我们关注复杂环境中的HSI模仿。在这种复杂设置下,我们观察到成功执行交互与保持自然、物理上合理的运动之间存在固有的权衡。为了解决这一挑战,我们提出了ComplexMimic,一个通过解释不完美的MoCap数据来重建多样化HSI的框架。首先,我们引入了双流策略,学习两个互补的专家:一个用于精确运动跟踪的模仿专家,以及一个用于复杂场景中碰撞感知适应的交互专家。其次,将所有专家平等对待的朴素多专家蒸馏通常会欠采样具有挑战性的行为,限制了有效学习。为了缓解这个问题,我们提出了一种难度感知蒸馏策略,该策略自适应地加权监督,并根据失败统计和学习进度信号优先处理困难但可学习的轨迹。在三个基准数据集上的大量实验表明,我们的方法优于当前最先进的方法。我们的实现可在以下网址获取:https://complexmimic.github.io。

英文摘要:

Physics-based Human-Scene Interaction (HSI) imitation learning is crucial for embodied intelligence as it bridges the gap between kinematic 3D motions and real-world dynamics. However, most existing methods focus on simplified scene settings, leaving complex environments largely unexplored, which limits their applicability in real-world scenarios. In this paper, we focus on HSI mimicry in complex environments. Under this complex setting, we observe an inherent trade-off between successfully performing interaction and maintaining natural, physically plausible motions. To address this challenge, we propose ComplexMimic, a framework that reconstructs diverse HSI by interpreting imperfect MoCap data. First, we introduce a Dual Flow Strategy, which learns two complementary experts: an imitation expert for accurate motion tracking and an interaction expert for collision-aware adaptation in complex scenes. Second, naive multi-expert distillation, which treats all experts equally, often under-samples challenging behaviors, limiting effective learning. To mitigate this issue, we propose a difficulty-aware distillation strategy that adaptively weights supervision and prioritizes hard-yet-learnable trajectories guided by failure statistics and learning progress signals. Extensive experiments on three benchmark datasets demonstrate that our approach outperforms current state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑