arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.01082cs.CVcs.GR
收藏

ReMoGen:通过多样化数据的模块学习实现实时的人际交互生成

ReMoGen: Real-time Human Interaction-to-Reaction Generation via Modular Learning from Diverse Data

ShanghaiTech University(上海科技大学);Guangzhou Institute of Energy Conversion, CAS(中国科学院广州能源研究所);University of Science and Technology of China(中国科学技术大学);The Chinese University of Hong Kong(香港中文大学);Nanyang Technological University(南洋理工大学)

Yaoqin Ye, Yiteng Xu, Qin Sun, Xinge Zhu, Yujing Sun, Yuexin Ma

AI总结:

本文提出ReMoGen框架,通过模块化学习生成实时的人际交互反应,利用大规模单人动作数据集学习通用动作先验,并通过Meta-Interaction模块适应不同交互领域,实现高效且高质量的动作响应。

摘要:

现实环境中的人类行为本质上是交互性的,个体的动作受到周围代理和场景的影响。这种能力对于虚拟化身、交互动画和人机协作等应用至关重要。本文针对实时的人际交互生成任务,从动态多源线索生成主体的未来动作,包括他人的动作、场景几何和可选的高层语义输入。该任务本质上具有挑战性,因为(i)交互数据有限且碎片化,分布在异构的单人、人-人和人-场景领域,以及(ii)需要在连续在线交互中产生低延迟且高保真的动作响应。为了解决这些挑战,我们提出了ReMoGen(反应动作生成),一种用于实时交互生成的模块化学习框架。ReMoGen利用从大规模单人动作数据集中学习的通用动作先验,并通过独立训练的Meta-Interaction模块适应目标交互领域,从而在数据稀缺和异构监督下实现稳健的泛化。为了支持响应式的在线交互,ReMoGen在段级生成的同时,结合一个轻量级的帧级段细化模块,该模块在帧级别整合新观察到的线索,提高了响应性和时间一致性,而无需昂贵的全序列推断。在人类-人类、人类-场景和混合模态交互设置中进行了广泛的实验,结果表明ReMoGen能够产生高质量、一致且响应迅速的反应,同时在多样化的交互场景中实现有效的泛化。

英文摘要:

Human behaviors in real-world environments are inherently interactive, with an individual's motion shaped by surrounding agents and the scene. Such capabilities are essential for applications in virtual avatars, interactive animation, and human-robot collaboration. We target real-time human interaction-to-reaction generation, which generates the ego's future motion from dynamic multi-source cues, including others' actions, scene geometry, and optional high-level semantic inputs. This task is fundamentally challenging due to (i) limited and fragmented interaction data distributed across heterogeneous single-person, human-human, and human-scene domains, and (ii) the need to produce low-latency yet high-fidelity motion responses during continuous online interaction. To address these challenges, we propose ReMoGen (Reaction Motion Generation), a modular learning framework for real-time interaction-to-reaction generation. ReMoGen leverages a universal motion prior learned from large-scale single-person motion datasets and adapts it to target interaction domains through independently trained Meta-Interaction modules, enabling robust generalization under data-scarce and heterogeneous supervision. To support responsive online interaction, ReMoGen performs segment-level generation together with a lightweight Frame-wise Segment Refinement module that incorporates newly observed cues at the frame level, improving both responsiveness and temporal coherence without expensive full-sequence inference. Extensive experiments across human-human, human-scene, and mixed-modality interaction settings show that ReMoGen produces high-quality, coherent, and responsive reactions, while generalizing effectively across diverse interaction scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑