CoInteract：通过空间结构化共生成实现物理一致的人-物体交互视频合成
CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generation
AI总结：
CoInteract通过空间结构化共生成方法，在人-物体交互视频合成中提升结构稳定性与物理合理性，采用扩散变换器框架结合人类感知混合专家和双流训练策略，实现更真实的交互效果。
摘要：
合成人-物体交互（HOI）视频在电子商务、数字广告和虚拟营销中有广泛应用。然而，尽管扩散模型具有逼真渲染能力，但仍频繁在敏感区域（如手和脸）的结构稳定性以及物理合理的接触（如避免手-物体穿透）上失败。我们提出CoInteract，一种端到端的HOI视频合成框架，基于人物参考图像、产品参考图像、文本提示和语音音频进行条件生成。CoInteract在扩散变换器（DiT）主干中引入两种互补设计。首先，我们提出人类感知混合专家（MoE），通过空间监督路由将令牌路由到轻量级、区域专用的专家，从而在参数开销极小的情况下提高细粒度结构保真度。其次，我们提出空间结构化共生成，一种双流训练范式，联合建模RGB外观流和辅助HOI结构流，以注入交互几何先验。在训练过程中，HOI流关注RGB令牌，并通过其监督正则化共享主干权重；在推理过程中，HOI分支被移除以实现零开销RGB生成。实验结果表明，CoInteract在结构稳定性、逻辑一致性和交互真实性方面显著优于现有方法。
英文摘要：
Synthesizing human--object interaction (HOI) videos has broad practical value in e-commerce, digital advertising, and virtual marketing. However, current diffusion models, despite their photorealistic rendering capability, still frequently fail on (i) the structural stability of sensitive regions such as hands and faces and (ii) physically plausible contact (e.g., avoiding hand--object interpenetration). We present CoInteract, an end-to-end framework for HOI video synthesis conditioned on a person reference image, a product reference image, text prompts, and speech audio. CoInteract introduces two complementary designs embedded into a Diffusion Transformer (DiT) backbone. First, we propose a Human-Aware Mixture-of-Experts (MoE) that routes tokens to lightweight, region-specialized experts via spatially supervised routing, improving fine-grained structural fidelity with minimal parameter overhead. Second, we propose Spatially-Structured Co-Generation, a dual-stream training paradigm that jointly models an RGB appearance stream and an auxiliary HOI structure stream to inject interaction geometry priors. During training, the HOI stream attends to RGB tokens and its supervision regularizes shared backbone weights; at inference, the HOI branch is removed for zero-overhead RGB generation. Experimental results demonstrate that CoInteract significantly outperforms existing methods in structural stability, logical consistency, and interaction realism.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
