Crowd4D：场景感知单目4D人群重建
Crowd4D: Scene-Aware Monocular 4D Crowd Reconstruction
浏览论文内容
中文总结 AI 辅助
针对单目视频恢复大规模场景下4D人群运动的难题，提出Crowd4D框架，通过多阶段优化联合优化人群与场景，引入HSIP解决对齐问题、CSCR提升时间稳定性，实验证明其性能优于现有方法，可实现复杂场景下单目4D人群稳健重建。
中文摘要 AI 辅助
从单目视频中恢复大规模场景下与场景一致的4D人群运动具有挑战性，现有方法依赖单平面假设存在问题。我们提出Crowd4D，首个场景感知4D人群重建框架，通过多阶段优化联合优化人群和场景。引入人类-场景交互代理HSIP解决对齐瓶颈，还引入人群结构一致性正则化CSCR提升遮挡下的时间稳定性。实验表明Crowd4D优于现有方法，能在复杂大规模真实场景中进行稳健的单目4D人群重建。
英文摘要
Recovering scene-consistent 4D crowd motion from monocular video in large-scale scenes remains challenging due to severe depth ambiguity and complex scene geometry. Existing monocular crowd reconstruction methods typically rely on single-plane assumptions, leading to unreliable metric scale and spatial drift under complex terrain. We propose Crowd4D, the first scene-aware 4D crowd reconstruction framework that jointly optimizes the crowd and scene from a monocular RGB video in large-scale scenes. Crowd4D explicitly incorporates scene geometry and ensures consistency across image and scene spaces via a multi-stage optimization strategy. A key bottleneck of this task lies in accurate human-scene alignment, particularly in scale and position. However, human and scene reconstructions are typically decoupled. To address this, we introduce the Human-Scene Interaction Proxy, abbreviated as HSIP, as an intermediate representation derived from Scene Interaction Point Clouds and a Scene Interaction Surface, abbreviated as SIPC and SIS. These representations encode explicit scene-aware geometric priors and redefine the optimization space for large-scale monocular 4D crowd reconstruction. To further improve temporal stability under occlusions, we introduce Crowd Structural Coherence Regularization, abbreviated as CSCR, which leverages HSIP-based spatial priors to impose soft temporal consistency on pairwise relative displacements and directions within local crowd neighborhoods. Extensive experiments demonstrate that Crowd4D consistently outperforms existing state-of-the-art methods and enables robust monocular 4D crowd reconstruction in complex, large-scale real-world scenes.
