ODHSR：基于单目视频的人体与场景在线密集3D重建
ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos
AI总结：
针对单目野外视频人体-场景重建需预校准相机、长训练时间的问题，提出ODHSR在线统一框架，结合3D高斯溅射、重建基跟踪与姿态估计、变形模块及遮挡感知等技术，在EMDB等数据集上实现多任务性能提升。
摘要：
从单目野外视频中创建逼真的场景和人体重建在以人为中心的3D世界感知中占据重要地位。近期神经渲染进展已实现整体人体-场景重建，但需预校准相机和人体姿态，且训练时间长达数天。本研究提出一种新型统一框架，可在线同时执行相机跟踪、人体姿态估计与人体-场景重建。利用3D高斯溅射（3D Gaussian Splatting）高效学习人体和场景的高斯基元，设计基于重建的相机跟踪和人体姿态估计模块，以实现整体理解及姿态与外观的有效解耦。具体而言，设计人体变形模块以忠实重建细节并增强对分布外姿态的泛化能力；为准确学习人体与场景的空间相关性，引入遮挡感知人体轮廓渲染和单目几何先验，进一步提升重建质量。在EMDB和NeuMan数据集上的实验表明，该方法在相机跟踪、人体姿态估计、新视角合成和运行时方面的性能优于或与现有方法相当。项目页面为https://eth-ait.github.io/ODHSR。
英文摘要：
Creating a photorealistic scene and human reconstruction from a single monocular in-the-wild video figures prominently in the perception of a human-centric 3D world. Recent neural rendering advances have enabled holistic human-scene reconstruction but require pre-calibrated camera and human poses, and days of training time. In this work, we introduce a novel unified framework that simultaneously performs camera tracking, human pose estimation and human-scene reconstruction in an online fashion. 3D Gaussian Splatting is utilized to learn Gaussian primitives for humans and scenes efficiently, and reconstruction-based camera tracking and human pose estimation modules are designed to enable holistic understanding and effective disentanglement of pose and appearance. Specifically, we design a human deformation module to reconstruct the details and enhance generalizability to out-of-distribution poses faithfully. Aiming to learn the spatial correlation between human and scene accurately, we introduce occlusion-aware human silhouette rendering and monocular geometric priors, which further improve reconstruction quality. Experiments on the EMDB and NeuMan datasets demonstrate superior or on-par performance with existing methods in camera tracking, human pose estimation, novel view synthesis and runtime. Our project page is at https://eth-ait.github.io/ODHSR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
