JOintGS: 为野外单目重建联合优化相机、人体和3D高斯分布
JOintGS: Joint Optimization of Cameras, Bodies and 3D Gaussians for In-the-Wild Monocular Reconstruction
AI总结：
JOintGS通过联合优化相机、人体和3D高斯分布，提升野外单目重建的精度与鲁棒性。
摘要：
从单目RGB视频重建高保真的可动画3D人体肖像仍然具有挑战性，特别是在不受约束的野外场景中，其中相机参数和人体姿态从现成方法（如COLMAP、HMR2.0）获得的参数往往不准确。Splatting（3DGS）的进展展示了出色的渲染质量和实时性能，但它们严重依赖于精确的相机校准和姿态注释，限制了其在现实世界中的应用。我们提出了JOintGS，一个统一的框架，通过协同细化机制从粗略初始化中联合优化相机外参、人体姿态和3D高斯表示。我们的关键见解是显式前景-背景解耦能够相互促进：静态背景高斯分布通过多视图一致性锚定相机估计；经过细化的相机通过准确的时间对应性改善人体姿态对齐；优化的人体姿态通过去除静态约束中的动态伪影来提高场景重建。我们进一步引入了一个时间动态模块来捕捉细粒度的姿态依赖变形，并引入了一个残差颜色场来建模光照变化。在NeuMan和EMDB数据集上的广泛实验表明，JOintGS在NeuMan数据集上比最先进的方法提高了2.1~dB的PSNR，同时保持实时渲染。值得注意的是，我们的方法在噪声初始化下的鲁棒性显著优于基线方法。我们的源代码可在https://github.com/MiliLab/JOintGS上获得。
英文摘要：
Reconstructing high-fidelity animatable 3D human avatars from monocular RGB videos remains challenging, particularly in unconstrained in-the-wild scenarios where camera parameters and human poses from off-the-shelf methods (e.g., COLMAP, HMR2.0) are often inaccurate. Splatting (3DGS) advances demonstrate impressive rendering quality and real-time performance, they critically depend on precise camera calibration and pose annotations, limiting their applicability in real-world settings. We present JOintGS, a unified framework that jointly optimizes camera extrinsics, human poses, and 3D Gaussian representations from coarse initialization through a synergistic refinement mechanism. Our key insight is that explicit foreground-background disentanglement enables mutual reinforcement: static background Gaussians anchor camera estimation via multi-view consistency; refined cameras improve human body alignment through accurate temporal correspondence; optimized human poses enhance scene reconstruction by removing dynamic artifacts from static constraints. We further introduce a temporal dynamics module to capture fine-grained pose-dependent deformations and a residual color field to model illumination variations. Extensive experiments on NeuMan and EMDB datasets demonstrate that JOintGS achieves superior reconstruction quality, with 2.1~dB PSNR improvement over state-of-the-art methods on NeuMan dataset, while maintaining real-time rendering. Notably, our method shows significantly enhanced robustness to noisy initialization compared to the baseline.Our source code is available at https://github.com/MiliLab/JOintGS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
