arXiv 2411.062322024-11cs.CV

RCR：基于直立空间的大场景单图鲁棒人群重建

RCR: Robust Crowd Reconstruction with Upright Space from a Single Large-scene Image

College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Beijing National Research Center for Information Science and Technology, Tsinghua University（清华大学北京信息科学与技术国家研究中心）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）

Jing Huang, Hao Wen, Tianyi Zhou, Haozhe Lin, Yu-kun Lai, Kun Li

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大场景单图人群重建中尺度差异、深度模糊与透视畸变导致的全局不一致问题，提出HVIP概念与RCR方法，结合迭代裁剪、直立空间归一化，无需测试时优化即可实现跨视场角的全局一致重建，并贡献两个基准数据集。

摘要：

本文聚焦于在任意相机视场角（FoV）下，从包含不同人体尺度的大场景单张图像中，实现空间一致性的数百人姿态与形状重建。由于二维人体尺度小且变化剧烈、深度模糊以及透视畸变等问题，现有方法均无法实现具有正确重投影的全局一致性重建。为应对这些挑战，我们首先提出一个新概念——人景虚拟交互点（HVIP），将复杂的三维人体定位转化为二维像素定位。随后，我们将其扩展为RCR（鲁棒人群重建），该方法无需测试时优化即可在不同相机视场角下实现全局一致性重建和稳定泛化。为感知不同像素大小的人体，我们提出迭代地面感知裁剪方法，自动裁剪图像并合并结果。为消除重建过程中相机与裁剪过程的影响，我们引入规范直立三维空间及对应的直立二维空间。为连接规范空间与相机空间，我们提出直立归一化，将局部裁剪输入变换至直立二维空间，并将输出从直立三维空间变换至统一相机空间。此外，我们贡献了两个基准数据集LargeCrowd和SynCrowd，用于评估大场景下的人群重建。实验结果表明了所提方法的有效性。源代码和数据将公开供研究使用。

英文摘要：

This paper focuses on spatially consistent hundreds of human pose and shape reconstruction from a single large-scene image with various human scales under arbitrary camera FoVs (Fields of View). Due to the small and highly varying 2D human scales, depth ambiguity, and perspective distortion, no existing methods can achieve globally consistent reconstruction with correct reprojection. To address these challenges, we first propose a new concept, Human-scene Virtual Interaction Point (HVIP), to convert the complex 3D human localization into 2D-pixel localization. We then extend it to RCR (Robust Crowd Reconstruction), which achieves globally consistent reconstruction and stable generalization on different camera FoVs without test-time optimization. To perceive humans in varying pixel sizes, we propose an Iterative Ground-aware Cropping to automatically crop the image and then merge the results. To eliminate the influence of the camera and cropping process during the reconstruction, we introduce a canonical Upright 3D Space and the corresponding Upright 2D Space. To link the canonical space and the camera space, we propose the Upright Normalization, which transforms the local crop input into the Upright 2D Space, and transforms the output from the Upright 3D Space into the unified camera space. Besides, we contribute two benchmark datasets, LargeCrowd and SynCrowd, for evaluating crowd reconstruction in large scenes. Experimental results demonstrate the effectiveness of the proposed method. The source code and data will be publicly available for research purposes.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。