Multiview detection uses multiple calibrated cameras with overlapping fields of views to locate occluded pedestrians. In this field, existing methods typically adopt a ``human modeling - aggregation'' strategy. To find robust pedestrian representations, some intuitively incorporate 2D perception results from each frame, while others use entire frame features projected to the ground plane. However, the former does not consider the human appearance and leads to many ambiguities, and the latter suffers from projection errors due to the lack of accurate height of the human torso and head. In this paper, we propose a new pedestrian representation scheme based on human point clouds modeling. Specifically, using ray tracing for holistic human depth estimation, we model pedestrians as upright, thin cardboard point clouds on the ground. Then, we aggregate the point clouds of the pedestrian cardboard across multiple views for a final decision. Compared with existing representations, the proposed method explicitly leverages human appearance and reduces projection errors significantly by relatively accurate height estimation. On four standard evaluation benchmarks, the proposed method achieves very competitive results. Our code and data will be released at https://github.com/ZichengDuan/MvCHM.
SRI International（SRI国际）；Polytechnic Institute – Rio de Janeiro State University（里约热内卢州立大学理工学院）；Brown University（布朗大学）
The three-dimensional reconstruction of scenes from multiple views has made impressive strides in recent years, chiefly by methods correlating isolated feature points, intensities, or curvilinear structure. In the general setting, i.e., without requiring controlled acquisition, limited number of objects, abundant patterns on objects, or object curves to follow particular models, the majority of these methods produce unorganized point clouds, meshes, or voxel representations of the reconstructed scene, with some exceptions producing 3D drawings as networks of curves. Many applications, e.g., ro…
Large Material Gaussian Model for Relightable 3D Generation
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；The University of Hong Kong（香港大学）；LIGHTSPEED；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））
This paper presents a new large multiview dataset called HUMBI for human body expressions with natural clothing. The goal of HUMBI is to facilitate modeling view-specific appearance and geometry of five primary body signals including gaze, face, hand, body, and garment from assorted people. 107 synchronized HD cameras are used to capture 772 distinctive subjects across gender, ethnicity, age, and style. With the multiview image streams, we reconstruct high fidelity body expressions using 3D mesh models, which allows representing view-specific appearance. We demonstrate that HUMBI is highly eff…
University of Minnesota（明尼苏达大学）；POSTECH（浦项科技大学）；BinaryVR
This paper presents a new large multiview dataset called HUMBI for human body expressions with natural clothing. The goal of HUMBI is to facilitate modeling view-specific appearance and geometry of gaze, face, hand, body, and garment from assorted people. 107 synchronized HD cameras are used to capture 772 distinctive subjects across gender, ethnicity, age, and physical condition. With the multiview image streams, we reconstruct high fidelity body expressions using 3D mesh models, which allows representing view-specific appearance using their canonical atlas. We demonstrate that HUMBI is highl…
Northwestern University（西北大学）；University of California at Los Angles（加州大学洛杉矶分校）
Existing methods on video-based action recognition are generally view-dependent, i.e., performing recognition from the same views seen in the training data. We present a novel multiview spatio-temporal AND-OR graph (MST-AOG) representation for cross-view action recognition, i.e., the recognition is performed on the video from an unknown and unseen view. As a compositional model, MST-AOG compactly represents the hierarchical combinatorial structures of cross-view actions by explicitly modeling the geometry, appearance and motion variations. This paper proposes effective methods to learn the str…
GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（南京航空航天大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Department of Computing, Hong Kong Polytechnic University（香港理工大学计算学系）；School of Computer Science, Peking University（北京大学计算机科学学院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy（广东省人工智能与数字经济实验室）；Huawei Consumer Business Group, Huawei（华为消费者业务集团）；Department of Information Engineering and Computer Science, University of Trento（特伦托大学信息工程与计算机科学系）
Robust Low-light Scene Restoration via Illumination Transition
The Hong Kong University of Science and Technology（香港科学与技术大学）；The Hong Kong Polytechnic University（香港理工大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）；University of Surrey（萨里大学）
Chair for Computer Aided Medical Procedures (CAMP) Technical University of Munich（计算机辅助医疗程序研究所（CAMP）慕尼黑技术大学）；Robotics Institute Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Multiview 3D face modeling has attracted increasing attention recently and has become one of the potential avenues in future video systems. We aim to make more reliable and robust automatic feature extraction and natural 3D feature construction from 2D features detected on a pair of frontal and profile view face images. We propose several heuristic algorithms to minimize possible errors introduced by prevalent nonperfect orthogonal condition and noncoherent luminance. In our approach, we first extract the 2D features that are visible to both cameras in both views. Then, we estimate the coordin…
We introduce a deep appearance model for rendering the human face. Inspired by Active Appearance Models, we develop a data-driven rendering pipeline that learns a joint representation of facial geometry and appearance from a multiview capture setup. Vertex positions and view-specific textures are modeled using a deep variational autoencoder that captures complex nonlinear effects while producing a smooth and compact latent representation. View-specific texture enables the modeling of view-dependent effects such as specularity. In addition, it can also correct for imperfect geometry stemming fr…
