A Riemannian Framework for Analysis of Human Body Surface
Univ. Lille(里尔大学);CNRS(法国国家科学研究中心);Centrale Lille(里尔中央理工学院);IMT Nord Europe(法国国立高等矿业电信学校联盟北欧分校);Institut Mines-Télécom(法国国立高等矿业电信学校联盟);Wolfgang Pauli Institut(沃尔夫冈·泡利研究所)
We propose a novel framework for comparing 3D human shapes under the change of shape and pose. This problem is challenging since 3D human shapes vary significantly across subjects and body postures. We solve this problem by using a Riemannian approach. Our core contribution is the mapping of the human body surface to the space of metrics and normals. We equip this space with a family of Riemannian metrics, called Ebin (or DeWitt) metrics. We treat a human body surface as a point in a "shape space" equipped with a family of Riemannian metrics. The family of metrics is invariant under rigid motions and reparametrizations; hence it induces a metric on the "shape space" of surfaces. Using the alignment of human bodies with a given template, we show that this family of metrics allows us to distinguish the changes in shape and pose. The proposed framework has several advantages. First, we define a family of metrics with desired invariance properties for the comparison of human shape. Second, we present an efficient framework to compute geodesic paths between human shape given the chosen metric. Third, this framework provides some basic tools for statistical shape analysis of human body surfaces. Finally, we demonstrate the utility of the proposed framework in pose and shape retrieval of human body.
BaRe-ESA: A Riemannian Framework for Unregistered Human Body Shapes
Florida State University(佛罗里达州立大学);University of Vienna(维也纳大学);University of Houston(休斯顿大学);Univ. Lille(里尔大学);CNRS(法国国家科学研究中心);Centrale Lille(里尔中央理工学院);Institut Mines-Télécom(法国国立高等矿业电信学校联盟);CRIStAL(里尔信息、信号与自动化研究中心);IMT Nord Europe(法国北部国立高等矿业电信学校)
Riemannian Motion Generation: A Unified Framework for Human Motion Representation and Generation via Riemannian Flow Matching
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University(数据科学与人工智能系,香港理工大学);Department of Applied Mathematics, The Hong Kong Polytechnic University(应用数学系,香港理工大学);Department of Data Science and Statistics, Southern University of Science and Technology(数据科学与统计系,南方科技大学)
ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild
The Chinese University of Hong Kong(香港中文大学);The University of Hong Kong(香港大学);SenseTime Research(商汤科技研究院);The University of Sydney(悉尼大学);Shanghai AI Laboratory(上海人工智能实验室)
BodyNet: Volumetric Inference of 3D Human Body Shapes
Inria(法国国家信息与自动化研究所);Adobe Research(奥多比研究院)
人体形状估计是视频编辑、动画和时尚行业的一项重要任务。然而,由于人体、服装和视角等因素的变化,从自然图像预测3D人体形状是非常具有挑战性的。解决该问题的现有方法通常尝试用姿势和形状的某些先验来拟合参数化身体模型。在这项工作中,我们主张一种替代的表示,并提出BodyNet,一个神经网络直接推理的体积体型从一个单一的图像。BodyNet是一个端到端的可训练网络,它受益于(i)体积3D损失,(ii)多视图重投影损失,以及(iii)2D姿势,2D身体部位分割和3D姿势的中间监督。我们的实验表明,它们中的每一个结果的性能提高。为了评估该方法,我们将SMPL模型拟合到我们的网络输出中,并在SURREAL和Unite the People数据集上显示了最先进的结果,优于最近的方法。除了实现最先进的性能,我们的方法还可以实现体积的身体部位分割。
SAM-Body4D: Training-Free 4D Human Body Mesh Recovery from Videos
Department of Automation, Tsinghua University, Beijing, China(自动化系,清华大学,北京,中国);University of Warwick, Coventry, UK(沃里克大学,英国);University of Sheffield, Sheffield, UK(谢菲尔德大学,英国)
Structural Connectome Atlas Construction in the Space of Riemannian Metrics
Scientific Computing and Imaging Institute, University of Utah(犹他大学科学计算与成像研究所);University of California Los Angeles(加州大学洛杉矶分校);Florida State University(佛罗里达州立大学);University of Virginia(弗吉尼亚大学)
Disentangled Human Body Representation Based on Unsupervised Semantic-Aware Learning
X-ray Product, Siemens Shanghai Medical Equipment Ltd.(西门子上海医疗设备有限公司X射线产品部);University of Science and Technology of China, School of Biomedical Engineering & Suzhou Institute for Advanced Research(中国科学技术大学生物医学工程学院及苏州先进研究院)
DiffBody: Human Body Restoration by Imagining with Generative Diffusion Prior
Cornell University(康奈尔大学);The Hong Kong Polytechnic University(香港理工大学);Pennsylvania State University(宾夕法尼亚州立大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Snap Inc.(Snap公司)
MUC: Mixture of Uncalibrated Cameras for Robust 3D Human Body Reconstruction
ShanghaiTech University(上海科技大学);Shanghai Jiao Tong University(上海交通大学);Shanghai United Imaging Intelligence Co., Ltd.(上海联影智能医疗科技有限公司);Shanghai Clinical Research and Trial Center(上海临床研究中心)
Expressive Forecasting of 3D Whole-body Human Motions
MiLAB, Westlake University(西湖大学 MiLAB);Zhejiang University(浙江大学);Nanjing University of Science and Technology(南京理工大学);Shenzhen Graduate School, Peking University(北京大学深圳研究生院);Xiaohongshu Inc.(小红书公司)
TRB: A Novel Triplet Representation for Understanding 2D Human Body
CUHK-Sensetime Joint Lab(香港中文大学-商汤科技联合实验室);SenseTime Group Limited(商汤科技集团有限公司);The University of Sydney(悉尼大学);SenseTime Computer Vision Research Group(商汤科技计算机视觉研究组)
Learning Motion Priors for 4D Human Body Capture in 3D Scenes
ETH Zürich(苏黎世联邦理工学院);Microsoft(微软)
从单目视频中恢复复杂场景中的高质量3D人体运动对于从AR/VR到机器人的许多应用都很重要。然而,捕捉现实的人的场景交互,同时处理遮挡和部分视图,是具有挑战性的,目前的方法仍然远远没有达到令人信服的结果。我们通过提出LEMO:LEarning human MOtion priors for 4D human body capture来解决这个问题。通过利用大规模运动捕捉数据集AMASS,我们引入了一种新的运动平滑先验,它大大减少了在序列中恢复的姿势所表现出的抖动。此外,为了处理身体场景交互中经常发生的接触和遮挡,我们设计了一个接触摩擦项和一个通过每个实例自监督训练获得的接触感知运动填充器。为了证明所提出的运动先验的有效性,我们将它们组合成一个新的管道,用于3D场景中的4D人体捕获。通过我们的管道,我们展示了高质量的4D人体捕捉,重建平滑的运动和物理上合理的身体场景交互。代码和数据可在https://sanweiliti.github.io/LEMO/LEMO.html上获得。
Compressing Human Body Video with Interactive Semantics: A Generative Approach
City University of Hong Kong(香港城市大学);Alibaba DAMO Academy & Hupan Laboratory(阿里达摩院及胡畔实验室);Shenzhen Research Institute, City University of Hong Kong(香港城市大学深圳研究院)