Estimating 3D human texture from a single image is essential in graphics and vision. It requires learning a mapping function from input images of humans with diverse poses into the parametric (UV) space and reasonably hallucinating invisible parts. To achieve a high-quality 3D human texture estimation, we propose a framework that adaptively samples the input by a deformable convolution where offsets are learned via a deep neural network. Additionally, we describe a novel cycle consistency loss that improves view generalization. We further propose to train our framework with an uncertainty-based pixel-level image reconstruction loss, which enhances color fidelity. We compare our method against the state-of-the-art approaches and show significant qualitative and quantitative improvements.
This work addresses the problem of model-based human pose estimation. Recent approaches have made significant progress towards regressing the parameters of parametric human body models directly from images. Because of the absence of images with 3D shape ground truth, relevant approaches rely on 2D annotations or sophisticated architecture designs. In this work, we advocate that there are more cues we can leverage, which are available for free in natural images, i.e., without getting more annotations, or modifying the network architecture. We propose a natural form of supervision, that capitali…
VGG-Tex: A Vivid Geometry-Guided Facial Texture Estimation Model for High Fidelity Monocular 3D Face Reconstruction
Renmin University of China（中国人民大学）；Tsinghua University（清华大学）；Beihang University（北京航空航天大学）；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）
Univ Lyon（里昂大学）；Ens de Lyon（里昂高等师范学院）；Univ Lyon 1（里昂第一大学）；CNRS（法国国家科学研究中心）；IRIT（图卢兹计算机研究所）；INP-ENSEEIHT（国立图卢兹应用科学学院-高等电力、电子、计算机、水力与电信学院）；University of Toulouse（图卢兹大学）
Texture segmentation constitutes a standard image processing task, crucial to many applications. The present contribution focuses on the particular subset of scale-free textures and its originality resides in the combination of three key ingredients: First, texture characterization relies on the concept of local regularity ; Second, estimation of local regularity is based on new multiscale quantities referred to as wavelet leaders ; Third, segmentation from local regularity faces a fundamental bias variance trade-off: In nature, local regularity estimation shows high variability that impairs t…
Technical University of Munich（慕尼黑工业大学）；Google Inc.（谷歌公司）
In this paper, we introduce neural texture learning for 6D object pose estimation from synthetic data and a few unlabelled real images. Our major contribution is a novel learning scheme which removes the drawbacks of previous works, namely the strong dependency on co-modalities or additional refinement. These have been previously necessary to provide training signals for convergence. We formulate such a scheme as two sub-optimisation problems on texture learning and pose learning. We separately learn to predict realistic texture of objects from real image collections and learn pose estimation …
We introduce VERTEX, an effective solution to recover 3D shape and intrinsic texture of vehicles from uncalibrated monocular input in real-world street environments. To fully utilize the template prior of vehicles, we propose a novel geometry and texture joint representation, based on implicit semantic template mapping. Compared to existing representations which infer 3D texture distribution, our method explicitly constrains the texture distribution on the 2D surface of the template as well as avoids limitations of fixed resolution and topology. Moreover, by fusing the global and local feature…
We propose a simple method for estimating noise level from a single color image. In most image-denoising algorithms, an accurate noise-level estimate results in good denoising performance; however, it is difficult to estimate noise level from a single image because it is an ill-posed problem. We tackle this problem by using prior knowledge that textures are highly correlated between RGB channels and noise is uncorrelated to other signals. We also extended our method for RAW images because they are available in almost all digital cameras and often used in practical situations. Experiments show …
Tampere University of Technology（坦佩雷理工大学）；Don State Technical University（顿河国立技术大学）
In the paper, a new method of blind estimation of noise variance in a single highly textured image is proposed. An input image is divided into 8x8 blocks and discrete cosine transform (DCT) is performed for each block. A part of 64 DCT coefficients with lowest energy calculated through all blocks is selected for further analysis. For the DCT coefficients, a robust estimate of noise variance is calculated. Corresponding to the obtained estimate, a part of blocks having very large values of local variance calculated only for the selected DCT coefficients are excluded from the further analysis. T…
The estimation of the camera poses associated with a set of images commonly relies on feature matches between the images. In contrast, we are the first to address this challenge by using objectness regions to guide the pose estimation problem rather than explicit semantic object detections. We propose Pose Refiner Network (PoserNet) a light-weight Graph Neural Network to refine the approximate pair-wise relative camera poses. PoserNet exploits associations between the objectness regions - concisely expressed as bounding boxes - across multiple views to globally refine sparsely connected view g…
Shanghai Jiao Tong University（上海交通大学）；The University of Manchester（曼彻斯特大学）；Vommatec Co. Ltd.（Vommatec有限公司）
The precise combination of image sensor and micro-lens array enables lenslet light field cameras to record both angular and spatial information of incoming light, therefore, one can calculate disparity and depth from light field images. In turn, 3D models of the recorded objects can be recovered, which is a great advantage over other imaging system. However, reflective and texture-less areas in light field images have complicated conditions, making it hard to correctly calculate disparity with existing algorithms. To tackle this problem, we introduce a novel end-to-end network VommaNet to retr…
High-Fidelity Facial Albedo Estimation via Texture Quantization
University of Technology Sydney（悉尼科技大学）；Shanghai Jiao Tong University（上海交通大学）；DeepGlint（旷视科技（北京））；Insightface；Zhejiang University（浙江大学）；Imperial College London（伦敦帝国学院）
Robust Low-Light Human Pose Estimation through Illumination-Texture Modulation
Nanjing University of Posts and Telecommunications（南京邮电大学）；Surrey Institute for People-Centred Artificial Intelligence（萨里以人为本人工智能研究所）；University of Surrey（萨里大学）
Towards Texture- And Shape-Independent 3D Keypoint Estimation in Birds
Department of Computer and Information Science, University of Konstanz, Germany（康斯坦茨大学计算机与信息科学系）；Centre for the Advanced Study of Collective Behaviour, University of Konstanz, Germany（康斯坦茨大学集体行为高级研究所以）；Department of Collective Behavior, Max Planck Institute of Animal Behavior, Germany（动物行为马克斯·普朗克研究所集体行为系）；Department of Biology, University of Konstanz, Germany（康斯坦茨大学生物学系）
Refining 3D Point Cloud Normal Estimation via Sample Selection
School of Information Science and Technology, Dalian Maritime University（大连海事大学信息科学技术学院）；Institute of Artificial Intelligence, Beijing University of Technology（北京工业大学人工智能研究院）；School of Science, Zhejiang Sci-Tech University（浙江理工大学理学院）；School of Mathematical Sciences, Dalian University of Technology（大连理工大学数学科学学院）
The Chinese University of Hong Kong（香港中文大学）；Sensetime Group Ltd.（商汤集团有限公司）；Shanghai Jiao Tong University（上海交通大学）；Nanyang Technological University（南洋理工大学）
When analyzing human motion videos, the output jitters from existing pose estimators are highly-unbalanced with varied estimation errors across frames. Most frames in a video are relatively easy to estimate and only suffer from slight jitters. In contrast, for rarely seen or occluded actions, the estimated positions of multiple joints largely deviate from the ground truth values for a consecutive sequence of frames, rendering significant jitters on them. To tackle this problem, we propose to attach a dedicated temporal-only refinement network to existing pose estimators for jitter mitigation, …
We propose a Transformer-based framework for 3D human texture estimation from a single image. The proposed Transformer is able to effectively exploit the global information of the input image, overcoming the limitations of existing methods that are solely based on convolutional neural networks. In addition, we also propose a mask-fusion strategy to combine the advantages of the RGB-based and texture-flow-based models. We further introduce a part-style loss to help reconstruct high-fidelity colors without introducing unpleasant artifacts. Extensive experiments demonstrate the effectiveness of t…
MCNet: A crowd denstity estimation network based on integrating multiscale attention module
College of Mechanical and Electronic Engineering, Dalian Minzu University（大连民族大学机电工程学院）；School of Computer Science and Engineering, Dalian Minzu University（大连民族大学计算机科学与工程学院）；Dalian University of Technology（大连理工大学）；Postdoctoral workstation of Dalian Yongjia Electronic Technology Co., Ltd（大连永佳电子科技有限公司博士后工作站）
National Aerospace University（国立航空航天大学）；University of Rennes 1（雷恩第一大学）
This paper deals with area-based subpixel image registration under rotation-isometric scaling-translation transformation hypothesis. Our approach is based on a parametrical modeling of geometrically transformed textural image fragments and maximum likelihood estimation of transformation vector between them. Due to the parametrical approach based on the fractional Brownian motion modeling of the local fragments texture, the proposed estimator MLfBm (ML stands for "Maximum Likelihood" and fBm for "Fractal Brownian motion") has the ability to better adapt to real image texture content compared to…
Orientation-conditioned Facial Texture Mapping for Video-based Facial Remote Photoplethysmography Estimation
Australian National University（澳大利亚国立大学）；Data61, Commonwealth and Scientific Industrial Research Organization（联邦科学与工业研究组织Data61）；University of Turku（图尔库大学）
University of Hail（哈伊勒大学）；International Gamming Lab（国际游戏实验室）
Crowd density estimation is an important task for crowd monitoring. Many efforts have been done to automate the process of estimating crowd density from images and videos. Despite series of efforts, it remains a challenging task. In this paper, we proposes a new texture feature-based approach for the estimation of crowd density based on Completed Local Binary Pattern (CLBP). We first divide the image into blocks and then re-divide the blocks into cells. For each cell, we compute CLBP and then concatenate them to describe the texture of the corresponding block. We then train a multi-class Suppo…
PhysPose: Refining 6D Object Poses with Physical Constraints
Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague（布拉格捷克理工大学捷克信息学、机器人学与控制论学院）；Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克理工大学电气工程学院）；Inria - Département d’Informatique de l’École normale supérieure, PSL Research University（法国国家信息与自动化研究所 - 巴黎高等师范学院信息学系，PSL大学）
Since now, many approaches has been proposed for surface defect detection based on image texture analysis techniques. One of the efficient texture analysis operations is local binary patterns which provides good accuracy.
Self-supervised deep learning methods have leveraged stereo images for training monocular depth estimation. Although these methods show strong results on outdoor datasets such as KITTI, they do not match performance of supervised methods on indoor environments with camera rotation. Indoor, rotated scenes are common for less constrained applications and pose problems for two reasons: abundance of low texture regions and increased complexity of depth cues for images under rotation. In an effort to extend self-supervised learning to more generalised environments we propose two additions. First, w…
This work presents a novel Convolutional Neural Network (CNN) architecture and a training procedure to enable robust and accurate pose estimation of a noncooperative spacecraft. First, a new CNN architecture is introduced that has scored a fourth place in the recent Pose Estimation Challenge hosted by Stanford's Space Rendezvous Laboratory (SLAB) and the Advanced Concepts Team (ACT) of the European Space Agency (ESA). The proposed architecture first detects the object by regressing a 2D bounding box, then a separate network regresses the 2D locations of the known surface keypoints from an imag…
Zhejiang University（浙江大学）；The University of Sydney（悉尼大学）；Image Derivative Inc.（图像衍生公司）
We propose a new method for object pose estimation without CAD models. The previous feature-matching-based method OnePose has shown promising results under a one-shot setting which eliminates the need for CAD models or object-specific training. However, OnePose relies on detecting repeatable image keypoints and is thus prone to failure on low-textured objects. We propose a keypoint-free pose estimation pipeline to remove the need for repeatable keypoint detection. Built upon the detector-free feature matching method LoFTR, we devise a new keypoint-free SfM method to reconstruct a semi-dense po…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Intel Labs（英特尔实验室）；SenseTime Group Limited（商汤集团有限公司）
We propose a method to refine geometry of 3D meshes from a consumer level depth camera, e.g. Kinect, by exploiting shading cues captured from an infrared (IR) camera. A major benefit to using an IR camera instead of an RGB camera is that the IR images captured are narrow band images that filter out most undesired ambient light, which makes our system robust against natural indoor illumination. Moreover, for many natural objects with colorful textures in the visible spectrum, the subjects appear to have a uniform albedo in the IR spectrum. Based on our analyses on the IR projector light of the …
ARTORG Center for Biomedical Engineering Research, University of Bern（伯尔尼大学ARTORG生物医学工程研究中心）
Food volume estimation is an essential step in the pipeline of dietary assessment and demands the precise depth estimation of the food surface and table plane. Existing methods based on computer vision require either multi-image input or additional depth maps, reducing convenience of implementation and practical significance. Despite the recent advances in unsupervised depth estimation from a single image, the achieved performance in the case of large texture-less areas needs to be improved. In this paper, we propose a network architecture that jointly performs geometric understanding (i.e., d…
University of Chinese Academy of Sciences（中国科学院大学）；Megvii Technology（旷视科技）；Peking University（北京大学）；Zhejiang University（浙江大学）
The estimation of 3D human body pose and shape from a single image has been extensively studied in recent years. However, the texture generation problem has not been fully discussed. In this paper, we propose an end-to-end learning strategy to generate textures of human bodies under the supervision of person re-identification. We render the synthetic images with textures extracted from the inputs and maximize the similarity between the rendered and input images by using the re-identification network as the perceptual metrics. Experiment results on pedestrian images show that our model can gene…
Hefei University of Technology（合肥工业大学）；Orbbec（奥比中光）；Sun Yat-sen University（中山大学）
It is well known that the passive stereo system cannot adapt well to weak texture objects, e.g., white walls. However, these weak texture targets are very common in indoor environments. In this paper, we present a novel stereo system, which consists of two cameras (an RGB camera and an IR camera) and an IR speckle projector. The RGB camera is used both for depth estimation and texture acquisition. The IR camera and the speckle projector can form a monocular structured-light (MSL) subsystem, while the two cameras can form a binocular stereo subsystem. The depth map generated by the MSL subsyste…
This paper addresses the task of relative camera pose estimation from raw image pixels, by means of deep neural networks. The proposed RPNet network takes pairs of images as input and directly infers the relative poses, without the need of camera intrinsic/extrinsic. While state-of-the-art systems based on SIFT + RANSAC, are able to recover the translation vector only up to scale, RPNet is trained to produce the full translation vector, in an end-to-end way. Experimental results on the Cambridge Landmark dataset show very promising results regarding the recovery of the full translation vector.…
Existing simultaneous localization and mapping (SLAM) algorithms are not robust in challenging low-texture environments because there are only few salient features. The resulting sparse or semi-dense map also conveys little information for motion planning. Though some work utilize plane or scene layout for dense map regularization, they require decent state estimation from other sources. In this paper, we propose real-time monocular plane SLAM to demonstrate that scene understanding could improve both state estimation and dense mapping especially in low-texture environments. The plane measurem…
IMATI-CNR（意大利国家研究委员会信息与远程技术研究所）；University of California, Berkeley（加州大学伯克利分校）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
We present the first method to perform automatic 3D pose, shape and texture capture of animals from images acquired in-the-wild. In particular, we focus on the problem of capturing 3D information about Grevy's zebras from a collection of images. The Grevy's zebra is one of the most endangered species in Africa, with only a few thousand individuals left. Capturing the shape and pose of these animals can provide biologists and conservationists with information about animal health and behavior. In contrast to research on human pose, shape and texture estimation, training data for endangered speci…
Structure-Centric Robust Monocular Depth Estimation via Knowledge Distillation
Beijing University of Posts and Telecommunications（北京邮电大学）；Institute of Computing Technology Chinese Academy of Sciences（中国科学院计算技术研究所）；Guangdong Hong Kong Macao Greater Bay Area National Technology Innovation Center（粤港澳大湾区国家技术创新中心）；Shouguang Cheng Zhi Feng Xing Technology Co., Ltd（寿光诚智丰兴科技有限公司）
