Monocular depth estimation, which plays a crucial role in understanding 3D scene geometry, is an ill-posed problem. Recent methods have gained significant improvement by exploring image-level information and hierarchical features from deep convolutional neural networks (DCNNs). These methods model depth estimation as a regression problem and train the regression networks by minimizing mean squared error, which suffers from slow convergence and unsatisfactory local solutions. Besides, existing depth estimation networks employ repeated spatial pooling operations, resulting in undesirable low-resolution feature maps. To obtain high-resolution depth maps, skip-connections or multi-layer deconvolution networks are required, which complicates network training and consumes much more computations. To eliminate or at least largely reduce these problems, we introduce a spacing-increasing discretization (SID) strategy to discretize depth and recast depth network learning as an ordinal regression problem. By training the network using an ordinary regression loss, our method achieves much higher accuracy and \dd{faster convergence in synch}. Furthermore, we adopt a multi-scale network structure which avoids unnecessary spatial pooling and captures multi-scale information in parallel.
The method described in this paper achieves state-of-the-art results on four challenging benchmarks, i.e., KITTI [17], ScanNet [9], Make3D [50], and NYU Depth v2 [42], and win the 1st prize in Robust Vision Challenge 2018. Code has been made available at: https://github.com/hufu6371/DORN.
OPPO US Research Center（OPPO美国研究中心）；InnoPeak Technology, Inc.（InnoPeak技术公司）；Wing LLC（Wing有限责任公司）
In this paper, we deal with the problem of monocular depth estimation for fisheye cameras in a self-supervised manner. A known issue of self-supervised depth estimation is that it suffers in low-light/over-exposure conditions and in large homogeneous regions. To tackle this issue, we propose a novel ordinal distillation loss that distills the ordinal information from a large teacher model. Such a teacher model, since having been trained on a large amount of diverse data, can capture the depth ordering information well, but lacks in preserving accurate scene geometry. Combined with self-supervi…
The University of Adelaide（阿德莱德大学）；Tsinghua University（清华大学）；Huazhong University of Science and Technology（华中科技大学）；Shanghai Institute for Advanced Communications and Data Science, Shanghai University（上海大学上海先进通信与数据科学研究院）
Most existing algorithms for depth estimation from single monocular images need large quantities of metric groundtruth depths for supervised learning. We show that relative depth can be an informative cue for metric depth estimation and can be easily obtained from vast stereo videos. Acquiring metric depths from stereo videos is sometimes impracticable due to the absence of camera parameters. In this paper, we propose to improve the performance of metric depth estimation with relative depths collected from stereo movie videos using existing stereo matching algorithm. We introduce a new "Relati…
We integrate sparse radar data into a monocular depth estimation model and introduce a novel preprocessing method for reducing the sparseness and limited field of view provided by radar. We explore the intrinsic error of different radar modalities and show our proposed method results in more data points with reduced error. We further propose a novel method for estimating dense depth maps from monocular 2D images and sparse radar measurements using deep learning based on the deep ordinal regression network by Fu et al. Radar data are integrated by first converting the sparse 2D points to a heig…
Single image depth estimation is a challenging problem. The current state-of-the-art method formulates the problem as that of ordinal regression. However, the formulation is not fully differentiable and depth maps are not generated in an end-to-end fashion. The method uses a naïve threshold strategy to determine per-pixel depth labels, which results in significant discretization errors. For the first time, we formulate a fully differentiable ordinal regression and train the network in end-to-end fashion. This enables us to include boundary and smoothness constraints in the optimization functio…
University of Pittsburgh（匹兹堡大学）；Laboratoire d’Informatique Gaspard Monge - CNRS UMR 8049, Universit$ ´ e $ Paris-Est（加斯帕德·蒙日信息学实验室 - 法国国家科学研究中心联合实验室8049，巴黎东部大学）
Monocular depth estimation, which plays a key role in understanding 3D scene geometry, is fundamentally an ill-posed problem. Existing methods based on deep convolutional neural networks (DCNNs) have examined this problem by learning convolutional networks to estimate continuous depth maps from monocular images. However, we find that training a network to predict a high spatial resolution continuous depth map often suffers from poor local solutions. In this paper, we hypothesize that achieving a compromise between spatial and depth resolutions can improve network training. Based on this "compr…
Paris-Saclay University（巴黎萨克雷大学）；ENSTA Paris（巴黎高等先进技术学院）；Institut Polytechnique de Paris（巴黎理工学院）
Monocular depth is important in many tasks, such as 3D reconstruction and autonomous driving. Deep learning based models achieve state-of-the-art performance in this field. A set of novel approaches for estimating monocular depth consists of transforming the regression task into a classification one. However, there is a lack of detailed descriptions and comparisons for Classification Approaches for Regression (CAR) in the community and no in-depth exploration of their potential for uncertainty estimation. To this end, this paper will introduce a taxonomy and summary of CAR approaches, a new un…
Remarkable progress has been made in 3D human pose estimation from a monocular RGB camera. However, only a few studies explored 3D multi-person cases. In this paper, we attempt to address the lack of a global perspective of the top-down approaches by introducing a novel form of supervision - Hierarchical Multi-person Ordinal Relations (HMOR). The HMOR encodes interaction information as the ordinal relations of depths and angles hierarchically, which captures the body-part and joint level semantic and maintains global consistency at the same time. In our approach, an integrated top-down model i…
Carnegie Mellon University Africa（卡内基梅隆大学非洲分校）；Carnegie Mellon University（卡内基梅隆大学）
Monocular Depth Estimation is usually treated as a supervised and regression problem when it actually is very similar to semantic segmentation task since they both are fundamentally pixel-level classification tasks. We applied depth increments that increases with depth in discretizing depth values and then applied Deeplab v2 and the result was higher accuracy. We were able to achieve a state-of-the-art result on the KITTI dataset and outperformed existing architecture by an 8% margin.
Image ordinal estimation is to predict the ordinal label of a given image, which can be categorized as an ordinal regression problem. Recent methods formulate an ordinal regression problem as a series of binary classification problems. Such methods cannot ensure that the global ordinal relationship is preserved since the relationships among different binary classifiers are neglected. We propose a novel ordinal regression approach, termed Convolutional Ordinal Regression Forest or CORF, for image ordinal estimation, which can integrate ordinal regression and differentiable decision trees with a…
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Indian Institute of Technology（印度理工学院）；Axogyan AI（Axogyan人工智能公司）
Monocular 3D human-pose estimation from static images is a challenging problem, due to the curse of dimensionality and the ill-posed nature of lifting 2D-to-3D. In this paper, we propose a Deep Conditional Variational Autoencoder based model that synthesizes diverse anatomically plausible 3D-pose samples conditioned on the estimated 2D-pose. We show that CVAE-based 3D-pose sample set is consistent with the 2D-pose and helps tackling the inherent ambiguity in 2D-to-3D lifting. We propose two strategies for obtaining the final 3D pose- (a) depth-ordering/ordinal relations to score and weight-ave…
Ordinal regression aims to classify instances into ordinal categories. In this paper, body mass index (BMI) category estimation from facial images is cast as an ordinal regression problem. In particular, noisy binary search algorithms based on pairwise comparisons are employed to exploit the ordinal relationship among BMI categories. Comparisons are performed with Siamese architectures, one of which uses the Bradley-Terry model probabilities as target. The Bradley-Terry model is an approach to describe probabilities of the possible outcomes when elements of a set are repeatedly compared with o…
A Survey on Ordinal Regression: Applications, Advances and Prospects
The Second Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院附属第二医院）；Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence（浙江省医学影像人工智能重点实验室）；HKUST (Guangzhou)（香港科技大学（广州））；AntGroup（蚂蚁集团）；University of Notre Dame（圣母大学）
This paper presents a language-powered paradigm for ordinal regression. Existing methods usually treat each rank as a category and employ a set of weights to learn these concepts. These methods are easy to overfit and usually attain unsatisfactory performance as the learned concepts are mainly derived from the training set. Recent large pre-trained vision-language models like CLIP have shown impressive performance on various visual tasks. In this paper, we propose to learn the rank concepts from the rich semantic CLIP latent space. Specifically, we reformulate this task as an image-language ma…
A novel ordinal regression algorithm, called moving window regression (MWR), is proposed in this paper. First, we propose the notion of relative rank ($ρ$-rank), which is a new order representation scheme for input and reference instances. Second, we develop global and local relative regressors ($ρ$-regressors) to predict $ρ$-ranks within entire and specific rank ranges, respectively. Third, we refine an initial rank estimate iteratively by selecting two reference instances to form a search window and then estimating the $ρ$-rank within the window. Extensive experiments results show that the p…
Fudan University（复旦大学）；Shanghai Center for Brain Science and Brain-inspired Technology（上海脑科学与类脑研究中心）
Deep learning-based methods have achieved promising performance in early detection and classification of lung nodules, most of which discard unsure nodules and simply deal with a binary classification -- malignant vs benign. Recently, an unsure data model (UDM) was proposed to incorporate those unsure nodules by formulating this problem as an ordinal regression, showing better performance over traditional binary classification. To further explore the ordinal relationship for lung nodule classification, this paper proposes a meta ordinal regression forest (MORF), which improves upon the state-o…
Prostate cancer (PCa) is one of the most common and aggressive cancers worldwide. The Gleason score (GS) system is the standard way of classifying prostate cancer and the most reliable method to determine the severity and treatment to follow. The pathologist looks at the arrangement of cancer cells in the prostate and assigns a score on a scale that ranges from 6 to 10. Automatic analysis of prostate whole-slide images (WSIs) is usually addressed as a binary classification problem, which misses the finer distinction between stages given by the GS. This paper presents a probabilistic deep learn…
Fudan University（复旦大学）；Shanghai Center for Brain Science and Brain-inspired Technology（上海脑科学与类脑研究中心）
The performance of medical image classification has been enhanced by deep convolutional neural networks (CNNs), which are typically trained with cross-entropy (CE) loss. However, when the label presents an intrinsic ordinal property in nature, e.g., the development from benign to malignant tumor, CE loss cannot take into account such ordinal information to allow for better generalization. To improve model generalization with ordinal information, we propose a novel meta ordinal regression forest (MORF) method for medical image classification with ordinal labels, which learns the ordinal relatio…
1 Shanghai Key Lab of Intelligent Information Processing, School of Computer Science 2 Institute of Science；Technology for Brain-inspired Intelligence Fudan University, Shanghai 200433, China
The progression of lung cancer implies the intrinsic ordinal relationship of lung nodules at different stages-from benign to unsure then to malignant. This problem can be solved by ordinal regression methods, which is between classification and regression due to its ordinal label. However, existing convolutional neural network (CNN)-based ordinal regression methods only focus on modifying classification head based on a randomly sampled mini-batch of data, ignoring the ordinal relationship resided in the data itself. In this paper, we propose a Meta Ordinal Weighting Network (MOW-Net) to explic…
Dual-level Fuzzy Learning with Patch Guidance for Image Ordinal Regression
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Public Health, Zhejiang University（浙江大学公共卫生学院）；State Key Laboratory of Transvascular Implantation Devices, The Second Affiliated Hospital Zhejiang University School of Medicine（浙江大学医学院第二附属医院血管内植入装置国家重点实验室）；Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence（浙江省医学影像人工智能重点实验室）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）；College of Engineering, University of Notre Dame（圣母大学工程学院）；Transvascular Implantation Devices Research Institute（血管内植入装置研究所）
Fudan University（复旦大学）；School of Computer Science, Fudan University（复旦大学计算机学院）；Shanghai Key Lab of Intelligent Information Processing（上海市智能信息处理重点实验室）
Computer vision researchers prefer to estimate age from face images because facial features provide useful information. However, estimating age from face images becomes challenging when people are distant from the camera or occluded. A person's gait is a unique biometric feature that can be perceived efficiently even at a distance. Thus, gait can be used to predict age when face images are not available. However, existing gait-based classification or regression methods ignore the ordinal relationship of different ages, which is an important clue for age estimation. This paper proposes an ordin…
Stage Aware Diagnosis of Diabetic Retinopathy via Ordinal Regression
Amrita School of Computing, Amrita Vishwa Vidyapeetham, Amaravati Campus, Andhra Pradesh - 522503, India（阿米特拉计算学校，阿米特拉世界大学，阿曼瓦蒂校区，安得拉邦，印度）；Amity Institute of Information Technology, Amity University Jharkhand, Ranchi, India（阿米特大学信息科技学院，阿米特大学贾坎德分校，拉奇，印度）
An Ordinal Regression Framework for a Deep Learning Based Severity Assessment for Chest Radiographs
RWTH Aachen University Hospital（亚琛工业大学医院）；Visual Computing Institute, RWTH Aachen University（亚琛工业大学视觉计算研究所）；Institute of Medical Informatics, RWTH Aachen University Hospital（亚琛工业大学医院医学信息学研究所）；Department of Oral and Maxillofacial Surgery, RWTH Aachen University Hospital（亚琛工业大学医院口腔颌面外科）
Learning from a label distribution has achieved promising results on ordinal regression tasks such as facial age and head pose estimation wherein, the concept of adaptive label distribution learning (ALDL) has drawn lots of attention recently for its superiority in theory. However, compared with the methods assuming fixed form label distribution, ALDL methods have not achieved better performance. We argue that existing ALDL algorithms do not fully exploit the intrinsic properties of ordinal regression. In this paper, we emphatically summarize that learning an adaptive label distribution on ord…
Fudan University（复旦大学）；The Pennsylvania State University（宾夕法尼亚州立大学）
Facial aging and facial rejuvenation analyze a given face photograph to predict a future look or estimate a past look of the person. To achieve this, it is critical to preserve human identity and the corresponding aging progression and regression with high accuracy. However, existing methods cannot simultaneously handle these two objectives well. We propose a novel generative adversarial network based approach, named the Conditional Multi-Adversarial AutoEncoder with Ordinal Regression (CMAAE-OR). It utilizes an age estimation technique to control the aging accuracy and takes a high-level feat…
The University of Adelaide（阿德莱德大学）；Beihang University（北京航空航天大学）
Video anomaly detection is of critical practical importance to a variety of real applications because it allows human attention to be focused on events that are likely to be of interest, in spite of an otherwise overwhelming volume of video. We show that applying self-trained deep ordinal regression to video anomaly detection overcomes two key limitations of existing methods, namely, 1) being highly dependent on manually labeled normal training data; and 2) sub-optimal feature learning. By formulating a surrogate two-class ordinal regression task we devise an end-to-end trainable video anomaly…
We consider the task of estimating the 3D orientation of an object of known category given an image of the object and a bounding box around it. Recently, CNN-based regression and classification methods have shown significant performance improvements for this task. This paper proposes a new CNN-based approach to monocular orientation estimation that advances the state of the art in four different directions. First, we take into account the Riemannian structure of the orientation space when designing regression losses and nonlinear activation functions. Second, we propose a mixed Riemannian regr…
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Beijing Normal University（北京师范大学）
Uncertainty is the only certainty there is. Modeling data uncertainty is essential for regression, especially in unconstrained settings. Traditionally the direct regression formulation is considered and the uncertainty is modeled by modifying the output space to a certain family of probabilistic distributions. On the other hand, classification based regression and ranking based solutions are more popular in practice while the direct regression methods suffer from the limited performance. How to model the uncertainty within the present-day technologies for regression remains an open issue. In t…
Understanding the 3D geometric structure of the Earth's surface has been an active research topic in photogrammetry and remote sensing community for decades, serving as an essential building block for various applications such as 3D digital city modeling, change detection, and city management. Previous researches have extensively studied the problem of height estimation from aerial images based on stereo or multi-view image matching. These methods require two or more images from different perspectives to reconstruct 3D coordinates with camera information provided. In this paper, we deal with t…
University of Exeter（埃克塞特大学）；University of Cardiff（卡的夫大学）；Shandong University（山东大学）；National University of Singapore（新加坡国立大学）；Chinese Academy of Sciences（中国科学院）；Agency for Science, Technology and Research（科技研究局）；University of Liverpool（利物浦大学）
Fudan University（复旦大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Shanghai Center for Brain Science and Brain-Inspired Technology（上海脑科学与类脑研究中心）
The goal of image ordinal estimation is to estimate the ordinal label of a given image with a convolutional neural network. Existing methods are mainly based on ordinal regression and particularly focus on modeling the ordinal mapping from the feature representation of the input to the ordinal label space. However, the manifold of the resultant feature representations does not maintain the intrinsic ordinal relations of interest, which hinders the effectiveness of the image ordinal estimation. Therefore, this paper proposes learning intrinsic Consistent Ordinal REpresentations (CORE) from ordi…
National University of Singapore（新加坡国立大学）；Huawei International Pte Ltd（华为国际私人有限公司）
In computer vision, it is often observed that formulating regression problems as a classification task often yields better performance. We investigate this curious phenomenon and provide a derivation to show that classification, with the cross-entropy loss, outperforms regression with a mean squared error loss in its ability to learn high-entropy feature representations. Based on the analysis, we propose an ordinal entropy loss to encourage higher-entropy feature spaces while maintaining ordinal relationships to improve the performance of regression tasks. Experiments on synthetic and real-wor…
The Chinese University of Hong Kong（香港中文大学）；Facebook AI Research（Facebook AI研究院）；Facebook Reality Labs（Facebook现实实验室）
Most existing monocular 3D pose estimation approaches only focus on a single body part, neglecting the fact that the essential nuance of human motion is conveyed through a concert of subtle movements of face, hands, and body. In this paper, we present FrankMocap, a fast and accurate whole-body 3D pose estimation system that can produce 3D face, hands, and body simultaneously from in-the-wild monocular images. The core idea of FrankMocap is its modular design: We first run 3D pose regression methods for face, hands, and body independently, followed by composing the regression outputs via an int…
Symmetric Perception and Ordinal Regression for Detecting Scoliosis Natural Image
Xuzhou Central Hospital（徐州市中心医院）；The Xuzhou Clinical School of Xuzhou Medical University（徐州医科大学徐州临床学院）；Xuzhou Rehabilitation Hospital（徐州市康复医院）；The Affiliated Xuzhou Rehabilitation Hospital of Xuzhou Medical University（徐州医科大学附属徐州康复医院）；School of Computer Science and Technology, China University of Mining and Technology（中国矿业大学计算机科学与技术学院）；The First Affiliated Hospital, Sun Yat-Sen University（中山大学附属第一医院）；The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China（中国科学技术大学附属第一医院（中国科学技术大学生命科学与医学部））
The Chinese University of Hong Kong（香港中文大学）；Facebook AI Research（Facebook人工智能研究院）；Facebook Reality Labs（Facebook现实实验室）
Although the essential nuance of human motion is often conveyed as a combination of body movements and hand gestures, the existing monocular motion capture approaches mostly focus on either body motion capture only ignoring hand parts or hand motion capture only without considering body motion. In this paper, we present FrankMocap, a motion capture system that can estimate both 3D hand and body motion from in-the-wild monocular inputs with faster speed (9.5 fps) and better accuracy than previous work. Our method works in near real-time (9.5 fps) and produces 3D body and hand motion capture out…
We address the problem of universal domain adaptation (UDA) in ordinal regression (OR), which attempts to solve classification problems in which labels are not independent, but follow a natural order. We show that the UDA techniques developed for classification and based on the clustering assumption, under-perform in OR settings. We propose a method that complements the OR classifier with an auxiliary task of order learning, which plays the double role of discriminating between common and private instances, and expanding class labels to the private target images via ranking. Combined with adve…
Classifying Phonotrauma Severity from Vocal Fold Images with Soft Ordinal Regression
Massachusetts Institute of Technology（麻省理工学院）；School of Communication Sciences and Disorders University of Central Florida（中央佛罗里达大学通信科学与障碍学系）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）；Center for Laryngeal Surgery and Voice Rehabilitation（嗓音外科与嗓音康复中心）；Department of Surgery（外科部门）；Institute of Health Professions（健康专业研究所）；Athinoula Martinos Center（阿提尼奥拉马丁诺斯中心）
JD AI Research（京东人工智能研究院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Harbin Institute of Technology（哈尔滨工业大学）
This paper focuses on the regression of multiple 3D people from a single RGB image. Existing approaches predominantly follow a multi-stage pipeline that first detects people in bounding boxes and then independently regresses their 3D body meshes. In contrast, we propose to Regress all meshes in a One-stage fashion for Multiple 3D People (termed ROMP). The approach is conceptually simple, bounding box-free, and able to learn a per-pixel representation in an end-to-end manner. Our method simultaneously predicts a Body Center heatmap and a Mesh Parameter map, which can jointly describe the 3D bod…
Universitat Pompeu Fabra（庞培法布拉大学）；Imperial College London（伦敦帝国理工学院）；University of Twente（特文特大学）
In this paper, we address the Multi-Instance-Learning (MIL) problem when bag labels are naturally represented as ordinal variables (Multi--Instance--Ordinal Regression). Moreover, we consider the case where bags are temporal sequences of ordinal instances. To model this, we propose the novel Multi-Instance Dynamic Ordinal Random Fields (MI-DORF). In this model, we treat instance-labels inside the bag as latent ordinal states. The MIL assumption is modelled by incorporating a high-order cardinality potential relating bag and instance-labels,into the energy function. We show the benefits of the …
Tremor is a key diagnostic feature of Parkinson's Disease (PD), Essential Tremor (ET), and other central nervous system (CNS) disorders. Clinicians or trained raters assess tremor severity with TETRAS scores by observing patients. Lacking quantitative measures, inter- or intra- observer variabilities are almost inevitable as the distinction between adjacent tremor scores is subtle. Moreover, clinician assessments also require patient visits, which limits the frequency of disease progress evaluation. Therefore it is beneficial to develop an automated assessment that can be performed remotely an…
Efficient Feature Aggregation and Scale-Aware Regression for Monocular 3D Object Detection
Department of Electronic Engineering, Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院电子工程系）；School of Mathematics and Statistics, University of Glasgow（格拉斯哥大学数学与统计学学院）
Enhancing Monocular Height Estimation via Weak Supervision from Imperfect Labels
Chair of Data Science in Earth Observation, Technical University of Munich (TUM)（地球观测数据科学教授职位，慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；School of Engineering and Design, Technical University of Munich (TUM)（工程与设计学院，慕尼黑技术大学）
Goethe University（歌德大学）；Norwegian Open AI Lab（挪威开放人工智能实验室）；NTNU（挪威科技大学）
Classical monocular vSLAM/VO methods suffer from the scale ambiguity problem. Hybrid approaches solve this problem by adding deep learning methods, for example by using depth maps which are predicted by a CNN. We suggest that it is better to base scale estimation on estimating the traveled distance for a set of subsequent images. In this paper, we propose a novel end-to-end many-to-one traveled distance estimator. By using a deep recurrent convolutional neural network (RCNN), the traveled distance between the first and last image of a set of consecutive frames is estimated by our DistanceNet. …
