Video anomaly detection is of critical practical importance to a variety of real applications because it allows human attention to be focused on events that are likely to be of interest, in spite of an otherwise overwhelming volume of video. We show that applying self-trained deep ordinal regression to video anomaly detection overcomes two key limitations of existing methods, namely, 1) being highly dependent on manually labeled normal training data; and 2) sub-optimal feature learning. By formulating a surrogate two-class ordinal regression task we devise an end-to-end trainable video anomaly detection approach that enables joint representation learning and anomaly scoring without manually labeled normal/abnormal data. Experiments on eight real-world video scenes show that our proposed method outperforms state-of-the-art methods that require no labeled training data by a substantial margin, and enables easy and accurate localization of the identified anomalies. Furthermore, we demonstrate that our method offers effective human-in-the-loop anomaly detection which can be critical in applications where anomalies are rare and the false-negative cost is high.
IBM Research AI（IBM研究人工智能）；The University of Tokyo（东京大学）
Visual anomaly detection is common in several applications including medical screening and production quality check. Although a definition of the anomaly is an unknown trend in data, in many cases some hints or samples of the anomaly class can be given in advance. Conventional methods cannot use the available anomaly data, and also do not have a robustness of noise. In this paper, we propose a novel spatially-weighted reconstruction-loss-based anomaly detection with a likelihood value from a regression model trained by all known data. The spatial weights are calculated by a region of interest …
Image ordinal estimation is to predict the ordinal label of a given image, which can be categorized as an ordinal regression problem. Recent methods formulate an ordinal regression problem as a series of binary classification problems. Such methods cannot ensure that the global ordinal relationship is preserved since the relationships among different binary classifiers are neglected. We propose a novel ordinal regression approach, termed Convolutional Ordinal Regression Forest or CORF, for image ordinal estimation, which can integrate ordinal regression and differentiable decision trees with a…
Ordinal regression aims to classify instances into ordinal categories. In this paper, body mass index (BMI) category estimation from facial images is cast as an ordinal regression problem. In particular, noisy binary search algorithms based on pairwise comparisons are employed to exploit the ordinal relationship among BMI categories. Comparisons are performed with Siamese architectures, one of which uses the Bradley-Terry model probabilities as target. The Bradley-Terry model is an approach to describe probabilities of the possible outcomes when elements of a set are repeatedly compared with o…
A Survey on Ordinal Regression: Applications, Advances and Prospects
The Second Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院附属第二医院）；Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence（浙江省医学影像人工智能重点实验室）；HKUST (Guangzhou)（香港科技大学（广州））；AntGroup（蚂蚁集团）；University of Notre Dame（圣母大学）
This paper presents a language-powered paradigm for ordinal regression. Existing methods usually treat each rank as a category and employ a set of weights to learn these concepts. These methods are easy to overfit and usually attain unsatisfactory performance as the learned concepts are mainly derived from the training set. Recent large pre-trained vision-language models like CLIP have shown impressive performance on various visual tasks. In this paper, we propose to learn the rank concepts from the rich semantic CLIP latent space. Specifically, we reformulate this task as an image-language ma…
A novel ordinal regression algorithm, called moving window regression (MWR), is proposed in this paper. First, we propose the notion of relative rank ($ρ$-rank), which is a new order representation scheme for input and reference instances. Second, we develop global and local relative regressors ($ρ$-regressors) to predict $ρ$-ranks within entire and specific rank ranges, respectively. Third, we refine an initial rank estimate iteratively by selecting two reference instances to form a search window and then estimating the $ρ$-rank within the window. Extensive experiments results show that the p…
Fudan University（复旦大学）；Shanghai Center for Brain Science and Brain-inspired Technology（上海脑科学与类脑研究中心）
Deep learning-based methods have achieved promising performance in early detection and classification of lung nodules, most of which discard unsure nodules and simply deal with a binary classification -- malignant vs benign. Recently, an unsure data model (UDM) was proposed to incorporate those unsure nodules by formulating this problem as an ordinal regression, showing better performance over traditional binary classification. To further explore the ordinal relationship for lung nodule classification, this paper proposes a meta ordinal regression forest (MORF), which improves upon the state-o…
Prostate cancer (PCa) is one of the most common and aggressive cancers worldwide. The Gleason score (GS) system is the standard way of classifying prostate cancer and the most reliable method to determine the severity and treatment to follow. The pathologist looks at the arrangement of cancer cells in the prostate and assigns a score on a scale that ranges from 6 to 10. Automatic analysis of prostate whole-slide images (WSIs) is usually addressed as a binary classification problem, which misses the finer distinction between stages given by the GS. This paper presents a probabilistic deep learn…
Fudan University（复旦大学）；Shanghai Center for Brain Science and Brain-inspired Technology（上海脑科学与类脑研究中心）
The performance of medical image classification has been enhanced by deep convolutional neural networks (CNNs), which are typically trained with cross-entropy (CE) loss. However, when the label presents an intrinsic ordinal property in nature, e.g., the development from benign to malignant tumor, CE loss cannot take into account such ordinal information to allow for better generalization. To improve model generalization with ordinal information, we propose a novel meta ordinal regression forest (MORF) method for medical image classification with ordinal labels, which learns the ordinal relatio…
1 Shanghai Key Lab of Intelligent Information Processing, School of Computer Science 2 Institute of Science；Technology for Brain-inspired Intelligence Fudan University, Shanghai 200433, China
The progression of lung cancer implies the intrinsic ordinal relationship of lung nodules at different stages-from benign to unsure then to malignant. This problem can be solved by ordinal regression methods, which is between classification and regression due to its ordinal label. However, existing convolutional neural network (CNN)-based ordinal regression methods only focus on modifying classification head based on a randomly sampled mini-batch of data, ignoring the ordinal relationship resided in the data itself. In this paper, we propose a Meta Ordinal Weighting Network (MOW-Net) to explic…
Dual-level Fuzzy Learning with Patch Guidance for Image Ordinal Regression
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Public Health, Zhejiang University（浙江大学公共卫生学院）；State Key Laboratory of Transvascular Implantation Devices, The Second Affiliated Hospital Zhejiang University School of Medicine（浙江大学医学院第二附属医院血管内植入装置国家重点实验室）；Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence（浙江省医学影像人工智能重点实验室）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）；College of Engineering, University of Notre Dame（圣母大学工程学院）；Transvascular Implantation Devices Research Institute（血管内植入装置研究所）
Fudan University（复旦大学）；School of Computer Science, Fudan University（复旦大学计算机学院）；Shanghai Key Lab of Intelligent Information Processing（上海市智能信息处理重点实验室）
Computer vision researchers prefer to estimate age from face images because facial features provide useful information. However, estimating age from face images becomes challenging when people are distant from the camera or occluded. A person's gait is a unique biometric feature that can be perceived efficiently even at a distance. Thus, gait can be used to predict age when face images are not available. However, existing gait-based classification or regression methods ignore the ordinal relationship of different ages, which is an important clue for age estimation. This paper proposes an ordin…
Stage Aware Diagnosis of Diabetic Retinopathy via Ordinal Regression
Amrita School of Computing, Amrita Vishwa Vidyapeetham, Amaravati Campus, Andhra Pradesh - 522503, India（阿米特拉计算学校，阿米特拉世界大学，阿曼瓦蒂校区，安得拉邦，印度）；Amity Institute of Information Technology, Amity University Jharkhand, Ranchi, India（阿米特大学信息科技学院，阿米特大学贾坎德分校，拉奇，印度）
An Ordinal Regression Framework for a Deep Learning Based Severity Assessment for Chest Radiographs
RWTH Aachen University Hospital（亚琛工业大学医院）；Visual Computing Institute, RWTH Aachen University（亚琛工业大学视觉计算研究所）；Institute of Medical Informatics, RWTH Aachen University Hospital（亚琛工业大学医院医学信息学研究所）；Department of Oral and Maxillofacial Surgery, RWTH Aachen University Hospital（亚琛工业大学医院口腔颌面外科）
Learning from a label distribution has achieved promising results on ordinal regression tasks such as facial age and head pose estimation wherein, the concept of adaptive label distribution learning (ALDL) has drawn lots of attention recently for its superiority in theory. However, compared with the methods assuming fixed form label distribution, ALDL methods have not achieved better performance. We argue that existing ALDL algorithms do not fully exploit the intrinsic properties of ordinal regression. In this paper, we emphatically summarize that learning an adaptive label distribution on ord…
Fudan University（复旦大学）；The Pennsylvania State University（宾夕法尼亚州立大学）
Facial aging and facial rejuvenation analyze a given face photograph to predict a future look or estimate a past look of the person. To achieve this, it is critical to preserve human identity and the corresponding aging progression and regression with high accuracy. However, existing methods cannot simultaneously handle these two objectives well. We propose a novel generative adversarial network based approach, named the Conditional Multi-Adversarial AutoEncoder with Ordinal Regression (CMAAE-OR). It utilizes an age estimation technique to control the aging accuracy and takes a high-level feat…
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Beijing Normal University（北京师范大学）
Uncertainty is the only certainty there is. Modeling data uncertainty is essential for regression, especially in unconstrained settings. Traditionally the direct regression formulation is considered and the uncertainty is modeled by modifying the output space to a certain family of probabilistic distributions. On the other hand, classification based regression and ranking based solutions are more popular in practice while the direct regression methods suffer from the limited performance. How to model the uncertainty within the present-day technologies for regression remains an open issue. In t…
Understanding the 3D geometric structure of the Earth's surface has been an active research topic in photogrammetry and remote sensing community for decades, serving as an essential building block for various applications such as 3D digital city modeling, change detection, and city management. Previous researches have extensively studied the problem of height estimation from aerial images based on stereo or multi-view image matching. These methods require two or more images from different perspectives to reconstruct 3D coordinates with camera information provided. In this paper, we deal with t…
The University of Sydney（悉尼大学）；University of Pittsburgh（匹兹堡大学）；Carnegie Mellon University（卡内基梅隆大学）；Université Paris-Est（巴黎东部大学）
Monocular depth estimation, which plays a crucial role in understanding 3D scene geometry, is an ill-posed problem. Recent methods have gained significant improvement by exploring image-level information and hierarchical features from deep convolutional neural networks (DCNNs). These methods model depth estimation as a regression problem and train the regression networks by minimizing mean squared error, which suffers from slow convergence and unsatisfactory local solutions. Besides, existing depth estimation networks employ repeated spatial pooling operations, resulting in undesirable low-res…
University of Exeter（埃克塞特大学）；University of Cardiff（卡的夫大学）；Shandong University（山东大学）；National University of Singapore（新加坡国立大学）；Chinese Academy of Sciences（中国科学院）；Agency for Science, Technology and Research（科技研究局）；University of Liverpool（利物浦大学）
National University of Singapore（新加坡国立大学）；Huawei International Pte Ltd（华为国际私人有限公司）
In computer vision, it is often observed that formulating regression problems as a classification task often yields better performance. We investigate this curious phenomenon and provide a derivation to show that classification, with the cross-entropy loss, outperforms regression with a mean squared error loss in its ability to learn high-entropy feature representations. Based on the analysis, we propose an ordinal entropy loss to encourage higher-entropy feature spaces while maintaining ordinal relationships to improve the performance of regression tasks. Experiments on synthetic and real-wor…
Symmetric Perception and Ordinal Regression for Detecting Scoliosis Natural Image
Xuzhou Central Hospital（徐州市中心医院）；The Xuzhou Clinical School of Xuzhou Medical University（徐州医科大学徐州临床学院）；Xuzhou Rehabilitation Hospital（徐州市康复医院）；The Affiliated Xuzhou Rehabilitation Hospital of Xuzhou Medical University（徐州医科大学附属徐州康复医院）；School of Computer Science and Technology, China University of Mining and Technology（中国矿业大学计算机科学与技术学院）；The First Affiliated Hospital, Sun Yat-Sen University（中山大学附属第一医院）；The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China（中国科学技术大学附属第一医院（中国科学技术大学生命科学与医学部））
We address the problem of universal domain adaptation (UDA) in ordinal regression (OR), which attempts to solve classification problems in which labels are not independent, but follow a natural order. We show that the UDA techniques developed for classification and based on the clustering assumption, under-perform in OR settings. We propose a method that complements the OR classifier with an auxiliary task of order learning, which plays the double role of discriminating between common and private instances, and expanding class labels to the private target images via ranking. Combined with adve…
Classifying Phonotrauma Severity from Vocal Fold Images with Soft Ordinal Regression
Massachusetts Institute of Technology（麻省理工学院）；School of Communication Sciences and Disorders University of Central Florida（中央佛罗里达大学通信科学与障碍学系）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）；Center for Laryngeal Surgery and Voice Rehabilitation（嗓音外科与嗓音康复中心）；Department of Surgery（外科部门）；Institute of Health Professions（健康专业研究所）；Athinoula Martinos Center（阿提尼奥拉马丁诺斯中心）
Tremor is a key diagnostic feature of Parkinson's Disease (PD), Essential Tremor (ET), and other central nervous system (CNS) disorders. Clinicians or trained raters assess tremor severity with TETRAS scores by observing patients. Lacking quantitative measures, inter- or intra- observer variabilities are almost inevitable as the distinction between adjacent tremor scores is subtle. Moreover, clinician assessments also require patient visits, which limits the frequency of disease progress evaluation. Therefore it is beneficial to develop an automated assessment that can be performed remotely an…
We integrate sparse radar data into a monocular depth estimation model and introduce a novel preprocessing method for reducing the sparseness and limited field of view provided by radar. We explore the intrinsic error of different radar modalities and show our proposed method results in more data points with reduced error. We further propose a novel method for estimating dense depth maps from monocular 2D images and sparse radar measurements using deep learning based on the deep ordinal regression network by Fu et al. Radar data are integrated by first converting the sparse 2D points to a heig…
This paper proposes a deep convolutional neural network model for ordinal regression by considering a family of probabilistic ordinal link functions in the output layer. The link functions are those used for cumulative link models, which are traditional statistical linear models based on projecting each pattern into a 1-dimensional space. A set of ordered thresholds splits this space into the different classes of the problem. In our case, the projections are estimated by a non-linear deep neural network. To further improve the results, we combine these ordinal models with a loss function that …
Dual-Resolution Attention-Gated Deep Learning with Ordinal Regression for Diabetic Retinopathy Grading: A Quantified Assessment of Cross-Domain Generalization
Tuwaiq Academy（图瓦伊克学院）；Tuwaiq Research and Development Centre（图瓦伊克研究与发展中心）
In education and intervention programs, user engagement has been identified as a major factor in successful program completion. Automatic measurement of user engagement provides helpful information for instructors to meet program objectives and individualize program delivery. In this paper, we present a novel approach for video-based engagement measurement in virtual learning programs. We propose to use affect states, continuous values of valence and arousal extracted from consecutive video frames, along with a new latent affective feature vector and behavioral features for engagement measurem…
In recent years, anomaly detection has become an essential field in medical image analysis. Most current anomaly detection methods for medical images are based on image reconstruction. In this work, we propose a novel anomaly detection approach based on coordinate regression. Our method estimates the position of patches within a volume, and is trained only on data of healthy subjects. During inference, we can detect and localize anomalies by considering the error of the position estimate of a given patch. We apply our method to 3D CT volumes and evaluate it on patients with intracranial haemor…
Middle East Technical University（中东技术大学）；Neuroscience and Neurotechnology Center of Excellence (NÖROM)（神经科学与神经技术卓越中心（NÖROM））；Marmara University School of Medicine（马尔马拉大学医学院）
In scoring systems used to measure the endoscopic activity of ulcerative colitis, such as Mayo endoscopic score or Ulcerative Colitis Endoscopic Index Severity, levels increase with severity of the disease activity. Such relative ranking among the scores makes it an ordinal regression problem. On the other hand, most studies use categorical cross-entropy loss function to train deep learning models, which is not optimal for the ordinal regression problem. In this study, we propose a novel loss function, class distance weighted cross-entropy (CDW-CE), that respects the order of the classes and t…
