When learning from streaming data, a change in the data distribution, also known as concept drift, can render a previously-learned model inaccurate and require training a new model. We present an adaptive learning algorithm that extends previous drift-detection-based methods by incorporating drift detection into a broader stable-state/reactive-state process. The advantage of our approach is that we can use aggressive drift detection in the stable state to achieve a high detection rate, but mitigate the false positive rate of standalone drift detection via a reactive state that reacts quickly to true drifts while eliminating most false positives. The algorithm is generic in its base learner and can be applied across a variety of supervised learning problems. Our theoretical analysis shows that the risk of the algorithm is competitive to an algorithm with oracle knowledge of when (abrupt) drifts occur. Experiments on synthetic and real datasets with concept drifts confirm our theoretical analysis.
Missing values, widely called as \textit{sparsity} in literature, is a common characteristic of many real-world datasets. Many imputation methods have been proposed to address this problem of data incompleteness or sparsity. However, the accuracy of a data imputation method for a given feature or a set of features in a dataset is highly dependent on the distribution of the feature values and its correlation with other features. Another problem that plagues industry deployments of machine learning (ML) solutions is concept drift detection, which becomes more challenging in the presence of missi…
Indian Institute of Technology Delhi（印度理工学院德里分校）；Indraprastha Institute of Information Technology, Delhi（德里英德拉普拉斯塔信息技术学院）
We propose Enhash, a fast ensemble learner that detects \textit{concept drift} in a data stream. A stream may consist of abrupt, gradual, virtual, or recurring events, or a mixture of various types of drift. Enhash employs projection hash to insert an incoming sample. We show empirically that the proposed method has competitive performance to existing ensemble learners in much lesser time. Also, Enhash has moderate resource requirements. Experiments relevant to performance comparison were performed on 6 artificial and 4 real data sets consisting of various types of drifts.
LEAD-Drift: Real-time and Explainable Intent Drift Detection by Learning a Data-Driven Risk Score
Information and Computer Science Department, King Fahd University of Petroleum and Minerals（信息与计算机科学系，国王法赫德石油与矿物大学）；IRC for Intelligent Secure Systems, King Fahd University of Petroleum and Minerals（智能安全系统研究组，国王法赫德石油与矿物大学）
Federal University of Amazonas（亚马逊联邦大学）；Federal Institute of Amazonas（亚马逊联邦研究所）；École de Technologie Supérieure（高等技术学院）
We propose an online method for concept driftdetection based on dynamic classifier ensemble selection. Theproposed method generates a pool of ensembles by promotingdiversity among classifier members and chooses expert ensemblesaccording to global prequential accuracy values. Unlike currentdynamic ensemble selection approaches that use only local knowl-edge to select the most competent ensemble for each instance,our method focuses on selection taking into account the decisionspace. Consequently, it is well adapted to the context of driftdetection in data stream problems. The results of the expe…
We develop a risk-neutral spot and equity option market simulator for a single underlying, under which the joint market process is a martingale. We leverage an efficient low-dimensional representation of the market which preserves no static arbitrage, and employ neural spline flows to simulate samples which are free from conditional drifts and are highly realistic in the sense that among all possible risk-neutral simulators, the obtained risk-neutral simulator is the closest to the historical data with respect to the Kullback-Leibler divergence. Numerical experiments demonstrate the effectiven…
The Alan Turing Institute（艾伦·图灵研究所）；University of Warwick（华威大学）
The increasing value of data held in enterprises makes it an attractive target to attackers. The increasing likelihood and impact of a cyber attack have highlighted the importance of effective cyber risk estimation. We propose two methods for modelling Value-at-Risk (VaR) which can be used for any time-series data. The first approach is based on Quantile Autoregression (QAR), which can estimate VaR for different quantiles, i.e. confidence levels. The second method, we term Competitive Quantile Autoregression (CQAR), dynamically re-estimates cyber risk as soon as new data becomes available. Thi…
University of Waikato（怀卡托大学）；Télécom ParisTech（巴黎高科电信学院）；Dalhousie University（达尔豪斯大学）
Concept drift detection is a crucial task in data stream evolving environments. Most of state of the art approaches designed to tackle this problem monitor the loss of predictive models. However, this approach falls short in many real-world scenarios, where the true labels are not readily available to compute the loss. In this context, there is increasing attention to approaches that perform concept drift detection in an unsupervised manner, i.e., without access to the true labels. We propose a novel approach to unsupervised concept drift detection based on a student-teacher learning paradigm.…
Bipartite ranking is an important supervised learning problem; however, unlike regression or classification, it has a quadratic dependence on the number of samples. To circumvent the prohibitive sample cost, many recent work focus on stochastic gradient-based methods. In this paper we consider an alternative approach, which leverages the structure of the widely-adopted pairwise squared loss, to obtain a stochastic and low cost algorithm that does not require stochastic gradients or learning rates. Using a novel uniform risk bound---based on matrix and vector concentration inequalities---we sho…
TECNALIA（泰克纳利亚研究院）；Basque Research and Technology Alliance (BRTA)（巴斯克研究与技术联盟）；University of the Basque Country UPV/EHU（巴斯克地区大学）；Basque Center for Applied Mathematics (BCAM)（巴斯克应用数学中心）；ParisTech（巴黎高科）；The University of Waikato（怀卡托大学）；Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI)（安达卢西亚数据科学与计算智能研究所）；University of Granada（格拉纳达大学）
Data stream mining extracts information from large quantities of data flowing fast and continuously (data streams). They are usually affected by changes in the data distribution, giving rise to a phenomenon referred to as concept drift. Thus, learning models must detect and adapt to such changes, so as to exhibit a good predictive performance after a drift has occurred. In this regard, the development of effective drift detection algorithms becomes a key factor in data stream mining. In this work we propose CU RIE, a drift detector relying on cellular automata. Specifically, in CU RIE the dist…
BCAM-Basque Center for Applied Mathematics（巴斯克应用数学中心）；University of the Basque Country UPV/EHU（巴斯克大学UPV/EHU）
The statistical characteristics of instance-label pairs often change with time in practical scenarios of supervised classification. Conventional learning techniques adapt to such concept drift accounting for a scalar rate of change by means of a carefully chosen learning rate, forgetting factor, or window size. However, the time changes in common scenarios are multidimensional, i.e., different statistical characteristics often change in a different manner. This paper presents adaptive minimax risk classifiers (AMRCs) that account for multidimensional time changes by means of a multivariate and…
Data stream poses additional challenges to statistical classification tasks because distributions of the training and target samples may differ as time passes. Such distribution change in streaming data is called concept drift. Numerous histogram-based distribution change detection methods have been proposed to detect drift. Most histograms are developed on grid-based or tree-based space partitioning algorithms which makes the space partitions arbitrary, unexplainable, and may cause drift blind-spots. There is a need to improve the drift detection accuracy for histogram-based methods with the …
We introduce the needs for explainable AI that arise from Standard No. 239 from the Basel Committee on Banking Standards (BCBS 239), which outlines 11 principles for effective risk data aggregation and risk reporting for financial institutions. Of these, explainableAI is necessary for compliance in two key aspects: data quality, and appropriate reporting for multiple stakeholders. We describe the implementation challenges for one specific regulatory requirement:that of having a complete data taxonomy that is appropriate for firmwide use. The constantly evolving nature of financial ontologies n…
In recent years, we have seen a handful of work on inference algorithms over non-stationary data streams. Given their flexibility, Bayesian non-parametric models are a good candidate for these scenarios. However, reliable streaming inference under the concept drift phenomenon is still an open problem for these models. In this work, we propose a variational inference algorithm for Dirichlet process mixture models. Our proposal deals with the concept drift by including an exponential forgetting over the prior global parameters. Our algorithm allows to adapt the learned model to the concept drift…
We propose two general and falsifiable hypotheses about expectations on generalization error when learning in the context of concept drift. One posits that as drift rate increases, the forgetting rate that minimizes generalization error will also increase and vice versa. The other posits that as a learner's forgetting rate increases, the bias/variance profile that minimizes generalization error will have lower variance and vice versa. These hypotheses lead to the concept of the sweet path, a path through the 3-d space of alternative drift rates, forgetting rates and bias/variance profiles on w…
In this work, we analyze the generalization ability of distributed online learning algorithms under stationary and non-stationary environments. We derive bounds for the excess-risk attained by each node in a connected network of learners and study the performance advantage that diffusion strategies have over individual non-cooperative processing. We conduct extensive simulations to illustrate the results.
Millions of vulnerable consumer IoT devices in home networks are the enabler for cyber crimes putting user privacy and Internet security at risk. Internet service providers (ISPs) are best poised to play key roles in mitigating risks by automatically inferring active IoT devices per household and notifying users of vulnerable ones. Developing a scalable inference method that can perform robustly across thousands of home networks is a non-trivial task. This paper focuses on the challenges of developing and applying data-driven inference models when labeled data of device behaviors is limited an…
Renaissance of RNNs in Streaming Clinical Time Series: Compact Recurrence Remains Competitive with Transformers
University of Texas at Dallas（德克萨斯大学达拉斯分校）；Independent Researcher（独立研究者）；Georgia Institute of Technology（佐治亚理工学院）；University of Michigan- Ann Arbor（密歇根大学安娜堡分校）
Topology Reorganized Graph Contrastive Learning with Mitigating Semantic Drift
College of Computer Science & Technology, Nanjing University of Aeronautics & Astronautics（南京航空航天大学计算机科学与技术学院）；MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing University of Aeronautics & Astronautics（南京航空航天大学工信部模式分析与机器智能重点实验室）
In document classification for, e.g., legal and biomedical text, we often deal with hundreds of classes, including very infrequent ones, as well as temporal concept drift caused by the influence of real world events, e.g., policy changes, conflicts, or pandemics. Class imbalance and drift can sometimes be mitigated by resampling the training data to simulate (or compensate for) a known target distribution, but what if the target distribution is determined by unknown future events? Instead of simply resampling uniformly to hedge our bets, we focus on the underlying optimization algorithms used …
Online Detection of Water Contamination Under Concept Drift
KIOS Research and Innovation Center of Excellence（KIOS研究与创新中心卓越中心）；Department of Electrical and Computer Engineering（电气与计算机工程系）；University of Cyprus（塞浦路斯大学）
Machine learning models serve critical functions, such as classifying loan applicants as good or bad risks. Each model is trained under the assumption that the data used in training and in the field come from the same underlying unknown distribution. Often, this assumption is broken in practice. It is desirable to identify when this occurs, to minimize the impact on model performance. We suggest a new approach to detecting change in the data distribution by identifying polynomial relations between the data features. We measure the strength of each identified relation using its R-square value. …
An Information-Theoretic Analysis for Federated Learning under Concept Drift
Department of Computer Science and Engineering, Southern University of Science and Technology（计算机科学与工程系，南方科技大学）；Zhejiang University–University of Illinois Urbana–Champaign Institute（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合研究所）
