Continuous learning from streaming data is among the most challenging topics in the contemporary machine learning. In this domain, learning algorithms must not only be able to handle massive volumes of rapidly arriving data, but also adapt themselves to potential emerging changes. The phenomenon of the evolving nature of data streams is known as concept drift. While there is a plethora of methods designed for detecting its occurrence, all of them assume that the drift is connected with underlying changes in the source of data. However, one must consider the possibility of a malicious injection of false data that simulates a concept drift. This adversarial setting assumes a poisoning attack that may be conducted in order to damage the underlying classification system by forcing adaptation to false data. Existing drift detectors are not capable of differentiating between real and adversarial concept drift. In this paper, we propose a framework for robust concept drift detection in the presence of adversarial and poisoning attacks. We introduce the taxonomy for two types of adversarial concept drifts, as well as a robust trainable drift detector. It is based on the augmented Restricted Boltzmann Machine with improved gradient computation and energy function. We also introduce Relative Loss of Robustness - a novel measure for evaluating the performance of concept drift detectors under poisoning attacks. Extensive computational experiments, conducted on both fully and sparsely labeled data streams, prove the high robustness and efficacy of the proposed drift detection framework in adversarial scenarios.
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
Predictive services nowadays play an important role across all business sectors. However, deployed machine learning models are challenged by changing data streams over time which is described as concept drift. Prediction quality of models can be largely influenced by this phenomenon. Therefore, concept drift is usually handled by retraining of the model. However, current research lacks a recommendation which data should be selected for the retraining of the machine learning model. Therefore, we systematically analyze different data selection strategies in this work. Subsequently, we instantiat…
Nowadays with a growing number of online controlling systems in the organization and also a high demand of monitoring and stats facilities that uses data streams to log and control their subsystems, data stream mining becomes more and more vital. Hoeffding Trees (also called Very Fast Decision Trees a.k.a. VFDT) as a Big Data approach in dealing with the data stream for classification and regression problems showed good performance in handling facing challenges and making the possibility of any-time prediction. Although these methods outperform other methods e.g. Artificial Neural Networks (AN…
Pontifícia Universidade Católica do Paraná（巴拉那天主教大学）；Instituto Federal do Paraná（巴拉那联邦学院）
Concept drift in process mining (PM) is a challenge as classical methods assume processes are in a steady-state, i.e., events share the same process version. We conducted a systematic literature review on the intersection of these areas, and thus, we review concept drift in process mining and bring forward a taxonomy of existing techniques for drift detection and online process mining for evolving environments. Existing works depict that (i) PM still primarily focuses on offline analysis, and (ii) the assessment of concept drift techniques in processes is cumbersome due to the lack of common e…
Institute for Databases and Information Systems, Ulm University（乌尔姆大学数据库与信息系统研究所）；Faculty of Computers and Information, Mansoura University（曼苏拉大学计算机与信息学院）；Faculty of Media Engineering and Technology, German University in Cairo（开罗德国大学媒体工程与技术学院）
Business processes evolve over time to adapt to changing business environments. This requires continuous monitoring of business processes to gain insights into whether they conform to the intended design or deviate from it. The situation when a business process changes while being analysed is denoted as Concept Drift. Its analysis is concerned with studying how a business process changes, in terms of detecting and localising changes and studying the effects of the latter. Concept drift analysis is crucial to enable early detection and management of changes, that is, whether to promote a change…
Airbus Defence & Space Cybersecurity（空客防务与航天网络安全部门）
Recent advancements in Artificial Intelligence (AI) have brought new capabilities to behavioural analysis (UEBA) for cyber-security consisting in the detection of hostile action based on the unusual nature of events observed on the Information System.In our previous work (presented at C\&ESAR 2018 and FIC 2019), we have associated deep neural networks auto-encoders for anomaly detection and graph-based events correlation to address major limitations in UEBA systems. This resulted in reduced false positive and false negative rates, improved alert explainability, while maintaining real-time perf…
Mining data streams poses a number of challenges, including the continuous and non-stationary nature of data, the massive volume of information to be processed and constraints put on the computational resources. While there is a number of supervised solutions proposed for this problem in the literature, most of them assume that access to the ground truth (in form of class labels) is unlimited and such information can be instantly utilized when updating the learning system. This is far from being realistic, as one must consider the underlying cost of acquiring labels. Therefore, solutions that …
College of Computer National University of Defense Technology（国防科技大学计算机学院）
In big data era, machine learning is one of fundamental techniques in intrusion detection systems (IDSs). However, practical IDSs generally update their decision module by feeding new data then retraining learning models in a periodical way. Hence, some attacks that comprise the data for training or testing classifiers significantly challenge the detecting capability of machine learning-based IDSs. Poisoning attack, which is one of the most recognized security threats towards machine learning-based IDSs, injects some adversarial samples into the training phase, inducing data drifting of traini…
TECNALIA（泰克纳利亚研究院）；Basque Research and Technology Alliance (BRTA)（巴斯克研究与技术联盟）；University of the Basque Country UPV/EHU（巴斯克地区大学）；Basque Center for Applied Mathematics (BCAM)（巴斯克应用数学中心）；ParisTech（巴黎高科）；The University of Waikato（怀卡托大学）；Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI)（安达卢西亚数据科学与计算智能研究所）；University of Granada（格拉纳达大学）
Data stream mining extracts information from large quantities of data flowing fast and continuously (data streams). They are usually affected by changes in the data distribution, giving rise to a phenomenon referred to as concept drift. Thus, learning models must detect and adapt to such changes, so as to exhibit a good predictive performance after a drift has occurred. In this regard, the development of effective drift detection algorithms becomes a key factor in data stream mining. In this work we propose CU RIE, a drift detector relying on cellular automata. Specifically, in CU RIE the dist…
Ben-Gurion University of the Negev（内盖夫本-古里安大学）；University of Cagliari（卡利亚里大学）
Recently, neural network (NN)-based methods, including autoencoders, have been proposed for the detection of cyber attacks targeting industrial control systems (ICSs). Such detectors are often retrained, using data collected during system operation, to cope with the natural evolution (i.e., concept drift) of the monitored signals. However, by exploiting this mechanism, an attacker can fake the signals provided by corrupted sensors at training time and poison the learning process of the detector such that cyber attacks go undetected at test time. With this research, we are the first to demonstr…
University of Leicester（莱斯特大学）；University of Birmingham（伯明翰大学）
In data stream mining, predictive models typically suffer drops in predictive performance due to concept drift. As enough data representing the new concept must be collected for the new concept to be well learnt, the predictive performance of existing models usually takes some time to recover from concept drift. To speed up recovery from concept drift and improve predictive performance in data stream mining, this work proposes a novel approach called Multi-sourcE onLine TrAnsfer learning for Non-statIonary Environments (Melanie). Melanie is the first approach able to transfer knowledge between…
Analyzing and Mitigating Poisoning Attacks on Flow-Based Intrusion Detection
CortAIx Labs, Thales Deutschland（Thales Deutschland）；Grey Solutions；University of Osnabrück（奥斯纳布吕克大学）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）；Ulm University（乌尔姆大学）
Concept Drift Adaptation in Text Stream Mining Settings: A Systematic Review
Instituto Federal de Santa Catarina（圣卡塔琳娜联邦教育、科学和技术学院）；Pontifícia Universidade Católica do Paraná (PUCPR)（巴拉那天主教大学）；Instituto Federal de São Paulo（圣保罗联邦教育、科学和技术学院）；Universidade de Campinas（坎皮纳斯大学）；École de Technologie Supérieure, Université du Québec（魁北克大学高等技术学院）；Universidade Estadual de Ponta Grossa（庞塔格罗萨州立大学）
The Hong Kong Polytechnic University（香港理工大学）；Michigan State University（密歇根州立大学）
Graph-based Anomaly Detection (GAD) is becoming prevalent due to the powerful representation abilities of graphs as well as recent advances in graph mining techniques. These GAD tools, however, expose a new attacking surface, ironically due to their unique advantage of being able to exploit the relations among data. That is, attackers now can manipulate those relations (i.e., the structure of the graph) to allow some target nodes to evade detection. In this paper, we exploit this vulnerability by designing a new type of targeted structural poisoning attacks to a representative regression-based…
Learning from data streams is an increasingly important topic in data mining, machine learning, and artificial intelligence in general. A major focus in the data stream literature is on designing methods that can deal with concept drift, a challenge where the generating distribution changes over time. A general assumption in most of this literature is that instances are independently distributed in the stream. In this work we show that, in the context of concept drift, this assumption is contradictory, and that the presence of concept drift necessarily implies temporal dependence; and thus som…
Modern machine learning systems need to be able to cope with constantly arriving and changing data. Two main areas of research dealing with such scenarios are continual learning and data stream mining. Continual learning focuses on accumulating knowledge and avoiding forgetting, assuming information once learned should be stored. Data stream mining focuses on adaptation to concept drift and discarding outdated information, assuming that only the most recent data is relevant. While these two areas are mainly being developed in separation, they offer complementary views on the problem of learnin…
A Framework for Evaluating and Benchmarking Concept Drift Detection Methods
University of Coimbra（科英布拉大学）；Victoria University of Wellington（惠灵顿维多利亚大学）；Commerzbank（德国商业银行）；University of Waikato（怀卡托大学）；AI Institute, University of Waikato（怀卡托大学人工智能研究所）
Tensor decompositions are used in various data mining applications from social network to medical applications and are extremely useful in discovering latent structures or concepts in the data. Many real-world applications are dynamic in nature and so are their data. To deal with this dynamic nature of data, there exist a variety of online tensor decomposition algorithms. A central assumption in all those algorithms is that the number of latent concepts remains fixed throughout the entire stream. However, this need not be the case. Every incoming batch in the stream may have a different number…
In this research we address the problem of capturing recurring concepts in a data stream environment. Recurrence capture enables the re-use of previously learned classifiers without the need for re-learning while providing for better accuracy during the concept recurrence interval. We capture concepts by applying the Discrete Fourier Transform (DFT) to Decision Tree classifiers to obtain highly compressed versions of the trees at concept drift points in the stream and store such trees in a repository for future use. Our empirical results on real world and synthetic data exhibiting varying degr…
Centro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS)（智能技术研究中心）；Universidade de Santiago de Compostela（圣地亚哥-德孔波斯特拉大学）
Real life business processes change over time, in both planned and unexpected ways. The detection of these changes is crucial for organizations to ensure that the expected and the real behavior are as similar as possible. These changes over time are called concept drift and its detection is a big challenge in process mining since the inherent complexity of the data makes difficult distinguishing between a change and an anomalous execution. In this paper, we present C2D2 (Conformance Checking-based Drift Detection), a new approach to detect sudden control-flow changes in the process models from…
Stream classification methods classify a continuous stream of data as new labelled samples arrive. They often also have to deal with concept drift. This paper focuses on seasonal drift in stream classification, which can be found in many real-world application data sources. Traditional approaches of stream classification consider seasonal drift by including seasonal dummy/indicator variables or building separate models for each season. But these approaches have strong limitations in high-dimensional classification problems, or with complex seasonal patterns. This paper explores how to best han…
The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems
Department of Computer Science, University of Texas at El Paso（德克萨斯大学埃尔帕索分校计算机科学系）；School of Computing, Southern Illinois University Carbondale（南方伊利诺伊大学卡本代尔分校计算机学院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
School of Computer and Information Technology, Shanxi University（山西大学计算机与信息技术学院）；Key Laboratory of Computational Intelligence and Chinese Information Processing, Ministry of Education（计算智能与中文信息处理教育部重点实验室）；School of Computer Science and Technology, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology（大连理工大学电子信息与电气工程学部计算机科学与技术学院）
Data stream mining problem has caused widely concerns in the area of machine learning and data mining. In some recent studies, ensemble classification has been widely used in concept drift detection, however, most of them regard classification accuracy as a criterion for judging whether concept drift happening or not. Information entropy is an important and effective method for measuring uncertainty. Based on the information entropy theory, a new algorithm using information entropy to evaluate a classification result is developed. It uses ensemble classification techniques, and the weight of e…
Islamic Azad University, Karaj Branch（伊斯兰阿扎德大学卡拉伊分校）；University Putra Malaysia (UPM)（马来西亚博特拉大学）
Very large databases are required to store massive amounts of data that are continuously inserted and queried. Analyzing huge data sets and extracting valuable pattern in many applications are interesting for researchers. We can identify two main groups of techniques for huge data bases mining. One group refers to streaming data and applies mining techniques whereas second group attempts to solve this problem directly with efficient algorithms. Recently many researchers have focused on data stream as an efficient strategy against huge data base mining instead of mining on entire data base. The…
National University of Singapore（新加坡国立大学）；IIT Bombay（印度理工学院孟买分校）；Harvard University（哈佛大学）
Given a stream of entries over time in a multi-dimensional data setting where concept drift is present, how can we detect anomalous activities? Most of the existing unsupervised anomaly detection approaches seek to detect anomalous events in an offline fashion and require a large amount of data for training. This is not practical in real-life scenarios where we receive the data in a streaming manner and do not know the size of the stream beforehand. Thus, we need a data-efficient method that can detect and adapt to changing data trends, or concept drift, in an online manner. In this work, we p…
Institute of Computing Technology, Chinese Academy of Sciences, China（中国科学院计算技术研究所）；Purple Mountain Laboratories, China（紫金山实验室）；University of Chinese Academy of Sciences, China（中国科学院大学）
University of Louisville（路易斯维尔大学）；Huai Hai Institute of Technology（淮海工学院）
Operating in a dynamic real world environment requires a forward thinking and adversarial aware design for classifiers, beyond fitting the model to the training data. In such scenarios, it is necessary to make classifiers - a) harder to evade, b) easier to detect changes in the data distribution over time, and c) be able to retrain and recover from model degradation. While most works in the security of machine learning has concentrated on the evasion resistance (a) problem, there is little work in the areas of reacting to attacks (b and c). Additionally, while streaming data research concentra…
Faculdade de Engenharia da Universidade do Porto (FEUP)（波尔图大学工程学院（FEUP））；INESC TEC（INESC TEC（科技与创新国家研究所））
The practicality of a video surveillance system is adversely limited by the amount of queries that can be placed on human resources and their vigilance in response. To transcend this limitation, a major effort under way is to include software that (fully or at least semi) automatically mines video footage, reducing the burden imposed to the system. Herein, we propose a semi-supervised incremental learning framework for evolving visual streams in order to develop a robust and flexible track classification system. Our proposed method learns from consecutive batches by updating an ensemble in eac…
Learning from data streams is among the most vital fields of contemporary data mining. The online analysis of information coming from those potentially unbounded data sources allows for designing reactive up-to-date models capable of adjusting themselves to continuous flows of data. While a plethora of shallow methods have been proposed for simpler low-dimensional streaming problems, almost none of them addressed the issue of learning from complex contextual data, such as images or texts. The former is represented mainly by adaptive decision trees that have been proven to be very efficient in …
Instituto Federal de Santa Catarina (IFSC)（圣卡塔琳娜联邦学院（IFSC））；Pontifícia Universidade Católica do Paraná (PUCPR)（巴拉那天主教大学（PUCPR））；École de Technologie Supérieure (ÉTS), Université du Québec（魁北克大学高等技术学院（ÉTS））；Universidade Estadual de Ponta Grossa (UEPG)（庞塔格罗萨州立大学（UEPG））
University of Vienna（维也纳大学）；Technical University of Munich（慕尼黑工业大学）
Domains such as manufacturing and medicine crave for continuous monitoring and analysis of their processes, especially in combination with time series as produced by sensors. Time series data can be exploited to, for example, explain and predict concept drifts during runtime. Generally, a certain data volume is required in order to produce meaningful analysis results. However, reliable data sets are often missing, for example, if event streams and times series data are collected separately, in case of a new process, or if it is too expensive to obtain a sufficient data volume. Additional chall…
Federal University of São Carlos（圣卡洛斯联邦大学）；University of Waikato（怀卡托大学）
Often, machine learning applications have to cope with dynamic environments where data are collected in the form of continuous data streams with potentially infinite length and transient behavior. Compared to traditional (batch) data mining, stream processing algorithms have additional requirements regarding computational resources and adaptability to data evolution. They must process instances incrementally because the data's continuous flow prohibits storing data for multiple passes. Ensemble learning achieved remarkable predictive performance in this scenario. Implemented as a set of (sever…
Anomaly detection is critical for finding suspicious behavior in innumerable systems. We need to detect anomalies in real-time, i.e. determine if an incoming entity is anomalous or not, as soon as we receive it, to minimize the effects of malicious activities and start recovery as soon as possible. Therefore, online algorithms that can detect anomalies in a streaming manner are essential. We first propose MIDAS which uses a count-min sketch to detect anomalous edges in dynamic graphs in an online manner, using constant time and memory. We then propose two variants, MIDAS-R which incorporates t…
Pontifícia Universidade Católica do Paraná (PUCPR)（巴拉那天主教大学）；Université de Rouen Normandie（鲁昂诺曼底大学）
Mining data streams is a challenge per se. It must be ready to deal with an enormous amount of data and with problems not present in batch machine learning, such as concept drift. Therefore, applying a batch-designed technique, such as dynamic selection of classifiers (DCS) also presents a challenge. The dynamic characteristic of ensembles that deal with streams presents barriers to the application of traditional DCS techniques in such classifiers. scikit-dyn2sel is an open-source python library tailored for dynamic selection techniques in streaming data. scikit-dyn2sel's development follows c…
University of São Paulo（圣保罗大学）；State University of Londrina（隆德里纳州立大学）
One of the current challenges in machine learning is how to deal with data coming at increasing rates in data streams. New predictive learning strategies are needed to cope with the high throughput data and concept drift. One of the data stream mining tasks where new learning strategies are needed is multi-target regression, due to its applicability in a high number of real world problems. While reliable and effective learning strategies have been proposed for batch multi-target regression, few have been proposed for multi-target online learning in data streams. Besides, most of the existing s…
Automated machine learning techniques benefited from tremendous research progress in recently. These developments and the continuous-growing demand for machine learning experts led to the development of numerous AutoML tools. However, these tools assume that the entire training dataset is available upfront and that the underlying distribution does not change over time. These assumptions do not hold in a data stream mining setting where an unbounded stream of data cannot be stored and is likely to manifest concept drift. Industry applications of machine learning on streaming data become more po…
Universidade de São Paulo（圣保罗大学）；University of New Mexico（新墨西哥大学）；University of New South Wales（新南威尔士大学）
Streaming data are increasingly present in real-world applications such as sensor measurements, satellite data feed, stock market, and financial data. The main characteristics of these applications are the online arrival of data observations at high speed and the susceptibility to changes in the data distributions due to the dynamic nature of real environments. The data stream mining community still faces some primary challenges and difficulties related to the comparison and evaluation of new proposals, mainly due to the lack of publicly available non-stationary real-world datasets. The compar…
Improving Sampling Methods for Fine-tuning SentenceBERT in Text Streams
Instituto Federal de Santa Catarina (IFSC)（圣卡塔琳娜联邦学院（IFSC））；Pontifícia Universidade Católica do Paraná (PUCPR)（巴拉那天主教大学（PUCPR））；École de Technologie Supérieure (ÉTS), Université du Québec（魁北克大学高等技术学院（ÉTS））；Universidade Estadual de Ponta Grossa (UEPG)（庞塔格罗萨州立大学（UEPG））
University of Coimbra（科英布拉大学）；Central University of Las Villas（拉斯维利亚斯中央大学）
With the massive data challenges nowadays and the rapid growing of technology, stream mining has recently received considerable attention. To address the large number of scenarios in which this phenomenon manifests itself suitable tools are required in various research fields. Instance-based data stream algorithms generally employ the Euclidean distance for the classification task underlying this problem. A novel way to look into this issue is to take advantage of a more flexible metric due to the increased requirements imposed by the data stream scenario. In this paper we present a new algori…
Know Your Streams: On the Conceptualization, Characterization, and Generation of Intentional Event Streams
School of Engineering and Design, Technical University of Munich（慕尼黑工业大学工程与设计学院）；University of Bayreuth（拜罗伊特大学）；Christian-Albrechts-University Kiel（基尔大学）；Database Systems and Data Mining, Ludwig Maximilian University of Munich（慕尼黑大学数据库系统与数据挖掘）；Munich Center for Machine Learning（慕尼黑机器学习中心）
本文针对物联网和传感器驱动系统带来的连续实时事件流挑战，提出Stream of Intent原型生成器，用于生成可复现的意图事件流以提升流过程挖掘算法的适应性与准确性。
SiameseDuo++: Active Learning from Data Streams with Dual Augmented Siamese Networks
KIOS Research and Innovation Center of Excellence, University of Cyprus（塞浦路斯大学KIOS卓越研究与创新中心）；Department of Electrical and Computer Engineering, University of Cyprus（塞浦路斯大学电气与计算机工程系）
University of Alberta（阿尔伯塔大学）；King Abdul-Aziz University（阿卜杜勒阿齐兹国王大学）；Systems Research Institute, Polish Academy of Science（波兰科学院系统研究所）；Johannes Kepler University（约翰内斯开普勒大学）；Nanyang Technological University（南洋理工大学）
The concept of ensemble learning offers a promising avenue in learning from data streams under complex environments because it addresses the bias and variance dilemma better than its single model counterpart and features a reconfigurable structure, which is well suited to the given context. While various extensions of ensemble learning for mining non-stationary data streams can be found in the literature, most of them are crafted under a static base classifier and revisits preceding samples in the sliding window for a retraining step. This feature causes computationally prohibitive complexity …
Harmful algal bloom forecasting. A comparison between stream and batch learning
Universidade da Coruña（拉科鲁尼亚大学）；Centro de investigación CITIC（CITIC研究中心）；University of A Coruña（拉科鲁尼亚大学）；Instituto de Investigaciones Mariñas (IIM-CSIC)（海洋研究所（CSIC））；Centro de Estudos do Ambiente e do Mar（环境与海洋研究中心）；University of Aveiro（阿威罗大学）
本文针对有害藻华预测中历史数据受限和概念漂移问题，提出基于 CROCO 海洋模型输出的机器学习工作流，比较七种算法在批量与流式学习范式下的表现，发现可解释流式模型 DoME 在 3 天预测中平均 R2 达 0.77，效果最佳。
