SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measure
University of Hull（赫尔大学）；Fraunhofer Institute for Experimental Software Engineering, IESE（弗劳恩霍夫实验软件工程研究所（IESE））；Fraunhofer-Gesellschaft（弗劳恩霍夫协会）；APD Communications（APD通信公司）；Nuremberg Tech（纽伦堡应用技术大学）
Ensuring safety and explainability of machine learning (ML) is a topic of increasing relevance as data-driven applications venture into safety-critical application domains, traditionally committed to high safety standards that are not satisfied with an exclusive testing approach of otherwise inaccessible black-box systems. Especially the interaction between safety and security is a central challenge, as security violations can lead to compromised safety. The contribution of this paper to addressing both safety and security within a single concept of protection applicable during the operation of ML systems is active monitoring of the behaviour and the operational context of the data-driven system based on distance measures of the Empirical Cumulative Distribution Function (ECDF). We investigate abstract datasets (XOR, Spiral, Circle) and current security-specific datasets for intrusion detection (CICIDS2017) of simulated network traffic, using distributional shift detection measures including the Kolmogorov-Smirnov, Kuiper, Anderson-Darling, Wasserstein and mixed Wasserstein-Anderson-Darling measures. Our preliminary findings indicate that the approach can provide a basis for detecting whether the application context of an ML component is valid in the safety-security. Our preliminary code and results are available at https://github.com/ISorokos/SafeML.
University of Mannheim（曼海姆大学）；Fraunhofer IESE（弗劳恩霍夫实验软件工程研究所）；University of Hull（赫尔大学）
Machine Learning~(ML) has provided promising results in recent years across different applications and domains. However, in many cases, qualities such as reliability or even safety need to be ensured. To this end, one important aspect is to determine whether or not ML components are deployed in situations that are appropriate for their application scope. For components whose environments are open and variable, for instance those found in autonomous vehicles, it is therefore important to monitor their operational situation to determine its distance from the ML components' trained scope. If that…
In this paper we explore the role of sample mean in building a neural network for classification. This role is surprisingly extensive and includes: direct computation of weights without training, performance monitoring for samples without known classification, and self-training for unlabeled data. Experimental computation on a CIFAR-10 data set provides promising empirical evidence on the efficacy of a simple and widely applicable approach to some difficult problems.
Université de Lorraine（洛林大学）；CNRS（法国国家科学研究中心）；ACTA Mobilier
In recent times, the manufacturing processes are faced with many external or internal (the increase of customized product rescheduling , process reliability,..) changes. Therefore, monitoring and quality management activities for these manufacturing processes are difficult. Thus, the managers need more proactive approaches to deal with this variability. In this study, a proactive quality monitoring and control approach based on classifiers to predict defect occurrences and provide optimal values for factors critical to the quality processes is proposed. In a previous work (Noyel et al. 2013), …
Vibration-based quality monitoring of manufactured components often employs pattern recognition methods. Albeit developing several classification methods, they usually provide high accuracy for specific types of datasets, but not for general cases. In this paper, this issue has been addressed by developing a novel ensemble classifier based on the Dempster-Shafer theory of evidence. To deal with conflicting evidences, three remedies are proposed prior to combination: (i) selection of proper classifiers by evaluating the relevancy between the predicted and target outputs, (ii) devising an optimi…
Classifier-Free Diffusion-Based Weakly-Supervised Approach for Health Indicator Derivation in Rotating Machines: Advancing Early Fault Detection and Condition Monitoring
With today's abundant streams of data, the only constant we can rely on is change. For stream classification algorithms, it is necessary to adapt to concept drift. This can be achieved by monitoring the model error, and triggering counter measures as changes occur. In this paper, we propose a drift detection mechanism that fits a beta distribution to the model error, and treats abnormal behavior as drift. It works with any given model, leverages prior knowledge about this model, and allows to set application-specific confidence thresholds. Experiments confirm that it performs well, in particul…
Institute of Science and Technology Austria（奥地利科学技术研究所）；University of Konstanz（康斯坦茨大学）
Neural-network classifiers achieve high accuracy when predicting the class of an input that they were trained to identify. Maintaining this accuracy in dynamic environments, where inputs frequently fall outside the fixed set of initially known classes, remains a challenge. The typical approach is to detect inputs from novel classes and retrain the classifier on an augmented dataset. However, not only the classifier but also the detection mechanism needs to adapt in order to distinguish between newly learned and yet unknown input classes. To address this challenge, we introduce an algorithmic f…
Dissolved Gas-in-oil analysis (DGA) is used to monitor the condition of bushings on large power transformers. There are different techniques used in determining the conditions from the data collected, but in this work the Artificial Intelligence techniques are investigated. This work investigates which gases in DGA are related to each other and which ones are important for making decisions. When the related and crucial gases are determined, the other gases are discarded thereby reducing the number of attributes in DGA. Hence a further investigation is done to see how these new datasets influen…
The automation of condition monitoring and workpiece inspection plays an essential role in maintaining high quality as well as high throughput of the manufacturing process. To this end, the recent rise of developments in machine learning has lead to vast improvements in the area of autonomous process supervision. However, the more complex and powerful these models become, the less transparent and explainable they generally are as well. One of the main challenges is the monitoring of live deployments of these machine learning systems and raising alerts when encountering events that might impact…
Swiss Federal Institute of Technology Lausanne (EPFL)（洛桑联邦理工学院）
A person's passage of time perception (POTP) is strongly linked to their mental state and stress response, and can therefore provide an easily quantifiable means of continuous mental health monitoring. In this work, we develop a custom experiment and Machine Learning (ML) models for predicting POTP from biomarkers acquired from wearable biosensors. We first confirm that individuals experience time passing slower than usual during fear or sadness (p = 0.046) and faster than usual during cognitive tasks (p = 2 x 10^-5). Then, we group together the experimental segments associated with fast, slow…
The purpose of this study is to optimize the selection of prophylactic cardioverter defibrillator implantation candidates. Currently, the main criterion for implantation is a low Left Ventricular Ejection Fraction (LVEF) whose specificity is relatively poor. We designed two classifiers aimed to predict, from long term ECG recordings (Holter), whether a low-LVEF patient is likely or not to undergo ventricular arrhythmia in the next six months. One classifier is a single hidden layer neural network whose variables are the most relevant features extracted from Holter recordings, and the other cla…
LASTIMI, High School of Technology Mohammed V University in Rabat Sale, Morocco；LASTIMI, Mohammedia School of Engineers Mohammed V University in Rabat Rabat, Morocco；CISIEV. Faculty of Science；Technology in Marrakech Cadi Ayyad University Marrakech, Morocco
Coronary artery disease (CAD) is one of the most common cardiac diseases worldwide and causes disability and economic burden. It is the world's leading and most serious cause of mortality, with approximately 80% of deaths reported in low- and middle-income countries. The preferred and most precise diagnostic tool for CAD is angiography, but it is invasive, expensive, and technically demanding. However, the research community is increasingly interested in the computer-aided diagnosis of CAD via the utilization of machine learning (ML) methods. The purpose of this work is to present an e-diagnos…
In this paper, we propose a novel framework for the smart healthcare system, where we employ the compressed sensing (CS) and the combination of the state-of-the-art machine learning based denoiser as well as the alternating direction of method of multipliers (ADMM) structure. This integration significantly simplifies the software implementation for the lowcomplexity encoder, thanks to the modular structure of ADMM. Furthermore, we focus on detecting fall down actions from image streams. Thus, teh primary purpose of thus study is to reconstruct the image as visibly clear as possible and hence i…
Driven by the goal to enable sleep apnea monitoring and machine learning-based detection at home with small mobile devices, we investigate whether interpretation-based indirect knowledge transfer can be used to create classifiers with acceptable performance. Interpretation-based indirect knowledge transfer means that a classifier (student) learns from a synthetic dataset based on the knowledge representation from an already trained Deep Network (teacher). We use activation maximization to generate visualizations and create a synthetic dataset to train the student classifier. This approach has …
University of Sheffield（谢菲尔德大学）；The Alan Turing Institute（艾伦·图灵研究所）
Classification models are a fundamental component of physical-asset management technologies such as structural health monitoring (SHM) systems and digital twins. Previous work introduced risk-based active learning, an online approach for the development of statistical classifiers that takes into account the decision-support context in which they are applied. Decision-making is considered by preferentially querying data labels according to expected value of perfect information (EVPI). Although several benefits are gained by adopting a risk-based active learning approach, including improved deci…
ETH Zurich（苏黎世联邦理工学院）；Technion-Israel Institute of Technology（以色列理工学院）；Swiss Data Science Center（瑞士数据科学中心）
Generative Adversarial Networks (GANs) have shown remarkable results in modeling complex distributions, but their evaluation remains an unsettled issue. Evaluations are essential for: (i) relative assessment of different models and (ii) monitoring the progress of a single model throughout training. The latter cannot be determined by simply inspecting the generator and discriminator loss curves as they behave non-intuitively. We leverage the notion of duality gap from game theory to propose a measure that addresses both (i) and (ii) at a low computational cost. Extensive experiments show the ef…
A Personalized Zero-Shot ECG Arrhythmia Monitoring System: From Sparse Representation Based Domain Adaption to Energy Efficient Abnormal Beat Detection for Practical ECG Surveillance
Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；University of Trento（特伦托大学）；Free University of Bozen Bolzano（博尔扎诺自由大学）
Recent papers have introduced a novel approach to explain why a Predictive Process Monitoring (PPM) model for outcome-oriented predictions provides wrong predictions. Moreover, they have shown how to exploit the explanations, obtained using state-of-the art post-hoc explainers, to identify the most common features that induce a predictor to make mistakes in a semi-automated way, and, in turn, to reduce the impact of those features and increase the accuracy of the predictive model. This work starts from the assumption that frequent control flow patterns in event logs may represent important fea…
Visible Light Communication~(VLC) systems provide not only illumination and data communication, but also indoor monitoring services if the effect that different events create on the received optical signal is properly tracked. For this purpose, the Channel State Information that a VLC receiver computes to equalize the subcarriers of the OFDM signal can be also reused to train an Unsupervised Learning classifier. This way, different clusters can be created on the collected CSI data, which could be then mapped into relevant events to-be-monitored in the indoor environments, such as the presence …
University of Lille（里尔大学）；ENSAIT（国立高等艺术与纺织工业学院）；GEMTEX–Laboratoire de Génie et Matériaux Textiles（纺织工程与材料实验室）；University of Kent（肯特大学）
Deep learning play a vital role in classifying different arrhythmias using the electrocardiography (ECG) data. Nevertheless, training deep learning models normally requires a large amount of data and it can lead to privacy concerns. Unfortunately, a large amount of healthcare data cannot be easily collected from a single silo. Additionally, deep learning models are like black-box, with no explainability of the predicted results, which is often required in clinical healthcare. This limits the application of deep learning in real-world health systems. In this paper, we design a new explainable a…
Reliable data quality monitoring is a key asset in delivering collision data suitable for physics analysis in any modern large-scale High Energy Physics experiment. This paper focuses on the use of artificial neural networks for supervised and semi-supervised problems related to the identification of anomalies in the data collected by the CMS muon detectors. We use deep neural networks to analyze LHC collision data, represented as images organized geographically. We train a classifier capable of detecting the known anomalous behaviors with unprecedented efficiency and explore the usage of conv…
As an indispensable defensive measure of network security, the intrusion detection is a process of monitoring the events occurring in a computer system or network and analyzing them for signs of possible incidents. It is a classifier to judge the event is normal or malicious. The information used for intrusion detection contains some redundant features which would increase the difficulty of training the classifier for intrusion detection and increase the time of making predictions. To simplify the training process and improve the efficiency of the classifier, it is necessary to remove these di…
To identify and classify toxic online commentary, the modern tools of data science transform raw text into key features from which either thresholding or learning algorithms can make predictions for monitoring offensive conversations. We systematically evaluate 62 classifiers representing 19 major algorithmic families against features extracted from the Jigsaw dataset of Wikipedia comments. We compare the classifiers based on statistically significant differences in accuracy and relative execution time. Among these classifiers for identifying toxic comments, tree-based algorithms provide the m…
Current popular phishing prevention techniques mainly utilize reactive blocklists, which leave a ``window of opportunity'' for attackers during which victims are unprotected. One possible approach to shorten this window aims to detect phishing attacks earlier, during website preparation, by monitoring Certificate Transparency (CT) logs. Previous attempts to work with CT log data for phishing classification exist, however they lack evaluations on actual CT log data. In this paper, we present a pipeline that facilitates such evaluations by addressing a number of problems when working with CT log…
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Saarland University（萨尔大学）
The contemporary process-aware information systems possess the capabilities to record the activities generated during the process execution. To leverage these process specific fine-granular data, process mining has recently emerged as a promising research discipline. As an important branch of process mining, predictive business process management, pursues the objective to generate forward-looking, predictive insights to shape business processes. In this study, we propose a conceptual framework sought to establish and promote understanding of decision-making environment, underlying business pro…
Snecma, Safran Group（斯奈克玛公司，赛峰集团）；Université Paris 1（巴黎第一大学）
Automatic anomaly detection is a major issue in various areas. Beyond mere detection, the identification of the source of the problem that produced the anomaly is also essential. This is particularly the case in aircraft engine health monitoring where detecting early signs of failure (anomalies) and helping the engine owner to implement efficiently the adapted maintenance operations (fixing the source of the anomaly) are of crucial importance to reduce the costs attached to unscheduled maintenance. This paper introduces a general methodology that aims at classifying monitoring signals into nor…
Snecma, Safran Group（赛峰集团斯奈克玛公司）；Université Paris 1（巴黎第一大学）
Automatic anomaly detection is a major issue in various areas. Beyond mere detection, the identification of the origin of the problem that produced the anomaly is also essential. This paper introduces a general methodology that can assist human operators who aim at classifying monitoring signals. The main idea is to leverage expert knowledge by generating a very large number of indicators. A feature selection method is used to keep only the most discriminant indicators which are used as inputs of a Naive Bayes classifier. The parameters of the classifier have been optimized indirectly by the s…
Within a structural health monitoring (SHM) framework, we propose a simulation-based classification strategy to move towards online damage localization. The procedure combines parametric Model Order Reduction (MOR) techniques and Fully Convolutional Networks (FCNs) to analyze raw vibration measurements recorded on the monitored structure. First, a dataset of possible structural responses under varying operational conditions is built through a physics-based model, allowing for a finite set of predefined damage scenarios. Then, the dataset is used for the offline training of the FCN. Because of …
University of Sheffield（谢菲尔德大学）；The Alan Turing Institute（艾伦·图灵研究所）
Gaining the ability to make informed decisions on operation and maintenance of structures provides motivation for the implementation of structural health monitoring (SHM) systems. However, descriptive labels for measured data corresponding to health-states of the monitored system are often unavailable. This issue limits the applicability of fully-supervised machine learning paradigms for the development of statistical classifiers to be used in decision-support in SHM systems. One approach to dealing with this problem is risk-based active learning. In such an approach, data-label querying is gu…
Information Technology University（信息技术大学）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）；The University of Lahore（拉合尔大学）；University of Glasgow（格拉斯哥大学）
In post-covid19 world, radio frequency (RF)-based non-contact methods, e.g., software-defined radios (SDR)-based methods have emerged as promising candidates for intelligent remote sensing of human vitals, and could help in containment of contagious viruses like covid19. To this end, this work utilizes the universal software radio peripherals (USRP)-based SDRs along with classical machine learning (ML) methods to design a non-contact method to monitor different breathing abnormalities. Under our proposed method, a subject rests his/her hand on a table in between the transmit and receive antenn…
American University of Beirut (AUB)（贝鲁特美国大学）；University of New Haven（纽黑文大学）
Deep learning is a type of machine learning that adapts a deep hierarchy of concepts. Deep learning classifiers link the most basic version of concepts at the input layer to the most abstract version of concepts at the output layer, also known as a class or label. However, once trained over a finite set of classes, some deep learning models do not have the power to say that a given input does not belong to any of the classes and simply cannot be linked. Correctly invalidating the prediction of unrelated classes is a challenging problem that has been tackled in many ways in the literature. Nove…
The University of Texas at Dallas（德克萨斯大学达拉斯分校）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Fundamental knowledge in activity recognition of individuals with motor disorders such as Parkinson's disease (PD) has been primarily limited to detection of steady-state/static tasks (sitting, standing, walking). To date, identification of non-steady-state locomotion on uneven terrains (stairs, ramps) has not received much attention. Furthermore, previous research has mainly relied on data from a large number of body locations which could adversely affect user convenience and system performance. Here, individuals with mild stages of PD and healthy subjects performed non-steady-state circuit t…
Laboratoire LIO, Centre de Recherche du CHUM（CHUM 研究中心 LIO 实验室）；LICEF institute, TELUQ university（TELUQ 大学 LICEF 研究所）；INRS, Centre Énergie, Matériaux et Télécommunications（INRS 能源、材料与电信中心）；École de Technologie Supérieure (ÉTS)（高等技术学院）
Human activity recognition (HAR) by wearable sensor devices embedded in the Internet of things (IOT) can play a significant role in remote health monitoring and emergency notification, to provide healthcare of higher standards. The purpose of this study is to investigate a human activity recognition method of accrued decision accuracy and speed of execution to be applicable in healthcare. This method classifies wearable sensor acceleration time series data of human movement using efficient classifier combination of feature engineering-based and feature learning-based data representation. Leave…
An autonomous Artificial Internet of Things (AIoT) system for elderly dementia patients monitoring in a smart home is presented. The system mainly implements two functions based on the activity inference of the sensor data, which are real time abnormal activity monitoring and trend prediction of disease related activities. Specifically, CASAS dataset is employed to train a Random Forest (RF) model for activity inference. Then, another RF model trained by the output data of activity inference is used for abnormal activity monitoring. Particularly, RF is chosen for these tasks because of its bal…
In the current times, the fear and danger of COVID-19 virus still stands large. Manual monitoring of social distancing norms is impractical with a large population moving about and with insufficient task force and resources to administer them. There is a need for a lightweight, robust and 24X7 video-monitoring system that automates this process. This paper proposes a comprehensive and effective solution to perform person detection, social distancing violation detection, face detection and face mask classification using object detection, clustering and Convolution Neural Network (CNN) based bin…
Duke Kunshan University（昆山杜克大学）；Wuhan University（武汉大学）；Lenovo Research（联想研究院）；Sun Yat-sen University（中山大学）
This paper introduces our approaches for the Mask and Breathing Sub-Challenge in the Interspeech COMPARE Challenge 2020. For the mask detection task, we train deep convolutional neural networks with filter-bank energies, gender-aware features, and speaker-aware features. Support Vector Machines follows as the back-end classifiers for binary prediction on the extracted deep embeddings. Several data augmentation schemes are used to increase the quantity of training data and improve our models' robustness, including speed perturbation, SpecAugment, and random erasing. For the speech breath monito…
University of Science and Technology of China（中国科学技术大学）；Baidu Inc.（百度公司）；Institute of Geology, China Earthquake Administration（中国地震局地质研究所）；Institute of Geophysics, China Earthquake Administration（中国地震局地球物理研究所）；Rutgers, the State University of New Jersey（罗格斯大学新泽西州立大学）
Identifying the arrival times of seismic P-phases plays a significant role in real-time seismic monitoring, which provides critical guidance for emergency response activities. While considerable research has been conducted on this topic, efficiently capturing the arrival times of seismic P-phases hidden within intensively distributed and noisy seismic waves, such as those generated by the aftershocks of destructive earthquakes, remains a real challenge since most common existing methods in seismology rely on laborious expert supervision. To this end, in this paper, we present a machine learnin…
University of Stellenbosch（斯泰伦博斯大学）；TASK Applied Science（TASK应用科学公司）
We have performed cough detection based on measurements from an accelerometer attached to the patient's bed. This form of monitoring is less intrusive than body-attached accelerometer sensors, and sidesteps privacy concerns encountered when using audio for cough detection. For our experiments, we have compiled a manually-annotated dataset containing the acceleration signals of approximately 6000 cough and 68000 non-cough events from 14 adult male patients in a tuberculosis clinic. As classifiers, we have considered convolutional neural networks (CNN), long-short-term-memory (LSTM) networks, an…
From Classification to Segmentation with Explainable AI: A Study on Crack Detection and Growth Monitoring
Intelligent Maintenance and Operations Systems (IMOS), EPFL（智能维护与运营系统研究所（IMOS），洛桑联邦理工学院）；Environmental Computational Science and Earth Observation (ECEO), EPFL（环境计算科学与地球观测研究所（ECEO），洛桑联邦理工学院）
I Can't Believe It's Not Robust: Catastrophic Collapse of Safety Classifiers under Embedding Drift
Independent（独立研究者）；Meta AI；AWS Generative AI Innovation Center, Amazon Web Services（AWS生成式AI创新中心，亚马逊网络服务）；Northeastern University, Seattle, WA, USA（东北大学，西雅图，华盛顿州，美国）；Stanford University（斯坦福大学）
Automated classification of animal vocalisations is a potentially powerful wildlife monitoring tool. Training robust classifiers requires sizable annotated datasets, which are not easily recorded in the wild. To circumvent this problem, we recorded four primate species under semi-natural conditions in a wildlife sanctuary in Cameroon with the objective to train a classifier capable of detecting species in the wild. Here, we introduce the collected dataset, describe our approach and initial results of classifier development. To increase the efficiency of the annotation process, we condensed the…
Federal University of Lavras（拉夫拉斯联邦大学）；University of Bologna（博洛尼亚大学）；INFN Bologna（意大利国家核物理研究所博洛尼亚分部）；Federal Center for Technological Education CEFET-MG（米纳斯吉拉斯联邦技术教育中心）
Power-quality disturbances lead to several drawbacks such as limitation of the production capacity, increased line and equipment currents, and consequent ohmic losses; higher operating temperatures, premature faults, reduction of life expectancy of machines, malfunction of equipment, and unplanned outages. Real-time detection and classification of disturbances are deemed essential to industry standards. We propose an Evolving Gaussian Fuzzy Classification (EGFC) framework for semi-supervised disturbance detection and classification combined with a hybrid Hodrick-Prescott and Discrete-Fourier-T…
Department of Informatics, University of Oslo（奥斯陆大学信息学系）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）；Department of Cardiology, Oslo University Hospital, Rikshospitalet（奥斯陆大学医院心脏科（Rikshospitalet））；Institute of Clinical Medicine, Faculty of Medicine, University of Oslo（奥斯陆大学医学院临床医学研究所）；KG Jebsen Center for Cardiac Research, University of Oslo（奥斯陆大学KG Jebsen心脏研究中心）；Center for Heart Failure Research, Oslo University Hospital（奥斯陆大学医院心力衰竭研究中心）；Department of Otorhinolaryngology, Head & Neck Surgery, Sleep Unit, Lovisenberg Diakonale Hospital（洛维森伯格教会医院耳鼻喉科、头颈外科、睡眠科）；Department of Otorhinolaryngology, Head & Neck Surgery, Oslo University Hospital, Rikshospitalet（奥斯陆大学医院（Rikshospitalet）耳鼻喉科、头颈外科）；Department of Thoracic Medicine, St. Olavs University Hospital（圣奥拉夫大学医院胸科医学部）
In this work, we present an approach for unsupervised domain adaptation (DA) with the constraint, that the labeled source data are not directly available, and instead only access to a classifier trained on the source data is provided. Our solution, iteratively labels only high confidence sub-regions of the target data distribution, based on the belief of the classifier. Then it iteratively learns new classifiers from the expanding high-confidence dataset. The goal is to apply the proposed approach on DA for the task of sleep apnea detection and achieve personalization based on the needs of the…
Kennesaw State University（肯尼索州立大学）；Marquette University（马凯特大学）
People's personal hygiene habits speak volumes about the condition of taking care of their bodies and health in daily lifestyle. Maintaining good hygiene practices not only reduces the chances of contracting a disease but could also reduce the risk of spreading illness within the community. Given the current pandemic, daily habits such as washing hands or taking regular showers have taken primary importance among people, especially for the elderly population living alone at home or in an assisted living facility. This paper presents a novel and non-invasive framework for monitoring human hygie…
