We propose a new unsupervised and non-parametric method to detect change points in intricate quasi-periodic signals. The detection relies on optimal transport theory combined with topological analysis and the bootstrap procedure. The algorithm is designed to detect changes in virtually any harmonic or a partially harmonic signal and is verified on three different sources of physiological data streams. We successfully find abnormal or irregular cardiac cycles in the waveforms for the six of the most frequent types of clinical arrhythmias using a single algorithm. The validation and the efficiency of the method are shown both on synthetic and on real time series. Our unsupervised approach reaches the level of performance of the supervised state-of-the-art techniques. We provide conceptual justification for the efficiency of the method and prove the convergence of the bootstrap procedure theoretically.
We study the minimax rates of the label shift problem in non-parametric classification. In addition to the unsupervised setting in which the learner only has access to unlabeled examples from the target domain, we also consider the setting in which a small number of labeled examples from the target domain is available to the learner. Our study reveals a difference in the difficulty of the label shift problem in the two settings, and we attribute this difference to the availability of data from the target domain to estimate the class conditional distributions in the latter setting. We also show…
Extending the ideas of [7], this paper aims at providing a kernel based non-parametric estimation of a new class of time varying AR(1) processes (Xt), with local stationarity and periodic features (with a known period T), inducing the definition Xt = at(t/nT)X t--1 + $ξ$t for t $\in$ N and with a t+T $\not\equiv$ at. Central limit theorems are established for kernel estima-tors as(u) reaching classical minimax rates and only requiring low order moment conditions of the white noise ($ξ$t)t up to the second order.
This paper deals with the parametric inference for integrated signals embedded in an additive Gaussian noise and observed at deterministic discrete instants which are not necessarily equidistant. The unknown parameter is multidimensional and compounded of a signal-of-interest parameter and a variance parameter of the noise. We state the consistency and the minimax efficiency of the maximum likelihood estimator and of the Bayesian estimator when the time of observation tends to $\infty$ and the delays between two consecutive observations tend to 0 or are only bounded. The class of signals in co…
Consider a periodic, mean-reverting Ornstein-Uhlenbeck process $X=\{X_t,t\geq0\}$ of the form $d X_{t}=\left(L(t)+αX_{t}\right) d t+ dB^H_{t}, \quad t \geq 0$, where $L(t)=\sum_{i=1}^{p}μ_iϕ_i (t)$ is a periodic parametric function, and $\{B^H_t,t\geq0\}$ is a fractional Brownian motion of Hurst parameter $\frac12\leq H<1$. In the "ergodic" case $α<0$, the parametric estimation of $(μ_1,\ldots,μ_p,α)$ based on continuous-time observation of $X$ has been considered in Dehling et al. \cite{DFK}, and in Dehling et al. \cite{DFW} for $H=\frac12$, and $\frac12<H<1$, respectively. In this paper we c…
This paper investigates a fully unsupervised statistical method for edge preserving image restoration and compression using a spatial decomposition scheme. Smoothed maximum likelihood is used for local estimation of edge pixels from mixture parametric models of local templates. For the complementary smooth part the traditional L2-variational problem is solved in the Fourier domain with Thin Plate Spline (TPS) regularization. It is well known that naive Fourier compression of the whole image fails to restore a piece-wise smooth noisy image satisfactorily due to Gibbs phenomenon. Images are inte…
Learning to control an environment without hand-crafted rewards or expert data remains challenging and is at the frontier of reinforcement learning research. We present an unsupervised learning algorithm to train agents to achieve perceptually-specified goals using only a stream of observations and actions. Our agent simultaneously learns a goal-conditioned policy and a goal achievement reward function that measures how similar a state is to the goal state. This dual optimization leads to a co-operative game, giving rise to a learned reward function that reflects similarity in controllable asp…
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
In this work, we tackle the problem of transform-invariant unsupervised learning in the space of Covariance matrices and applications thereof. We begin by introducing the Spectral Polytope Covariance Matrix (SPCM) Similarity function; a similarity function for Covariance matrices, invariant to any type of transformation. We then derive the SPCM-CRP mixture model, a transform-invariant non-parametric clustering approach for Covariance matrices that leverages the proposed similarity function, spectral embedding and the distance-dependent Chinese Restaurant Process (dd-CRP) (Blei and Frazier, 201…
Microsoft Research Asia（微软亚洲研究院）；Tsinghua University（清华大学）；Xi’an Jiaotong University（西安交通大学）
This paper presents parametric instance classification (PIC) for unsupervised visual feature learning. Unlike the state-of-the-art approaches which do instance discrimination in a dual-branch non-parametric fashion, PIC directly performs a one-branch parametric instance classification, revealing a simple framework similar to supervised classification and without the need to address the information leakage issue. We show that the simple PIC framework can be as effective as the state-of-the-art approaches, i.e. SimCLR and MoCo v2, by adapting several common component settings used in the state-o…
UC Berkeley（加州大学伯克利分校）；ICSI（国际计算机科学研究所）；Chinese University of Hong Kong（香港中文大学）；Amazon Rekognition（亚马逊 Rekognition）
Neural net classifiers trained on data with annotated class labels can also capture apparent visual similarity among categories without being directed to do so. We study whether this observation can be extended beyond the conventional domain of supervised learning: Can we learn a good feature representation that captures apparent similarity among instances, instead of classes, by merely asking the feature to be discriminative of individual instances? We formulate this intuition as a non-parametric classification problem at the instance-level, and use noise-contrastive estimation to tackle the …
Télécom Paris, Institut Polytechnique de Paris（巴黎理工学院巴黎电信学院）；Sonos Inc.（搜诺思公司）
Supervised deep learning approaches to underdetermined audio source separation achieve state-of-the-art performance but require a dataset of mixtures along with their corresponding isolated source signals. Such datasets can be extremely costly to obtain for musical mixtures. This raises a need for unsupervised methods. We propose a novel unsupervised model-based deep learning approach to musical source separation. Each source is modelled with a differentiable parametric source-filter model. A neural network is trained to reconstruct the observed mixture as a sum of the sources by estimating th…
Recently, $k$NN-MT has shown the promising capability of directly incorporating the pre-trained neural machine translation (NMT) model with domain-specific token-level $k$-nearest-neighbor ($k$NN) retrieval to achieve domain adaptation without retraining. Despite being conceptually attractive, it heavily relies on high-quality in-domain parallel corpora, limiting its capability on unsupervised domain adaptation, where in-domain parallel corpora are scarce or nonexistent. In this paper, we propose a novel framework that directly uses in-domain monolingual sentences in the target language to con…
The Graduate University for Advanced Studies, SOKENDAI（综合研究大学院大学）；The Institute of Statistical Mathematics, Research Organization of Information and Systems（统计数理研究所，信息与系统研究机构）；National Institute for Materials Science（国立材料科学研究所）；National Institute of Advanced Industrial Science and Technology（国立产业技术综合研究所）
In 1869, the first draft of the periodic table was published by Russian chemist Dmitri Mendeleev. In terms of data science, his achievement can be viewed as a successful example of feature embedding based on human cognition: chemical properties of all known elements at that time were compressed onto the two-dimensional grid system for tabular display. In this study, we seek to answer the question of whether machine learning can reproduce or recreate the periodic table by using observed physicochemical properties of the elements. To achieve this goal, we developed a periodic table generator (PT…
In spectral clustering and spectral image segmentation, the data is partioned starting from a given matrix of pairwise similarities S. the matrix S is constructed by hand, or learned on a separate training set. In this paper we show how to achieve spectral clustering in unsupervised mode. Our algorithm starts with a set of observed pairwise features, which are possible components of an unknown, parametric similarity function. This function is learned iteratively, at the same time as the clustering of the data. The algorithm shows promosing results on synthetic and real data.
Yale University（耶鲁大学）；Siemens Medical Solutions USA, Inc.（美国西门子医疗解决方案公司）
Subject motion in whole-body dynamic PET introduces inter-frame mismatch and seriously impacts parametric imaging. Traditional non-rigid registration methods are generally computationally intense and time-consuming. Deep learning approaches are promising in achieving high accuracy with fast speed, but have yet been investigated with consideration for tracer distribution changes or in the whole-body scope. In this work, we developed an unsupervised automatic deep learning-based framework to correct inter-frame body motion. The motion estimation network is a convolutional neural network with a c…
Skoltech（斯科尔科沃科技学院）；IITP RAS（俄罗斯科学院信息传输问题研究所）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）
Cohort analysis is a pervasive activity in web analytics. One divides users into groups according to specific criteria and tracks their behavior over time. Despite its extensive use, academic circles do not discuss cohort analysis to evaluate user behavior online. This work introduces an unsupervised non-parametric approach to group Internet users based on their activities. In comparison, canonical methods in marketing and engineering-based techniques underperform. COHORTNEY is the first machine learning-based cohort analysis algorithm with a robust theoretical explanation.
An automatic mouse behavior recognition system can considerably reduce the workload of experimenters and facilitate the analysis process. Typically, supervised approaches, unsupervised approaches and semi-supervised approaches are applied for behavior recognition purpose under a setting which has all of predefined behaviors. In the real situation, however, as mouses can show various types of behaviors, besides the predefined behaviors that we want to analyze, there are many undefined behaviors existing. Both supervised approaches and conventional semi-supervised approaches cannot identify thes…
One of the central problems in massive Internet of Things (IoT) deployments is the monitoring of the status of a massive number of links. The problem is aggravated by the irregularity of the traffic transmitted over the link, as the traffic intermittency can be disguised as a link failure and vice versa. In this work we present a traffic model for IoT devices running quasi-periodic applications and we present both supervised and unsupervised machine learning methods for monitoring the network performance of IoT deployments with quasi-periodic reporting, such as smart-metering, environmental mo…
Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspective
University of Chinese Academy of Sciences（中国科学院大学）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；Microsoft（微软公司）；Stony Brook University（石溪大学）；Kuaishou Technology（快手科技）；Tsinghua University（清华大学）
