Unsupervised discovery of latent representations, in addition to being useful for density modeling, visualisation and exploratory data analysis, is also increasingly important for learning features relevant to discriminative tasks. Autoencoders, in particular, have proven to be an effective way to learn latent codes that reflect meaningful variations in data. A continuing challenge, however, is guiding an autoencoder toward representations that are useful for particular tasks. A complementary challenge is to find codes that are invariant to irrelevant transformations of the data. The most common way of introducing such problem-specific guidance in autoencoders has been through the incorporation of a parametric component that ties the latent representation to the label information. In this work, we argue that a preferable approach relies instead on a nonparametric guidance mechanism. Conceptually, it ensures that there exists a function that can predict the label information, without explicitly instantiating that function. The superiority of this guidance mechanism is confirmed on two datasets. In particular, this approach is able to incorporate invariance information (lighting, elevation, etc.) from the small NORB object recognition dataset and yields state-of-the-art performance for a single layer, non-convolutional network.
Disentangled representations via score-based variational autoencoders
Center for Neural Science, New York University（纽约大学神经科学中心）；Center for Computational Neuroscience, Flatiron Institute（Flatiron研究所计算神经科学中心）；Center for Data Science, New York University（纽约大学数据科学中心）
CLVAE: A Variational Autoencoder for Long-Term Customer Revenue Forecasting
Research Institute for Statistics and Information Science, University of Geneva（统计与信息科学研究所，日内瓦大学）；Institute of Management, University of Geneva（管理学院，日内瓦大学）
MoCA: Multi-modal Cross-masked Autoencoder for Time Series in Digital Health
Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego, San Diego, CA, USA（赫伯特·韦瑟姆公共卫生与人类长寿科学学院，加州大学圣地亚哥分校）；Halıcıoğlu Data Science Institute, University of California San Diego, San Diego, CA, USA（哈利奇奥卢数据科学研究所，加州大学圣地亚哥分校）；Department of Computer Science and Engineering, University of California San Diego, San Diego, CA, USA（计算机科学与工程系，加州大学圣地亚哥分校）；Division of Public Health Sciences, Fred Hutchinson Cancer Center, Seattle, WA, USA（公共卫生科学部，Fred Hutchinson癌症中心）
