Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion
National Institute of Health Data Science, Peking University（北京大学健康医疗大数据国家研究院）；Institute of Medical Technology, Health Science Center of Peking University（北京大学医学部医学技术研究院）
Unsupervised/self-supervised time series representation learning is a challenging problem because of its complex dynamics and sparse annotations. Existing works mainly adopt the framework of contrastive learning with the time-based augmentation techniques to sample positives and negatives for contrastive training. Nevertheless, they mostly use segment-level augmentation derived from time slicing, which may bring about sampling bias and incorrect optimization with false negatives due to the loss of global context. Besides, they all pay no attention to incorporate the spectral information in feature representation. In this paper, we propose a unified framework, namely Bilinear Temporal-Spectral Fusion (BTSF). Specifically, we firstly utilize the instance-level augmentation with a simple dropout on the entire time series for maximally capturing long-term dependencies. We devise a novel iterative bilinear temporal-spectral fusion to explicitly encode the affinities of abundant time-frequency pairs, and iteratively refines representations in a fusion-and-squeeze manner with Spectrum-to-Time (S2T) and Time-to-Spectrum (T2S) Aggregation modules. We firstly conducts downstream evaluations on three major tasks for time series including classification, forecasting and anomaly detection. Experimental results shows that our BTSF consistently significantly outperforms the state-of-the-art methods.
Carnegie Mellon University（卡内基梅隆大学）；Sun Yat-sen University（中山大学）；University of Cambridge（剑桥大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Sea AI Lab（Sea人工智能实验室）；Mila-Québec AI Institute（米拉-魁北克人工智能研究所）；HEC Montréal（蒙特利尔高等商学院）
IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation
University of Hong Kong（香港大学）；MIT（麻省理工学院）；Amazon Web Services（亚马逊云科技）；ASAPP Inc.（ASAPP公司）
文本属性转移的目的是自动重写句子，使它们具有某些语言属性，同时保持其语义内容。由于缺乏监督并行数据，这项任务仍然具有挑战性。现有的方法试图明确地解开内容和属性信息，但这是困难的，往往会导致不良的内容保存和不合语法。相比之下，我们提出了一种更简单的方法，迭代匹配和翻译（Iterative Matching and Translation，IMAT），它：（1）通过对齐源语料库和目标语料库中语义相似的句子的子集来构建伪平行语料库;（2）应用标准的序列到序列模型来学习属性转移;（3）通过改进对齐中的缺陷来迭代地改进学习的转移函数。在情感修改和形式转换任务中，我们的方法大大优于复杂的最先进的系统。作为一个辅助的贡献，我们产生了一个公开的测试集与人类生成的传输参考。
IterMask3D: Unsupervised Anomaly Detection and Segmentation with Test-Time Iterative Mask Refinement in 3D Brain MR
Department of Engineering Science, University of Oxford, UK（工程科学系，牛津大学）；Department of Computer Science, Hong Kong Baptist University（计算机科学系，香港 Baptist 大学）；Nuffield Department of Clinical Neurosciences, University of Oxford（临床神经科学系，牛津大学）；Department of Neuroradiology, Oxford University Hospitals NHS Foundation Trust（放射学系，牛津大学医院 NHS 基础研究信托）
Mutual information neural estimation for unsupervised multi-modal registration of brain images
Australian Institute for Machine Learning（澳大利亚机器学习研究所）；University of Adelaide（阿德莱德大学）；South Australian Health and Medical Research Institute（南澳大利亚健康与医学研究所）
Shuffle and Learn: Minimizing Mutual Information for Unsupervised Hashing
University of British Columbia（不列颠哥伦比亚大学）
无监督二进制表示允许快速数据检索而无需任何注释，从而实现快速人员重新识别和多媒体检索等实际应用。有人认为，二进制空间中的冲突是高性能无监督哈希的主要障碍之一，因为目前的方法未能捕捉到精确的代码冲突在整个域。提出了一种新的松弛方法Shuffle and Learn来解决无监督哈希中的代码冲突。联合概率的近似导数和二进制层的梯度被引入以桥接从散列到输入的更新。给出了$ε$ --联合概率逼近导数收敛性的证明，保证了互信息更新的精确性。所提出的算法进行迭代全局更新，以最小化互信息，发散代码之前，定期无监督优化。实验表明，所提出的方法可以放松局部最优的代码优化，并有助于生成更有区别性和信息量的二进制表示，没有任何注释。在三个开放数据集上进行了无监督二进制代码图像检索的性能基准测试，并且该模型在所有这些数据集上的图像检索任务上都达到了最先进的精度。提供了数据集和可复制的代码。
