Unsupervised extraction of objects from low-level visual data is an important goal for further progress in machine learning. Existing approaches for representing objects without labels use structured generative models with static images. These methods focus a large amount of their capacity on reconstructing unimportant background pixels, missing low contrast or small objects. Conversely, we present a new method that avoids losses in pixel space and over-reliance on the limited signal a static image provides. Our approach takes advantage of objects' motion by learning a discriminative, time-contrastive loss in the space of slot representations, attempting to force each slot to not only capture entities that move, but capture distinct objects from the other slots. Moreover, we introduce a new quantitative evaluation metric to measure how "diverse" a set of slot vectors are, and use it to evaluate our model on 20 Atari games.
University of Amsterdam（阿姆斯特丹大学）；Google Research（谷歌研究院）
Contrastive, self-supervised learning of object representations recently emerged as an attractive alternative to reconstruction-based training. Prior approaches focus on contrasting individual object representations (slots) against one another. However, a fundamental problem with this approach is that the overall contrastive loss is the same for (i) representing a different object in each slot, as it is for (ii) (re-)representing the same object in all slots. Thus, this objective does not inherently push towards the emergence of object-centric representations in the slots. We address this prob…
Pohang University of Science and Technology (POSTECH)（浦项科技大学）
Zero-shot slot filling has received considerable attention to cope with the problem of limited available data for the target domain. One of the important factors in zero-shot learning is to make the model learn generalized and reliable representations. For this purpose, we present mcBERT, which stands for momentum contrastive learning with BERT, to develop a robust zero-shot slot filling model. mcBERT uses BERT to initialize the two encoders, the query encoder and key encoder, and is trained by applying momentum contrastive learning. Our experimental results on the SNIPS benchmark show that mc…
Contrastive Training of Complex-Valued Autoencoders for Object Discovery
The Swiss AI Lab, IDSIA, USI & SUPSI（瑞士AI实验室，IDSIA，USI与SUPSI）；Center for Brain Science, Harvard University（哈佛大学脑科学中心）；AI Initiative, KAUST（阿卜杜拉国王科技大学AI计划）
Harbin Institute of Technology（哈尔滨工业大学）；Northwestern University（西北大学）；Alibaba Group（阿里巴巴集团）
In this paper, we explore the slot tagging with only a few labeled support sentences (a.k.a. few-shot). Few-shot slot tagging faces a unique challenge compared to the other few-shot classification problems as it calls for modeling the dependencies between labels. But it is hard to apply previously learned label dependencies to an unseen domain, due to the discrepancy of label sets. To tackle this, we introduce a collapsed dependency transfer mechanism into the conditional random field (CRF) to transfer abstract label dependency patterns as transition scores. In the few-shot setting, the emissi…
There are a wide range of applications that involve multi-modal data, such as cross-modal retrieval, visual question-answering, and image captioning. Such applications are primarily dependent on aligned distributions of the different constituent modalities. Existing approaches generate latent embeddings for each modality in a joint fashion by representing them in a common manifold. However these joint embedding spaces fail to sufficiently reduce the modality gap, which affects the performance in downstream tasks. We hypothesize that these embeddings retain the intra-class relationships but are…
SIGMMA: Hierarchical Graph-Based Multi-Scale Multi-modal Contrastive Alignment of Histopathology Image and Spatial Transcriptome
Wellcome Sanger Institute（沃森桑格研究所）；Cambridge Centre for AI in Medicine（剑桥人工智能医学中心）；Institute of AI for Health（人工智能与健康研究所）；Cambridge Stem Cell Institute（剑桥干细胞研究所）
Contrastive Learning and Mixture of Experts Enables Precise Vector Embeddings
Center for Bioinformatics and Computational Biology, University of Delaware（特拉华大学生物信息与计算生物学中心）；University of Chicago（芝加哥大学）；The College of the University of Chicago（芝加哥大学学院）；Department of Biomedical Engineering, University of Delaware（特拉华大学生物医学工程系）；Luddy School of Informatics, Computing, and Engineering, Indiana University（印第安纳大学勒迪信息学、计算与工程学院）
本文提出基于共被引数据集和对比学习，将Mixture of Experts扩展应用于预训练BERT，显著提升了科学文本在多领域的向量嵌入精度与通用性。
Applying Self-supervised Learning to Network Intrusion Detection for Network Flows with Graph Neural Network
College of Computer and Information Engineering, Central South University of Forestry and Technology（中南林业科技大学计算机与信息工程学院）；School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）
