现代Hopfield网络和注意力在免疫库分类中的应用
Modern Hopfield Networks and Attention for Immune Repertoire Classification
摘要：
机器学习的核心机制是识别、存储和识别模式。如何学习、访问和检索这些模式在Hopfield网络和最近的Transformer架构中至关重要。我们发现，Transformer架构的注意力机制实际上是现代Hopfield网络的更新规则，可以存储指数级的许多模式。我们利用现代Hopfield网络的高存储容量来解决计算生物学中具有挑战性的多实例学习（MIL）问题：免疫库分类。解决这一问题的准确和可解释的机器学习方法可以为新疫苗和疗法铺平道路，这是目前因COVID-19危机而加剧的一个非常相关的研究课题。基于个体的大量免疫序列的免疫库分类是具有前所未有的大量实例的MIL问题，比目前考虑的问题大两个数量级，并且具有极低的见证率。在这项工作中，我们提出了我们的新方法DeepRC，它将类似transformer的注意力或等效的现代Hopfield网络集成到深度学习架构中，用于大规模MIL，如免疫库分类。我们证明，DeepRC在大规模实验（包括模拟和真实世界的病毒感染数据）的预测性能方面优于所有其他方法，并能够提取与给定疾病类别相关的序列基序。源代码和数据集：https://github.com/ml-jku/DeepRC
英文摘要：
A central mechanism in machine learning is to identify, store, and recognize patterns. How to learn, access, and retrieve such patterns is crucial in Hopfield networks and the more recent transformer architectures. We show that the attention mechanism of transformer architectures is actually the update rule of modern Hopfield networks that can store exponentially many patterns. We exploit this high storage capacity of modern Hopfield networks to solve a challenging multiple instance learning (MIL) problem in computational biology: immune repertoire classification. Accurate and interpretable machine learning methods solving this problem could pave the way towards new vaccines and therapies, which is currently a very relevant research topic intensified by the COVID-19 crisis. Immune repertoire classification based on the vast number of immunosequences of an individual is a MIL problem with an unprecedentedly massive number of instances, two orders of magnitude larger than currently considered problems, and with an extremely low witness rate. In this work, we present our novel method DeepRC that integrates transformer-like attention, or equivalently modern Hopfield networks, into deep learning architectures for massive MIL such as immune repertoire classification. We demonstrate that DeepRC outperforms all other methods with respect to predictive performance on large-scale experiments, including simulated and real-world virus infection data, and enables the extraction of sequence motifs that are connected to a given disease class. Source code and datasets: https://github.com/ml-jku/DeepRC
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
