AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation
Aleph Alpha（阿列夫阿尔法）；Technical University Darmstadt（达姆施塔特工业大学）；Hessian Center for Artificial Intelligence (hessian.AI)（黑森州人工智能中心）；German Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；LAION
Björn Deiseroth, Mayukh Deb, Samuel Weinbach, Manuel Brack, Patrick Schramowski, Kristian Kersting
Generative transformer models have become increasingly complex, with large numbers of parameters and the ability to process multiple input modalities. Current methods for explaining their predictions are resource-intensive. Most crucially, they require prohibitively large amounts of extra memory, since they rely on backpropagation which allocates almost twice as much GPU memory as the forward pass. This makes it difficult, if not impossible, to use them in production. We present AtMan that provides explanations of generative transformer models at almost no extra cost. Specifically, AtMan is a modality-agnostic perturbation method that manipulates the attention mechanisms of transformers to produce relevance maps for the input with respect to the output prediction. Instead of using backpropagation, AtMan applies a parallelizable token-based search method based on cosine similarity neighborhood in the embedding space. Our exhaustive experiments on text and image-text benchmarks demonstrate that AtMan outperforms current state-of-the-art gradient-based methods on several metrics while being computationally efficient. As such, AtMan is suitable for use in large model inference deployments.
Microsoft（微软）；Center of Excellence in Artificial Intelligence (Deep Learning Brazil)（人工智能卓越中心（巴西深度学习中心））
近年来，Transformer架构及其变体在许多机器学习任务中取得了显著的成功。这种成功本质上与处理长序列的能力和来自注意力机制的上下文相关权重的存在有关。我们认为，这些功能适合的核心作用的Meta-强化学习算法。事实上，元RL代理需要从一系列轨迹中推断任务。此外，它需要一个快速的适应策略，以适应新的任务-这可以通过使用自我注意机制来实现。在这项工作中，我们提出了TrMRL（Transformers for Meta- Reinforcement Learning），一个元强化学习代理，它使用Transformer架构模仿内存恢复机制。它通过Transformer层将最近的工作记忆联系起来，递归地构建情景记忆。我们表明，自我注意力计算出一个共识表示，使每层的贝叶斯风险最小化，并提供有意义的特征来计算最佳行动。我们进行了实验，在高维连续控制环境中的运动和灵巧的操作。结果表明，在这些环境中，与基线相比，TrMRL具有相当或更好的渐近性能、样本效率和分布外泛化。
