为遗忘赋予面孔：持续学习与机制可解释性相遇
Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretability
AI总结：
本文提出一个机制框架，从几何角度解释持续学习中的灾难性遗忘，通过玩具模型分析并验证了深度对遗忘的影响，展示了如何利用Crosscoders分析实际模型。
摘要：
在持续学习中，灾难性遗忘通常通过性能或最后一层表示层面来衡量，忽略了底层机制。我们引入了一个机制框架，从几何角度解释灾难性遗忘作为对个体特征编码转换的结果。这些转换可以通过减少特征分配的容量或通过下游计算破坏其读取来导致遗忘。对一个可处理的玩具模型的分析正式化了这一观点，使我们能够识别最佳和最差的情况。通过在该模型上的实验，我们经验性地测试了我们的正式分析，并突显了深度的有害影响。最后，我们通过使用Crosscoders展示了我们的框架如何用于实际模型的分析，通过一个在顺序CIFAR-10上训练的视觉Transformer的案例研究。我们的工作为持续学习提供了一个新的、以特征为中心的词汇。
英文摘要：
Catastrophic forgetting in continual learning is often measured at the performance or last-layer representation level, overlooking the underlying mechanisms. We introduce a mechanistic framework that offers a geometric interpretation of catastrophic forgetting as the result of transformations to the encoding of individual features. These transformations can lead to forgetting by reducing the allocated capacity of features or by disrupting their readout by downstream computations. Analysis of a tractable toy model formalizes this view, allowing us to identify best- and worst-case scenarios. Through experiments on this model, we empirically test our formal analysis and highlight the detrimental effect of depth. Finally, we demonstrate how our framework can be used in the analysis of practical models through the use of Crosscoders. We do so through a case study example of a Vision Transformer trained on sequential CIFAR-10. Our work provides a new, feature-centric vocabulary for continual learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
