诊断灾难：持续学习中的大部分准确率损失可由读出层失配解释
Diagnosing Catastrophe: Large parts of accuracy loss in continual learning can be accounted for by readout misalignment
AI总结：
本研究针对持续学习中的灾难性遗忘问题，发现准确率损失主要源于隐藏表征与读出层的失配，表征几何结构部分保留、仅少量信息不可逆丢失，且变化程度与隐藏表征维度相关，为深度学习持续更新及对齐生物视觉提供了启示。
摘要：
与灵长类动物不同，在不断变化的数据分布上训练人工神经网络会导致其在旧任务上的性能快速下降，这一现象通常被称为灾难性遗忘。本文研究了导致这种性能下降的表征变化，并确定了共同造成该现象的三个不同过程。其中最主要的因素是隐藏表征与读出（readout）层之间的失配：失配由学习额外任务引发，会导致内部表征发生偏移。在这种失配下，表征的几何结构得到了部分保留，仅有一小部分信息会不可逆地丢失。所有类型的表征变化都与隐藏表征的维度呈正相关。这些发现不仅对需要持续更新的深度学习应用具有启示意义，还有助于让人工神经网络（ANN）模型与相当稳健的生物视觉系统对齐。
英文摘要：
Unlike primates, training artificial neural networks on changing data distributions leads to a rapid decrease in performance on old tasks. This phenomenon is commonly referred to as catastrophic forgetting. In this paper, we investigate the representational changes that underlie this performance decrease and identify three distinct processes that together account for the phenomenon. The largest component is a misalignment between hidden representations and readout layers. Misalignment occurs due to learning on additional tasks and causes internal representations to shift. Representational geometry is partially conserved under this misalignment and only a small part of the information is irrecoverably lost. All types of representational changes scale with the dimensionality of hidden representations. These insights have implications for deep learning applications that need to be continuously updated, but may also aid aligning ANN models to the rather robust biological vision.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
