LACE：持续学习中的损失自适应容量扩展
LACE: Loss-Adaptive Capacity Expansion for Continual Learning
AI总结：
LACE通过监控损失信号动态扩展模型表示能力，实现持续学习中的自适应容量调整，实验显示其在域边界处精确触发扩展，且在资源受限环境下有效。
摘要：
在持续学习中，固定表示能力是一个基本限制：从业者必须在训练前猜测合适的模型宽度，而无法预知数据中包含多少种不同概念。我们提出了LACE（损失自适应容量扩展），一种简单的在线机制，通过监控自身损失信号在训练过程中扩展模型的表示能力。当持续损失偏差超过阈值——表明当前容量不足以处理新遇到的数据时，LACE会向投影层添加新维度并联合训练现有参数。在合成和真实数据实验中，LACE仅在域边界处触发扩展（100%边界精度，零误报），在从少量维度开始时能匹配大固定容量模型的准确性，并产生对性能至关重要的适配器维度（移除所有适配器时准确率下降3%）。我们进一步通过逐层聚类在GPT-2激活中展示无监督域分离，显示跨层的U形分离曲线，这激励了深度网络中的自适应容量分配。LACE无需标签、无需重放缓冲区和外部控制器，使其适合资源受限下的设备持续学习。
英文摘要：
Fixed representational capacity is a fundamental constraint in continual learning: practitioners must guess an appropriate model width before training, without knowing how many distinct concepts the data contains. We propose LACE (Loss-Adaptive Capacity Expansion), a simple online mechanism that expands a model's representational capacity during training by monitoring its own loss signal. When sustained loss deviation exceeds a threshold - indicating that the current capacity is insufficient for newly encountered data - LACE adds new dimensions to the projection layer and trains them jointly with existing parameters. Across synthetic and real-data experiments, LACE triggers expansions exclusively at domain boundaries (100% boundary precision, zero false positives), matches the accuracy of a large fixed-capacity model while starting from a fraction of its dimensions, and produces adapter dimensions that are collectively critical to performance (3% accuracy drop when all adapters removed). We further demonstrate unsupervised domain separation in GPT-2 activations via layer-wise clustering, showing a U-shaped separability curve across layers that motivates adaptive capacity allocation in deep networks. LACE requires no labels, no replay buffers, and no external controllers, making it suitable for on-device continual learning under resource constraints.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
