动态持续学习：利用参数不确定性提升网络适应性
Dynamic Continual Learning: Harnessing Parameter Uncertainty for Improved Network Adaptation
AI总结：
针对深度神经网络微调时遗忘旧任务的问题，提出利用参数不确定性识别关键参数并正则化训练，通过低学习率和强正则化限制其变化，基于贝叶斯矩传播框架实现，在顺序基准数据集上提升了平均测试准确率和后向迁移性能。
摘要：
在将深度神经网络（DNNs）微调至新数据时，DNNs 容易覆盖先前学习任务中实现特定功能所需的网络参数，导致在这些任务上的性能下降。我们提出利用基于参数的不确定性来确定哪些参数与网络已学函数相关，并正则化训练以防止这些重要参数发生变化。我们通过两种方式实现这种正则化：（1）通过将更关键参数与较低学习率关联，约束关键参数不发生显著变化，从而限制这些参数的改动；（2）通过施加更高的正则化权重来限制重要参数的变化，使参数恢复到学习后续任务之前的状态。我们利用贝叶斯矩传播框架，该框架在训练网络参数的同时学习其相关不确定性，并允许每个参数为网络的预测分布贡献不确定性，从而避免现有基于采样方法的缺陷。所提出的方法在常见的顺序基准数据集上进行了评估，并与持续学习社区中已发表的现有方法进行了比较。最终，我们展示了与基于采样的方法及其他非基于不确定性的方法相比，在平均测试准确率和后向迁移指标上持续学习性能的提升。
英文摘要：
When fine-tuning Deep Neural Networks (DNNs) to new data, DNNs are prone to overwriting network parameters required for task-specific functionality on previously learned tasks, resulting in a loss of performance on those tasks. We propose using parameter-based uncertainty to determine which parameters are relevant to a network's learned function and regularize training to prevent change in these important parameters. We approach this regularization in two ways: (1), we constrain critical parameters from significant changes by associating more critical parameters with lower learning rates, thereby limiting alterations in those parameters; (2), important parameters are restricted from change by imposing a higher regularization weighting, causing parameters to revert to their states prior to the learning of subsequent tasks. We leverage a Bayesian Moment Propagation framework which learns network parameters concurrently with their associated uncertainties while allowing each parameter to contribute uncertainty to the network's predictive distribution, avoiding the pitfalls of existing sampling-based methods. The proposed approach is evaluated for common sequential benchmark datasets and compared to existing published approaches from the Continual Learning community. Ultimately, we show improved Continual Learning performance for Average Test Accuracy and Backward Transfer metrics compared to sampling-based methods and other non-uncertainty-based approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
