面向持续学习的步长优化
Step-size Optimization for Continual Learning
AI总结：
针对持续学习中步长调整的问题，本文指出RMSProp、Adam等启发式步长自适应方法的缺陷，验证了IDBD元梯度下降算法可显式优化步长，并提出结合两类方法的未来研究方向。
摘要：
在持续学习中，学习器需要在整个生命周期内持续从数据中学习。一个核心问题是决定哪些知识需要保留、哪些知识需要舍弃。在神经网络中，这可以通过使用步长向量来调整梯度样本对网络权重的改变幅度来实现。常见算法（如RMSProp和Adam）采用启发式方法，具体来说是归一化，来自适应调整该步长向量。本文指出，这类启发式方法忽略了其自适应调整对整体目标函数的影响，例如会使步长向量偏离更优的步长向量。另一方面，随机元梯度下降算法（如IDBD (Sutton, 1992)）则针对整体目标函数显式优化步长向量。我们在简单问题上证明，IDBD能够持续优化步长向量，而RMSProp和Adam无法做到这一点。我们阐释了两种方法的差异及其各自的局限性。最后我们提出，结合两种方法是提升神经网络在持续学习中性能的一个有前景的未来方向。
英文摘要：
In continual learning, a learner has to keep learning from the data over its whole life time. A key issue is to decide what knowledge to keep and what knowledge to let go. In a neural network, this can be implemented by using a step-size vector to scale how much gradient samples change network weights. Common algorithms, like RMSProp and Adam, use heuristics, specifically normalization, to adapt this step-size vector. In this paper, we show that those heuristics ignore the effect of their adaptation on the overall objective function, for example by moving the step-size vector away from better step-size vectors. On the other hand, stochastic meta-gradient descent algorithms, like IDBD (Sutton, 1992), explicitly optimize the step-size vector with respect to the overall objective function. On simple problems, we show that IDBD is able to consistently improve step-size vectors, where RMSProp and Adam do not. We explain the differences between the two approaches and their respective limitations. We conclude by suggesting that combining both approaches could be a promising future direction to improve the performance of neural networks in continual learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
