SALR：用于提升泛化能力的锐度感知学习率调度器
SALR: Sharpness-aware Learning Rate Scheduler for Improved Generalization
AI总结：
提出SALR锐度感知学习率更新技术，通过动态调整学习率逃离尖锐谷地，提升深度学习泛化能力与收敛速度。
摘要：
为了改善深度学习的泛化能力并自动化学习率调度过程，我们提出了SALR：一种旨在恢复平坦极小值的锐度感知学习率更新技术。我们的方法基于损失函数的局部锐度，动态更新基于梯度的优化器的学习率。这使得优化器能够在尖锐谷地自动增加学习率，从而增加逃离这些区域的机会。我们展示了SALR在多种算法和广泛网络中的有效性。实验表明，SALR提升了泛化能力，加快了收敛速度，并驱使解到达显著更平坦的区域。
英文摘要：
In an effort to improve generalization in deep learning and automate the process of learning rate scheduling, we propose SALR: a sharpness-aware learning rate update technique designed to recover flat minimizers. Our method dynamically updates the learning rate of gradient-based optimizers based on the local sharpness of the loss function. This allows optimizers to automatically increase learning rates at sharp valleys to increase the chance of escaping them. We demonstrate the effectiveness of SALR when adopted by various algorithms over a broad range of networks. Our experiments indicate that SALR improves generalization, converges faster, and drives solutions to significantly flatter regions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
