AdamX：余弦相似度与梯度下降的结合
AdamX: Cosine similarity meets gradient descent
浏览论文内容
中文总结 AI 辅助
AdamX是一种结合余弦相似度控制更新幅度的一阶优化器，通过方差修正促进早期平滑训练，在多种基准上实现竞争性收敛速度。
中文摘要 AI 辅助
我们介绍了AdamX，一种一阶优化器，它结合了余弦相似度作为控制更新幅度的自适应机制。所提出的方法具有可扩展性、模型无关性，并且易于集成到现有的训练流程中。我们进一步引入了一种方差修正方案，以促进训练早期阶段的更平滑优化。总体而言，我们提供了经验证据表明，AdamX在多种基准数据集和架构上实现了具有竞争力的收敛速度。性能通过固定超参数预算下达到预定义性能阈值所需的轮数来评估。代码和实验可在以下网址获取：此https URL。
英文摘要
We introduce AdamX, a first-order optimizer that incorporates cosine similarity as an adaptive mechanism for controlling update magnitudes. The proposed method is scalable, model-agnostic, and straightforward to integrate into existing training pipelines. We further introduce a variance rectification scheme that promotes smoother optimization during the early stages of training. Overall, we provide empirical evidence that AdamX achieves competitive convergence rates across a range of benchmark datasets and architectures. Performance is evaluated in terms of the number of epochs required to reach predefined performance thresholds under a fixed hyperparameter budget. Code and Experiments available at: https://github.com/FranciscoCaldas/adamX.
发表机构
- NOVA School of Science of Technology Universidade Nova de Lisboa, Caparica, Portugal
机构由 AI 辅助整理，请以论文原文为准。
