arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.08162cs.LG
收藏

关于Adam优化器的一个猜想

On a Conjecture Regarding the Adam Optimizer

AIP Labs(AIP实验室);University of Toronto(多伦多大学)

Mohamed Akrout, Douglas Tweed

摘要:

为什么Adam优化器在深度学习应用中表现如此出色?亚当的创始人,金马和巴,提出了一个数学论点,旨在帮助解释其成功,但博克和同事们报告说,一个关键的一块是失踪的论点$-$一个未经证明的引理,我们将称为博克猜想。在这里,我们表明,这一猜想是假的,但我们证明了一个修改后的版本,它$-$一个概括的结果Reddi和同事$-$,它可以取代亚当的分析。

英文摘要:

Why does the Adam optimizer work so well in deep-learning applications? Adam's originators, Kingma and Ba, presented a mathematical argument that was meant to help explain its success, but Bock and colleagues have since reported that a key piece is missing from that argument $-$ an unproven lemma which we will call Bock's conjecture. Here we show that this conjecture is false, but we prove a modified version of it $-$ a generalization of a result of Reddi and colleagues $-$ which can take its place in analyses of Adam.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑