arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.04049cs.LG
收藏

并行算法与神经执行相一致

Parallel Algorithms Align with Neural Execution

University of Augsburg(奥格斯堡大学);University of Cambridge(剑桥大学);Google DeepMind(谷歌DeepMind)

Valerie Engelmayer, Dobrik Georgiev, Petar Veličković

AI总结:

本文指出神经算法推理器的并行特性,提出通过教授并行算法以减少冗余计算和所需层数,在CLRS框架上验证了该方法能大幅缩短训练时间并提升预测性能。

摘要:

神经算法推理器是并行处理器。向其教授顺序算法违背了这一特性,导致其大量计算冗余。然而,并行算法可充分利用其计算能力,因此仅需更少的层即可执行。正如我们在 CLRS 框架上比较搜索、排序和寻找强连通分量的并行实现与顺序实现时所观察到的,这大幅缩短了训练时间。此外,并行版本实现了(通常显著)更优的预测性能。

英文摘要:

Neural algorithmic reasoners are parallel processors. Teaching them sequential algorithms contradicts this nature, rendering a significant share of their computations redundant. Parallel algorithms however may exploit their full computational power, therefore requiring fewer layers to be executed. This drastically reduces training times, as we observe when comparing parallel implementations of searching, sorting and finding strongly connected components to their sequential counterparts on the CLRS framework. Additionally, parallel versions achieve (often strongly) superior predictive performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑