神经网络中的周期外推推广
Periodic Extrapolative Generalisation in Neural Networks
摘要：
在神经网络的强泛化研究中，最简单的计算模式--周期性的学习是一个公开的问题。我们正式的问题外推概括周期性信号和系统地调查的概括能力的经典，人口为基础的，最近提出的周期性架构上的一组基准任务。我们发现，周期性和“蛇”激活函数一贯失败，在定期外推，无论其周期性参数的可训练性。此外，我们的研究结果表明，传统的顺序模型仍然优于专门为外推设计的新架构，而这些又被基于群体的训练所击败。我们使我们的基准和评估工具包，PerKit，可用和易于访问，以促进该领域的未来工作。
英文摘要：
The learning of the simplest possible computational pattern -- periodicity -- is an open problem in the research of strong generalisation in neural networks. We formalise the problem of extrapolative generalisation for periodic signals and systematically investigate the generalisation abilities of classical, population-based, and recently proposed periodic architectures on a set of benchmarking tasks. We find that periodic and "snake" activation functions consistently fail at periodic extrapolation, regardless of the trainability of their periodicity parameters. Further, our results show that traditional sequential models still outperform the novel architectures designed specifically for extrapolation, and that these are in turn trumped by population-based training. We make our benchmarking and evaluation toolkit, PerKit, available and easily accessible to facilitate future work in the area.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
