学习将程序编译为神经网络
Learning to Compile Programs to Neural Networks
AI总结:
提出神经替代品编译技术,利用超网络直接从程序文本生成神经替代品,解耦生成与执行,提升数据效率、结果相似度并减少训练轮数。
摘要:
程序的神经替代品(neural surrogate)是一种模仿程序行为的神经网络。研究人员利用这些神经替代品来自动调整程序输入、使程序适应新环境以及加速计算。传统上,研究人员通过从单个程序的输入输出示例中训练来开发神经替代品。另一种方法是,在包含许多程序的大型数据集上训练的语言模型可以消费程序文本,充当神经替代品。然而,使用语言模型既生成替代品又充当替代品,会导致资源消耗与准确性之间的权衡。我们提出了神经替代品编译(neural surrogate compilation),这是一种直接从程序文本生成神经替代品的技术,无需将神经替代品的生成与执行耦合。我们使用在C程序数据集上训练的超网络实现了神经替代品编译器,并发现它们生成的神经替代品在数据效率上提高了1.9至9.5倍,产生的视觉结果与真实值的相似度提高了1.0至1.3倍,并且训练所需的轮数比从头训练的神经替代品减少了4.3至7.3倍。
英文摘要:
A $\textit{neural surrogate of a program}$ is a neural network that mimics the behavior of a program. Researchers have used these neural surrogates to automatically tune program inputs, adapt programs to new settings, and accelerate computations. Researchers traditionally develop neural surrogates by training on input-output examples from a single program. Alternatively, language models trained on a large dataset including many programs can consume program text, to act as a neural surrogate. Using a language model to both generate a surrogate and act as a surrogate, however, leading to a trade-off between resource consumption and accuracy. We present $\textit{neural surrogate compilation}$, a technique for producing neural surrogates directly from program text without coupling neural surrogate generation and execution. We implement neural surrogate compilers using hypernetworks trained on a dataset of C programs and find that they produce neural surrogates that are $1.9$-$9.5\times$ as data-efficient, produce visual results that are $1.0$-$1.3\times$ more similar to ground truth, and train in $4.3$-$7.3\times$ fewer epochs than neural surrogates trained from scratch.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。