神经网络的符号综合
Symbolic Synthesis of Neural Networks
摘要：
神经网络非常适合分布式和连续表示，但很难从少量数据中进行归纳。符号系统通常通过利用模块化来受益于表示的局部和离散特征来实现数据有效的泛化。这些功能允许符号程序一次改进一个模块，并在它们可以成功处理的值中经历组合增长。然而，很难设计一个组件，可以用来形成符号抽象，并充分过参数化学习任意高维变换。我提出了基于图的符号合成神经网络（G-SSNN），一类神经模块，其操作的表示与合成符号程序修改，包括一组固定的本地和离散的功能。我证明了G-SSNN模块中注入特征的选择调节了基线神经模型的数据效率和泛化能力，创建了可预测的增强和缩减泛化模式。通过训练G-SSNN，我们还可以获得有关符号程序的理想语义的信息，而无需手动工程。这些信息是紧凑的，易于抽象，但也可以灵活地重新组织为其他高维设置。在未来的工作中，我将研究数据的有效推广和转让的学习符号表示在更复杂的G-SSNN设计的基础上更复杂的类的符号程序。实验代码和数据可在https://github.com/shlomenu/symbolically_synthesized_networks上获得。
英文摘要：
Neural networks adapt very well to distributed and continuous representations, but struggle to generalize from small amounts of data. Symbolic systems commonly achieve data efficient generalization by exploiting modularity to benefit from local and discrete features of a representation. These features allow symbolic programs to be improved one module at a time and to experience combinatorial growth in the values they can successfully process. However, it is difficult to design a component that can be used to form symbolic abstractions and which is adequately overparametrized to learn arbitrary high-dimensional transformations. I present Graph-based Symbolically Synthesized Neural Networks (G-SSNNs), a class of neural modules that operate on representations modified with synthesized symbolic programs to include a fixed set of local and discrete features. I demonstrate that the choice of injected features within a G-SSNN module modulates the data efficiency and generalization of baseline neural models, creating predictable patterns of both heightened and curtailed generalization. By training G-SSNNs, we also derive information about desirable semantics of symbolic programs without manual engineering. This information is compact and amenable to abstraction, but can also be flexibly recontextualized for other high-dimensional settings. In future work, I will investigate data efficient generalization and the transferability of learned symbolic representations in more complex G-SSNN designs based on more complex classes of symbolic programs. Experimental code and data are available at https://github.com/shlomenu/symbolically_synthesized_networks .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
