arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.06805cs.LGstat.ML
收藏

神经定理证明器在没有探索的情况下不会学习规则

Neural Theorem Provers Do Not Learn Rules Without Exploration

University of Southern California(南加州大学);Google AI(谷歌人工智能)

Michiel de Jong, Fei Sha

摘要:

神经符号处理旨在将逻辑学习方法的推广和神经网络的性能相结合。Rocktaschel等人(2017)的神经定理证明(NTP)模型学习概念的嵌入并执行逻辑统一。虽然NTP在准确预测事实方面很有前途和有效,但我们对它如何提取数据之间的真实关系知之甚少。为此,我们创建了具有注入关系的合成逻辑数据集,这些数据集可以实时生成,以测试包括NTP在内的基于神经的关系学习算法。我们表明,它很难恢复关系,但最简单的设置。批判性分析和诊断实验表明,优化算法由于其贪婪的赢家通吃的策略,在确定最丰富的结构(证明路径)追求从穷人的局部极小值。我们改变了NTP算法,以增加探索,这大大提高了性能。我们认为并证明,这是有见地的基准与地面真理关系的合成数据,评估模型和揭示算法问题。

英文摘要:

Neural symbolic processing aims to combine the generalization of logical learning approaches and the performance of neural networks. The Neural Theorem Proving (NTP) model by Rocktaschel et al (2017) learns embeddings for concepts and performs logical unification. While NTP is promising and effective in predicting facts accurately, we have little knowledge how well it can extract true relationship among data. To this end, we create synthetic logical datasets with injected relationships, which can be generated on-the-fly, to test neural-based relation learning algorithms including NTP. We show that it has difficulty recovering relationships in all but the simplest settings. Critical analysis and diagnostic experiments suggest that the optimization algorithm suffers from poor local minima due to its greedy winner-takes-all strategy in identifying the most informative structure (proof path) to pursue. We alter the NTP algorithm to increase exploration, which sharply improves performance. We argue and demonstate that it is insightful to benchmark with synthetic data with ground-truth relationships, for both evaluating models and revealing algorithmic issues.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑