arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.02830cs.CLcs.AIcs.LG
收藏

PRover:规则上可解释推理的证明生成

PRover: Proof Generation for Interpretable Reasoning over Rules

UNC Chapel Hill(北卡罗来纳大学教堂山分校)

Swarnadeep Saha, Sayan Ghosh, Shashank Srivastava, Mohit Bansal

摘要:

Clark等人(2020)的最新研究表明,Transformers可以通过回答自然语言中显式提供的知识的问题来充当“软定理证明器”。在我们的工作中,我们通过提出PROVER,一个可解释的基于transformer的模型,共同回答规则库上的二元问题,并生成相应的证明,更接近于模拟形式定理证明器。我们的模型学习预测节点和边缘对应的证明图在一个有效的约束训练范式。在推理过程中,生成满足一组全局约束的有效证明。我们进行实验合成,手工创作,和人类释义的规则库,以显示有前途的结果QA和证明生成,具有较强的泛化性能。首先,与RuleTakers相比,PROVER以87%的准确率生成证明,同时保留或提高QA任务的性能(zero-shot评估提高了6%)。第二,当在需要较低推理深度的问题上进行训练时,它可以更好地推广到更高的深度(高达15%的改进)。第三,PROVER仅使用40%的训练数据就获得了98%的近乎完美的QA准确度。然而,为需要更高推理深度的问题生成证明变得具有挑战性,并且对于“深度5”,准确率下降到65%,这表明未来工作的重要范围。我们的代码和模型可在https://github.com/swarnaHub/PRover上公开获取

英文摘要:

Recent work by Clark et al. (2020) shows that transformers can act as 'soft theorem provers' by answering questions over explicitly provided knowledge in natural language. In our work, we take a step closer to emulating formal theorem provers, by proposing PROVER, an interpretable transformer-based model that jointly answers binary questions over rule-bases and generates the corresponding proofs. Our model learns to predict nodes and edges corresponding to proof graphs in an efficient constrained training paradigm. During inference, a valid proof, satisfying a set of global constraints is generated. We conduct experiments on synthetic, hand-authored, and human-paraphrased rule-bases to show promising results for QA and proof generation, with strong generalization performance. First, PROVER generates proofs with an accuracy of 87%, while retaining or improving performance on the QA task, compared to RuleTakers (up to 6% improvement on zero-shot evaluation). Second, when trained on questions requiring lower depths of reasoning, it generalizes significantly better to higher depths (up to 15% improvement). Third, PROVER obtains near perfect QA accuracy of 98% using only 40% of the training data. However, generating proofs for questions requiring higher depths of reasoning becomes challenging, and the accuracy drops to 65% for 'depth 5', indicating significant scope for future work. Our code and models are publicly available at https://github.com/swarnaHub/PRover

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑