arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2101.11802cs.CL
收藏

用于数值推理的弱监督神经符号模块网络

Weakly Supervised Neuro-Symbolic Module Networks for Numerical Reasoning

Salesforce AI Research(赛富时人工智能研究院)

Amrita Saha, Shafiq Joty, Steven C. H. Hoi

摘要:

神经模块网络(NMN)已经非常成功地将显式推理作为各种问答任务中的可学习模块,包括机器阅读理解(MRC)中文本上最通用的数值推理形式。然而,为了实现这一点,当代NMN需要强有力的监督,在执行查询作为一个专门的程序推理模块,并没有推广到更开放的设置没有这样的监督。因此,我们提出了弱监督神经符号模块网络(WNSMN)作为基于MRC的数值推理的唯一监督。它学习执行从查询的依存句法分析获得的噪声启发式程序,作为神经和符号推理模块上的离散动作,并在强化学习框架中端到端地训练它,并从答案匹配中获得离散奖励。在DROP的数字答案子集上,WNSMN在可比的弱监督设置下训练时,在精确匹配准确率方面比NMN高出32%,比无推理语言模型GenBERT高出8%。这展示了模块化网络的有效性和可推广性,它可以以端到端的方式处理噪声程序上的显式离散推理。

英文摘要:

Neural Module Networks (NMNs) have been quite successful in incorporating explicit reasoning as learnable modules in various question answering tasks, including the most generic form of numerical reasoning over text in Machine Reading Comprehension (MRC). However, to achieve this, contemporary NMNs need strong supervision in executing the query as a specialized program over reasoning modules and fail to generalize to more open-ended settings without such supervision. Hence we propose Weakly-Supervised Neuro-Symbolic Module Network (WNSMN) trained with answers as the sole supervision for numerical reasoning based MRC. It learns to execute a noisy heuristic program obtained from the dependency parsing of the query, as discrete actions over both neural and symbolic reasoning modules and trains it end-to-end in a reinforcement learning framework with discrete reward from answer matching. On the numerical-answer subset of DROP, WNSMN out-performs NMN by 32% and the reasoning-free language model GenBERT by 8% in exact match accuracy when trained under comparable weak supervised settings. This showcases the effectiveness and generalizability of modular networks that can handle explicit discrete reasoning over noisy programs in an end-to-end manner.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑