arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.08148cs.AIcs.CL
收藏

神经符号推理策略的实证研究

Empirical Investigation of Neural Symbolic Reasoning Strategies

Tohoku University(东北大学);Langsmith, Inc.(Langsmith公司);RIKEN(理化学研究所)

Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui

摘要:

当生成中间推理步骤时,神经推理准确性提高。然而,这种改善的来源尚不清楚。在这里,我们调查和分解的好处,产生符号推理的中间步骤。具体来说,我们分解的推理策略w.r.t.步骤粒度和链接策略。对于纯符号数值推理数据集(例如,A=1,B=3,C=A+3,C?),我们发现推理策略的选择显著地影响性能,随着外推长度变长,差距变得更大。令人惊讶的是,我们还发现,某些配置导致几乎完美的性能,即使在长度外推的情况下。我们的研究结果表明,进一步探索有效的策略神经推理模型的重要性。

英文摘要:

Neural reasoning accuracy improves when generating intermediate reasoning steps. However, the source of this improvement is yet unclear. Here, we investigate and factorize the benefit of generating intermediate steps for symbolic reasoning. Specifically, we decompose the reasoning strategy w.r.t. step granularity and chaining strategy. With a purely symbolic numerical reasoning dataset (e.g., A=1, B=3, C=A+3, C?), we found that the choice of reasoning strategies significantly affects the performance, with the gap becoming even larger as the extrapolation length becomes longer. Surprisingly, we also found that certain configurations lead to nearly perfect performance, even in the case of length extrapolation. Our results indicate the importance of further exploring effective strategies for neural reasoning models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑