SEER:通过强化学习促进结构化推理与解释
SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learning
AI总结:
本文提出SEER方法,通过最大化基于结构的回报和细粒度奖励函数来捕捉推理步骤间的层次与分支关系,从而促进问答系统中的结构化推理与解释,在多个基准上显著优于现有方法。
摘要:
通过从问题到答案的结构化解释来阐明推理过程至关重要,因为它能显著增强问答(QA)系统的可解释性、可追溯性和可信度。然而,结构化解释要求模型执行复杂的结构化推理,这带来了巨大挑战。现有大多数方法侧重于通过监督学习进行单步推理,忽略了步骤间的逻辑依赖关系。此外,现有的基于强化学习(RL)的方法忽视了结构化关系,未能在结构化推理中充分发挥RL的潜力。本文提出SEER,一种通过最大化基于结构的回报来促进结构化推理与解释的新方法。我们提出的基于结构的回报精确描述了结构化推理中固有的层次和分支结构,有效捕捉了不同推理步骤间的复杂关系。此外,我们引入了细粒度奖励函数,以细致刻画多样的推理步骤。大量实验表明,SEER显著优于最先进方法:在EntailmentBank上较基于RL的方法取得6.9%的绝对提升,在STREET基准上平均提升4.4%,并展现出卓越的效率和跨数据集泛化性能。代码见https://github.com/Chen-GX/SEER。
英文摘要:
Elucidating the reasoning process with structured explanations from question to answer is crucial, as it significantly enhances the interpretability, traceability, and trustworthiness of question-answering (QA) systems. However, structured explanations demand models to perform intricately structured reasoning, which poses great challenges. Most existing methods focus on single-step reasoning through supervised learning, ignoring logical dependencies between steps. Moreover, existing reinforcement learning (RL) based methods overlook the structured relationships, underutilizing the potential of RL in structured reasoning. In this paper, we propose SEER, a novel method that maximizes a structure-based return to facilitate structured reasoning and explanation. Our proposed structure-based return precisely describes the hierarchical and branching structure inherent in structured reasoning, effectively capturing the intricate relationships between different reasoning steps. In addition, we introduce a fine-grained reward function to meticulously delineate diverse reasoning steps. Extensive experiments show that SEER significantly outperforms state-of-the-art methods, achieving an absolute improvement of 6.9% over RL-based methods on EntailmentBank, a 4.4% average improvement on STREET benchmark, and exhibiting outstanding efficiency and cross-dataset generalization performance. Our code is available at https://github.com/Chen-GX/SEER.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。