arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.11166cs.CL
收藏

GenDec:一种用于多跳推理的鲁棒生成式问题分解方法

GenDec: A robust generative Question-decomposition method for Multi-hop reasoning

Tokyo Institute of Technology(东京工业大学);Westlake University(西湖大学);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院)

Jian Wu, Linyi Yang, Yuliang Ji, Wenhao Huang, Börje F. Karlsson, Manabu Okumura

AI总结:

本文提出一种生成式问题分解方法GenDec,通过生成独立完整的子问题并结合额外证据,增强LLMs在RAG中的多跳推理能力,实验验证了其在多个数据集和模型上的有效性与鲁棒性。

摘要:

多跳问答(MHQA)涉及逐步推理以回答复杂问题并寻找多个相关支持事实。然而,现有大型语言模型(LLMs)在多跳问答中的推理能力仍处于探索阶段,在回答多跳问题时表现不足。此外,LLMs是否遵循预期的推理链以得出正确最终答案尚不明确。本文从可解释QA角度提出一种生成式问题分解方法(GenDec),通过基于合并额外提取的证据生成独立且完整的子问题,以增强LLMs在RAG中的推理能力。为展示GenDec的影响、泛化性和鲁棒性,我们进行了两项实验:首先将GenDec与小型QA系统结合用于段落检索和QA任务;其次检验包括GPT-4和GPT-3.5在内的多种最先进LLMs结合GenDec的推理能力。我们在HotpotQA、2WikihopMultiHopQA、MuSiQue和PokeMQA数据集上进行实验。

英文摘要:

Multi-hop QA (MHQA) involves step-by-step reasoning to answer complex questions and find multiple relevant supporting facts. However, Existing large language models'(LLMs) reasoning ability in multi-hop question answering remains exploration, which is inadequate in answering multi-hop questions. Moreover, it is unclear whether LLMs follow a desired reasoning chain to reach the right final answer. In this paper, we propose a \textbf{gen}erative question \textbf{dec}omposition method (GenDec) from the perspective of explainable QA by generating independent and complete sub-questions based on incorporating additional extracted evidence for enhancing LLMs' reasoning ability in RAG. To demonstrate the impact, generalization, and robustness of Gendec, we conduct two experiments, the first is combining GenDec with small QA systems on paragraph retrieval and QA tasks. We secondly examine the reasoning capabilities of various state-of-the-art LLMs including GPT-4 and GPT-3.5 combined with GenDec. We experiment on the HotpotQA, 2WikihopMultiHopQA, MuSiQue, and PokeMQA datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑