基于LLM的智能体在忠实多跳问答中的解码方法分析
An Analysis of Decoding Methods for LLM-based Agents for Faithful Multi-Hop Question Answering
AI总结：
本文系统分析了ReAct框架与DeCoRe、DoLa、CAD等解码策略的结合对LLM多跳问答忠实度的影响，发现该方法显著提升了下游任务的准确性。
摘要：
大型语言模型（LLMs）经常产生事实性不准确的输出——这种现象被称为幻觉——这限制了它们在知识密集型NLP任务中的准确性。检索增强生成和诸如Reasoning and Acting（ReAct）的智能体框架可以通过让模型访问外部知识来解决这个问题。然而，LLMs通常无法对检索到的信息保持忠实。缓解这一问题至关重要，特别是当需要LLMs对检索到的信息进行推理时。最近的研究探索了无需训练的解码策略，以提高模型生成的忠实度。我们系统地分析了ReAct框架与解码策略（即DeCoRe、DoLa和CAD）的结合如何影响LLM生成答案的忠实度。我们的结果表明，将用于知识检索的智能体框架与增强忠实度的解码方法相结合，可以提高下游多跳问答任务的准确性。例如，在使用ReAct和DoLa时，我们观察到HotpotQA上的F1分数从19.5提高到32.6。
英文摘要：
Large Language Models (LLMs) frequently produce factually inaccurate outputs - a phenomenon known as hallucination - which limits their accuracy in knowledge-intensive NLP tasks. Retrieval-augmented generation and agentic frameworks such as Reasoning and Acting (ReAct) can address this issue by giving the model access to external knowledge. However, LLMs often fail to remain faithful to retrieved information. Mitigating this is critical, especially if LLMs are required to reason about the retrieved information. Recent research has explored training-free decoding strategies to improve the faithfulness of model generations. We present a systematic analysis of how the combination of the ReAct framework and decoding strategies (i.e., DeCoRe, DoLa, and CAD) can influence the faithfulness of LLM-generated answers. Our results show that combining an agentic framework for knowledge retrieval with decoding methods that enhance faithfulness can increase accuracy on the downstream Multi-Hop Question Answering tasks. For example, we observe an F1 increase from 19.5 to 32.6 on HotpotQA when using ReAct and DoLa.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
