arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.13514cs.CLcs.AI
收藏

Self-DC:何时推理,何时行动?面向组合未知问题的自我分而治之

Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questions

The Chinese University of Hong Kong(香港中文大学);Nankai University(南开大学);MoE Key Lab of High Confidence Software Technologies, CUHK(香港中文大学教育部高可信软件技术重点实验室);Westlake University(西湖大学);Jarvis Research Center, Tencent YouTu Lab(腾讯优图实验室贾维斯研究中心);Southern University of Science and Technology(南方科技大学)

Hongru Wang, Boyang Xue, Baohang Zhou, Tianhua Zhang, Cunxiang Wang, Huimin Wang, Guanhua Chen, Kam-fai Wong

AI总结:

针对由已知和未知子问题组成的组合未知问题,提出Self-DC框架,使LLM自适应选择内部推理或外部检索,并构建CuQA数据集,以更少外部调用实现更优效果与效率权衡。

摘要:

以往研究通常集中于利用大型语言模型(LLMs)的内部知识来回答已知问题(即内部推理,如“生成后读取”)。相反,对于超出其已知范围的问题,这些模型依赖外部知识检索来提供准确回答(即外部行动,如“检索后读取”)。然而,很少有先前工作考虑组合问题,这类问题由若干已知和未知的子问题组成,需要动态结合前述两种方法(即内部推理与外部行动),以在效果与效率之间取得更优权衡。为此,我们提出了一个自我分而治之(Self-DC)框架,并配套构建了首个组合未知问答数据集(CuQA)。该框架使LLMs能够按需自适应地在使用内部知识与检索外部知识之间进行选择,从而在效果与效率之间实现更优权衡。在两个数据集上的实验结果表明,与多个强基线相比,Self-DC能够以更少的外部调用达到相当甚至更好的性能。

英文摘要:

Previous research has typically concentrated on leveraging the internal knowledge of Large Language Models (LLMs) to answer known questions (i.e., \textit{internal reasoning such as generate-then-read}). In contrast, for questions that fall outside their known scope, these models rely on external knowledge retrieval to provide accurate responses (i.e., \textit{external acting such as retrieve-then-read}). However, few previous works consider the \textit{compositional questions}, which consist of several known and unknown sub-questions, necessitating the dynamic combination of previous two methods (i.e., \textit{internal reasoning and external acting}) to achieve a better trade-off between effectiveness and efficiency. To this end, we introduce a \textbf{Self} \textbf{D}ivide-and-\textbf{C}onquer (\textit{\texttt{Self-DC}}) framework, accompanying with the first \textbf{C}ompositional \textbf{u}nknown \textbf{Q}uestion-\textbf{A}nswering dataset (CuQA). This framework enables LLMs to adaptively choose between using internal knowledge and retrieving external knowledge as needed, resulting in a better trade-off between effectiveness and efficiency. Experimental results on two datasets demonstrate that \textit{\texttt{Self-DC}} can achieve comparable or even better performance with much fewer external calls compared with several strong baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑