arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.04485cs.CLcs.AIcs.LG
收藏

基于大语言模型的主动任务消歧

Active Task Disambiguation with LLMs

University of Cambridge(剑桥大学)

Katarzyna Kobalczyk, Nicolas Astorga, Tennison Liu, Mihaela van der Schaar

AI总结:

本文从贝叶斯实验设计视角形式化任务模糊性问题,提出主动任务消歧方法,使LLM智能体通过生成最大化信息增益的澄清性问题来缩小可行解空间,实验证明该方法优于仅依赖问题空间内推理的方法。

摘要:

尽管大语言模型(LLMs)在各类基准测试中表现优异,但其处理现实中常见的模糊问题描述的能力仍未得到充分探索。为弥补这一空白,我们提出了任务模糊性的形式化定义,并从贝叶斯实验设计(Bayesian Experimental Design)的视角构建任务消歧问题。通过提出澄清性问题,LLM智能体能够获取额外的任务规格信息,逐步缩小可行解空间,降低生成不理想输出的风险。然而,生成有效的澄清性问题要求LLM智能体进行一种元认知推理,而这一能力目前LLMs可能尚不具备。我们提出的主动任务消歧方法使LLM智能体能够生成最大化信息增益的针对性问题。实际上,该方法将推理负担从对可行解空间的隐式推理转移到显式推理。实验结果表明,与仅依赖问题空间内推理的方法相比,这种问题选择方式能够实现更有效的任务消歧。

英文摘要:

Despite the impressive performance of large language models (LLMs) across various benchmarks, their ability to address ambiguously specified problems--frequent in real-world interactions--remains underexplored. To address this gap, we introduce a formal definition of task ambiguity and frame the problem of task disambiguation through the lens of Bayesian Experimental Design. By posing clarifying questions, LLM agents can acquire additional task specifications, progressively narrowing the space of viable solutions and reducing the risk of generating unsatisfactory outputs. Yet, generating effective clarifying questions requires LLM agents to engage in a form of meta-cognitive reasoning, an ability LLMs may presently lack. Our proposed approach of active task disambiguation enables LLM agents to generate targeted questions maximizing the information gain. Effectively, this approach shifts the load from implicit to explicit reasoning about the space of viable solutions. Empirical results demonstrate that this form of question selection leads to more effective task disambiguation in comparison to approaches relying on reasoning solely within the space of questions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑