用于大语言模型中顺序信息收集的摊销贝叶斯实验设计
Amortising Bayesian Experimental Design for Sequential Information Gathering in LLMs
AI总结：
研究大语言模型在顺序决策中信息收集问题，提出摊销顺序信息收集方法，将贝叶斯实验设计融入模型策略，经实验验证该方法能有效提升成功率并降低推理成本。
摘要：
大语言模型虽有强大推理和世界知识能力，但在顺序决策设置的多轮交互中难以有效收集信息。我们引入摊销顺序信息收集（ASIG），一种通过带有预期信息增益奖励的组相对策略优化的多轮扩展，将贝叶斯实验设计摊销到大语言模型策略中的微调方法。在20个问题任务上评估，ASIG使7B基础模型成功率翻倍，相对于BED - LLM（一个有竞争力的推理时基线），推理成本降低超过25倍。应用于训练期间未见过的医学诊断基准MediQ，ASIG在7B规模上提高了信息寻求性能，表明学习到的策略可以分布外转移。我们的发现表明，将BED摊销到大语言模型策略中为顺序信息收集提供了一种有效且计算高效的方法。
英文摘要：
Large language models (LLMs) exhibit strong reasoning and world-knowledge capabilities, yet often struggle to gather information effectively across the multi-turn interactions required in sequential decision-making settings. We introduce Amortised Sequential Information Gathering (ASIG), a fine-tuning approach that amortises Bayesian Experimental Design (BED) into LLM policies via a multi-turn extension of Group Relative Policy Optimisation with an Expected Information Gain reward. Evaluated on the 20 Questions task, ASIG more than doubles the success rate of the 7B base model and reduces inference cost by over $25\times$ relative to BED-LLM, a competitive inference-time baseline. Applied to MediQ, a medical diagnosis benchmark unseen during training, ASIG improves information-seeking performance at the 7B scale, suggesting that the learned strategies can transfer out of distribution. Our findings show that amortising BED into LLM policies provides an effective and computationally efficient approach to sequential information gathering.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
