arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.00555cs.LGcs.AIstat.APstat.ML
收藏

Prune 'n Predict:利用共形预测优化LLM决策

Prune 'n Predict: Optimizing LLM Decision-making with Conformal Prediction

JPMorganChase AI Research(摩根大通AI研究院);University of Wisconsin-Madison(威斯康星大学麦迪逊分校)

Harit Vishwakarma, Alan Mishler, Thomas Cook, Niccolò Dalmasso, Natraj Raman, Sumitra Ganesh

AI总结:

提出CROQ方法,利用共形预测缩小选项集并修订问题,结合CP-OPT优化框架最小化预测集大小,显著提升LLM在多项选择和工具使用任务中的决策准确性与可靠性。

摘要:

大型语言模型(LLMs)正在赋能包括工具或API使用以及回答多项选择题(MCQs)在内的多项应用中的决策。然而,不正确的输出在医疗和金融等高风险领域构成了重大风险。为了量化LLM的不确定性从而缓解这些风险,最近的研究采用了共形预测(CP),这是一个与模型和分布无关的框架,它使用LLM输出生成一个以高概率包含真实答案的预测集。利用CP,我们提出了共形问题修订(CROQ),它通过将可用选项缩小到预测集中的选项,并向LLM提出修订后的问题来修订问题。我们期望LLM在选项更少的修订问题上更准确。此外,我们期望CROQ在CP生成的预测集较小时有效。常用的logit分数通常会导致较大的集合,削弱了CROQ的有效性。为了克服这一点,我们提出了CP-OPT,这是一个优化框架,用于学习在保持覆盖的同时最小化集合大小的分数。我们在MMLU、ToolAlpaca和TruthfulQA数据集上使用多个LLM进行了广泛实验,结果表明CROQ提高了标准推理的准确性,当与CP-OPT配对时,增益更为显著。

英文摘要:

Large language models (LLMs) are empowering decision-making in several applications, including tool or API usage and answering multiple-choice questions (MCQs). However, incorrect outputs pose significant risks in high-stakes domains like healthcare and finance. To quantify LLM uncertainty and thereby mitigate these risks, recent works employ conformal prediction (CP), a model- and distribution-agnostic framework that uses LLM outputs to generate a \emph{prediction set} containing the true answer with high probability. Leveraging CP, we propose \emph{conformal revision of questions} (CROQ), which revises the question by narrowing down the available choices to those in the prediction set and asking the LLM the revised question. We expect LLMs to be more accurate on revised questions with fewer choices. Furthermore, we expect CROQ to be effective when the prediction sets from CP are small. Commonly used logit scores often lead to large sets, diminishing CROQ's effectiveness. To overcome this, we propose CP-OPT, an optimization framework to learn scores that minimize set sizes while maintaining coverage. Our extensive experiments on MMLU, ToolAlpaca, and TruthfulQA datasets with multiple LLMs show that CROQ improves accuracy over the standard inference, with more pronounced gains when paired with CP-OPT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑