arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.15501cs.AI
收藏

大型决策模型:将大型语言模型重新用作单次决策器

Large Decision Models: Repurposing Large Language Models as Single-Pass Decision Makers

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);National University of Singapore(新加坡国立大学);Zhejiang University(浙江大学);University of Pennsylvania(宾夕法尼亚大学);University of Rochester(罗切斯特大学)

Ziyi Chen, Yan Zhang, Jianhui Wei, Jianing Yin, Daoan Zhang

AI总结:

该研究提出大型决策模型(LDMs),通过多样化决策监督和自设推理蒸馏优化,在DecisionVista数据集上提升了决策准确率并大幅降低延迟,还设计了置信度路由以平衡性能与效率。

摘要:

大型语言模型通过并行预填充和顺序解码进行决策。尽管这种方法有效,但会产生解码成本,且无法明确呈现决策概率。现有的特定任务分类器需要针对每个任务进行训练,对未见决策模式的泛化能力有限。我们提出大型决策模型(Large Decision Models,LDMs),将语言模型重新调整为在单次预填充前向传播中返回运行时指定标签的明确概率。我们首先通过多样化决策监督(Diverse Decision Supervision,DDS)使模型适配该接口,覆盖不同领域、规则和标签。为进一步缩小与推理模型的性能差距,自设推理蒸馏(Self-Posed Reasoning Distillation,SPRD)添加由冻结教师模型生成的任务,对其答案一致性进行筛选,并基于教师模型重复推理得到的标签分布进行训练。我们在DecisionVista数据集上进行评估,将新模式测试与保留的人工标注源配对。在三个基础模型上,该方案相较于步数匹配基线分别提升了1.8和1.5个百分点的准确率,在Gemma-4-31B上实现了相较于推理展开182倍的延迟加速。新模式测试中推理模型仍表现更强,这促使我们为不确定决策设计置信度路由。在开发集上,路由策略恢复了推理模型92%的中位数准确率优势,同时仅生成其27%-58%的token。代码可在该https URL获取。

英文摘要:

Large language models make decisions through parallel prefill and sequential decoding. Although effective, this approach incurs decoding costs and does not explicitly expose decision probabilities. Existing task-specific classifiers require per-task training and offer limited generalization to unseen decision schemas. We propose Large Decision Models (LDMs), repurposing language models to return explicit probabilities over labels specified at run time using one prefill forward pass. We first adapt models to this interface through Diverse Decision Supervision (DDS), covering varied domains, rules and labels. To further narrow the performance gap to a reasoning model, Self-Posed Reasoning Distillation (SPRD) adds tasks generated by a frozen teacher, filters them for answer consistency, and trains on label distributions obtained from repeated teacher reasoning. We evaluate on DecisionVista, pairing new-schema tests with held-out human-labelled sources. Across three base models, the recipe improves accuracy over a step-matched baseline by 1.8 and 1.5 percentage points, respectively, achieving a 182-fold speedup in latency over reasoning rollout on Gemma-4-31B. Reasoning remains stronger on the new-schema test, motivating a confidence router for uncertain decisions. On development sets, routing recovers a median 92% of reasoning's accuracy advantage while generating 27-58% of its tokens. Code is available at https://github.com/JianhuiWei7/LDM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑