arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.03595cs.CL
收藏

在大语言模型生成中解耦任务解决与输出格式

Decoupling Task-Solving and Output Formatting in LLM Generation

University of California, Los Angeles(加州大学洛杉矶分校)

Haikang Deng, Po-Nien Kung, Nanyun Peng

AI总结:

研究针对大语言模型任务指令与格式要求交织致性能下降的问题,提出解耦框架Deco-G,将格式遵循委托给单独模块,引入指令感知蒸馏等三项创新,实验证明其优于基线方法,能保证格式符合。

摘要:

大语言模型(LLMs)在解决复杂问题(如数学推理和自动评估)方面日益熟练。然而,当提示将任务指令与严格的格式要求交织在一起时,性能往往会下降。这种交织给模型带来了相互竞争的目标,阻碍了其推理能力。为解决此问题,我们引入了Deco-G,这是一个明确将格式遵循与问题解决解耦的解码框架。Deco-G将格式遵循委托给一个单独的格式估计模块(FEM),该模块进行概率前瞻以估计未来格式符合率并重新权衡令牌概率,使LLM能够仅专注于任务解决。为使该方法既实用又高效,我们引入了三项关键创新:指令感知蒸馏、灵活的前缀树构建算法和隐马尔可夫模型状态剪枝。在数学推理、事件论点提取以及LLM作为评判等方面的实验表明,Deco-G始终优于提示或结构化生成基线,且保证格式符合。我们在这个https URL上发布了代码。

英文摘要:

Large language models (LLMs) are increasingly adept at solving complex problems, such as mathematical reasoning and automatic evaluation. However, performance often degrades when prompts intertwine task instructions with rigid formatting requirements. This entanglement creates competing goals for the model, hindering its reasoning capabilities. To address this, we introduce Deco-G, a decoding framework that explicitly decouples format adherence from problem solving. Deco-G delegates format adherence to a separate Format Estimation Module (FEM), which performs probabilistic lookahead to estimate future format compliance rate and reweighs token probabilities, allowing the LLM to focus solely on task resolution. To make this approach both practical and efficient, we introduce three key innovations: instruction-aware distillation, a flexible trie-building algorithm, and HMM state pruning. Experiments across mathematical reasoning, event argument extraction, and LLM-as-a-judge demonstrate that Deco-G constantly gains over prompting or structured generation baselines, with guaranteed format compliance. We release our code at https://github.com/haikangdeng/deco-g.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑