基于DeCRIM的大语言模型自校正：分解、批判与优化，提升多约束指令遵循能力
LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints
AI总结：
针对LLM处理多约束指令的不足，研究推出RealInstruct基准，提出DeCRIM自校正流程，可提升开源模型性能，结合强反馈时其表现能超越GPT-4。
摘要：
指令遵循是大语言模型（LLM）的核心能力，但近期研究表明，LLM常难以处理含多约束的指令，例如要求创作社交媒体帖子时需“语气幽默”且“不使用话题标签”。尽管如此，多数评估仅聚焦于合成数据。为解决此问题，我们推出RealInstruct——首个利用真实用户向AI助手提问的查询，用于评估LLM遵循真实世界多约束指令能力的基准。我们还探究了基于模型的评估作为该任务人工标注的高性价比替代方案。研究发现，即便是专有模型GPT-4，在超过21%的指令上也至少未满足一项约束，凸显了当前最先进模型的局限性。为缩小开源模型与专有模型间的性能差距，我们提出Decompose, Critique and Refine（DeCRIM）自校正流程，以增强LLM的约束遵循能力。DeCRIM的工作机制为：将原始指令分解为约束列表，再通过一个批判（Critic）模型判断LLM的响应何时、何处需要优化。结果显示，即便采用弱反馈，DeCRIM仍能将Mistral在RealInstruct基准上的性能提升7.3%，在IFEval基准上提升8.0%；此外，我们还证明，采用强反馈时，结合DeCRIM的开源LLM在两个基准上均可超越GPT-4。
英文摘要：
Instruction following is a key capability for LLMs. However, recent studies have shown that LLMs often struggle with instructions containing multiple constraints (e.g. a request to create a social media post "in a funny tone" with "no hashtag"). Despite this, most evaluations focus solely on synthetic data. To address this, we introduce RealInstruct, the first benchmark designed to evaluate LLMs' ability to follow real-world multi-constrained instructions by leveraging queries real users asked AI assistants. We also investigate model-based evaluation as a cost-effective alternative to human annotation for this task. Our findings reveal that even the proprietary GPT-4 model fails to meet at least one constraint on over 21% of instructions, highlighting the limitations of state-of-the-art models. To address the performance gap between open-source and proprietary models, we propose the Decompose, Critique and Refine (DeCRIM) self-correction pipeline, which enhances LLMs' ability to follow constraints. DeCRIM works by decomposing the original instruction into a list of constraints and using a Critic model to decide when and where the LLM's response needs refinement. Our results show that DeCRIM improves Mistral's performance by 7.3% on RealInstruct and 8.0% on IFEval even with weak feedback. Moreover, we demonstrate that with strong feedback, open-source LLMs with DeCRIM can outperform GPT-4 on both benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
