arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.03269cs.CLcs.AI

指令鸿沟:LLMs在遵循指令时迷失

The Instruction Gap: LLMs get lost in Following Instruction

Vishesh Tripathi;Uday Allu;Biddwan Ahmed

Vishesh Tripathi, Uday Allu, Biddwan Ahmed

AI总结:

本研究揭示了LLMs在企业部署中遵循指令的不足,通过评估13种模型发现Claude-Sonnet-4和GPT-5表现最佳,指出指令遵循能力的鸿沟及改进方向。

摘要:

大型语言模型(LLMs)在自然语言理解和生成方面表现出色,但其在企业环境中的部署却暴露了一个关键缺陷:对定制指令的一致性遵循不足。本研究对13种领先的LLMs在指令合规性、响应准确性及现实世界RAG(检索增强生成)场景中的性能指标进行了全面评估。通过系统测试样本和企业级评估协议,我们证明了不同模型在指令遵循方面存在显著差异,Claude-Sonnet-4和GPT-5取得了最高成绩。我们的发现揭示了

英文摘要:

Large Language Models (LLMs) have shown remarkable capabilities in natural language understanding and generation, yet their deployment in enterprise environments reveals a critical limitation: inconsistent adherence to custom instructions. This study presents a comprehensive evaluation of 13 leading LLMs across instruction compliance, response accuracy, and performance metrics in realworld RAG (Retrieval-Augmented Generation) scenarios. Through systematic testing with samples and enterprise-grade evaluation protocols, we demonstrate that instruction following varies dramatically across models, with Claude-Sonnet-4 and GPT-5 achieving the highest results. Our findings reveal the "instruction gap" - a fundamental challenge where models excel at general tasks but struggle with precise instruction adherence required for enterprise deployment. This work provides practical insights for organizations deploying LLM-powered solutions and establishes benchmarks for instruction-following capabilities across major model families.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑