大型语言模型在遵循指令时是否“内部知晓”？
Do LLMs "know" internally when they follow instructions?
AI总结：
本研究探究LLMs是否在内部表征中编码指令遵循成功信息，识别出可预测遵循结果的“指令遵循维度”，并通过沿此维度修改表征提升遵循率，为可靠智能体构建提供洞见。
摘要：
指令遵循对于使用大型语言模型（LLMs）构建AI智能体至关重要，因为这些模型必须严格遵守用户提供的约束和指南。然而，LLMs即使面对简单明了的指令也常常无法遵循。为了改进指令遵循行为并防止不良输出，需要更深入地理解LLMs的内部状态如何与这些结果相关联。在本工作中，我们研究了LLMs是否在其表征中编码了与指令遵循成功相关的信息——我们将这一属性称为“内部知晓”。我们的分析在输入嵌入空间中识别出一个方向，称为“指令遵循维度”，该方向可预测响应是否会遵循给定指令。我们发现该维度在未见过的任务上具有良好的泛化能力，但在未见过的指令类型上泛化不佳。我们证明，与随机改变相比，沿此维度修改表征可提高指令遵循成功率，且不损害响应质量。进一步调查显示，该维度与提示的措辞关系更为密切，而非任务或指令的内在难度。这项工作为LLMs指令遵循的内部工作机制提供了洞见，为构建可靠的LLM智能体铺平了道路。
英文摘要：
Instruction-following is crucial for building AI agents with large language models (LLMs), as these models must adhere strictly to user-provided constraints and guidelines. However, LLMs often fail to follow even simple and clear instructions. To improve instruction-following behavior and prevent undesirable outputs, a deeper understanding of how LLMs' internal states relate to these outcomes is required. In this work, we investigate whether LLMs encode information in their representations that correlate with instruction-following success - a property we term knowing internally. Our analysis identifies a direction in the input embedding space, termed the instruction-following dimension, that predicts whether a response will comply with a given instruction. We find that this dimension generalizes well across unseen tasks but not across unseen instruction types. We demonstrate that modifying representations along this dimension improves instruction-following success rates compared to random changes, without compromising response quality. Further investigation reveals that this dimension is more closely related to the phrasing of prompts rather than the inherent difficulty of the task or instructions. This work provides insight into the internal workings of LLMs' instruction-following, paving the way for reliable LLM agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
