大型语言模型在指令遵循中的不确定性估计是否准确？
Do LLMs estimate uncertainty well in instruction-following?
AI总结：
本研究首次系统评估LLMs在指令遵循中的不确定性估计，发现现有方法在细微错误下表现不佳，提出受控评估框架以促进更可靠的AI代理。
摘要：
大型语言模型（LLMs）若能在各领域精确遵循用户指令，则有望成为有价值的个人AI代理。然而，近期研究显示LLMs在指令遵循能力上存在显著局限，引发对其在高风险应用中可靠性的担忧。准确估计LLMs在遵循指令时的不确定性，对于缓解部署风险至关重要。我们提出了，据我们所知，首个在指令遵循背景下对LLMs不确定性估计能力的系统性评估。我们的研究识别了现有指令遵循基准中的关键挑战，其中多个因素与指令遵循产生的不确定性纠缠在一起，使得跨方法和模型的隔离与比较复杂化。为解决这些问题，我们引入了一个受控评估设置，包含两个基准版本的数据，从而能够在各种条件下全面比较不确定性估计方法。我们的发现表明，现有不确定性方法表现不佳，尤其是在模型在指令遵循中犯细微错误时。虽然内部模型状态提供了一些改进，但在更复杂场景中仍显不足。我们受控评估设置的见解提供了对LLMs在指令遵循任务中不确定性估计局限性和潜力的关键理解，为更值得信赖的AI代理铺平了道路。
英文摘要：
Large language models (LLMs) could be valuable personal AI agents across various domains, provided they can precisely follow user instructions. However, recent studies have shown significant limitations in LLMs' instruction-following capabilities, raising concerns about their reliability in high-stakes applications. Accurately estimating LLMs' uncertainty in adhering to instructions is critical to mitigating deployment risks. We present, to our knowledge, the first systematic evaluation of the uncertainty estimation abilities of LLMs in the context of instruction-following. Our study identifies key challenges with existing instruction-following benchmarks, where multiple factors are entangled with uncertainty stems from instruction-following, complicating the isolation and comparison across methods and models. To address these issues, we introduce a controlled evaluation setup with two benchmark versions of data, enabling a comprehensive comparison of uncertainty estimation methods under various conditions. Our findings show that existing uncertainty methods struggle, particularly when models make subtle errors in instruction following. While internal model states provide some improvement, they remain inadequate in more complex scenarios. The insights from our controlled evaluation setups provide a crucial understanding of LLMs' limitations and potential for uncertainty estimation in instruction-following tasks, paving the way for more trustworthy AI agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
