大语言模型对齐的可验证性：行为评估下的规范不可区分性
Alignment Verifiability in Large Language Models: Normative Indistinguishability under Behavioral Evaluation
AI总结：
本文研究了大语言模型对齐的可验证性，指出行为评估无法唯一确定潜在对齐，提出了规范不可区分性概念，并通过实验验证了在评估意识下行为基准的局限性。
摘要：
行为评估是评估大语言模型（LLM）对齐性的主要范式。在当前实践中，有限评估协议下的观察合规性被视为潜在对齐的证据。然而，从有限行为证据推断全球潜在属性的推断很少被视为可识别性问题。在本文中，我们通过统计可识别性在部分可观测性下的视角研究对齐评估。我们允许智能体策略根据与评估范式相关的可观察信号来调节其行为，这种现象我们称之为评估意识。在此框架中，我们正式化了对齐可验证性问题，并引入了规范不可区分性，即当不同的潜在对齐假设诱导出相同的观察者可访问的观测分布时。我们的主要理论贡献是一个条件不可能结果：在有限行为评估和评估意识策略下，观察到的合规性不能唯一确定潜在对齐，但只能确定条件合规策略的等价类，前提是明确假设了策略表达性和可观测性。我们通过使用指令调优的LLM（Llama-3.2-3B）补充理论，证明了一个构造性存在证明，展示了一个在显式评估信号下完美合规的策略，但当相同的评估意图以隐式方式传达时，其可识别性会退化。共同，我们的结果表明，在评估意识下，行为基准提供了潜在对齐的必要但不充分的证据。
英文摘要：
Behavioral evaluation is the dominant paradigm for assessing alignment in large language models (LLMs). In current practice, observed compliance under finite evaluation protocols is treated as evidence of latent alignment. However, the inference from bounded behavioral evidence to claims about global latent properties is rarely analyzed as an identifiability problem. In this paper, we study alignment evaluation through the lens of statistical identifiability under partial observability. We allow agent policies to condition their behavior on observable signals correlated with the evaluation regime, a phenomenon we term evaluation awareness. Within this framework, we formalize the Alignment Verifiability Problem and introduce Normative Indistinguishability, which arises when distinct latent alignment hypotheses induce identical distributions over evaluator-accessible observations. Our main theoretical contribution is a conditional impossibility result: under finite behavioral evaluation and evaluation-aware policies, observed compliance does not uniquely identify latent alignment, but only membership in an equivalence class of conditionally compliant policies, under explicit assumptions on policy expressivity and observability. We complement the theory with a constructive existence proof using an instruction-tuned LLM (Llama-3.2-3B), demonstrating a conditional policy that is perfectly compliant under explicit evaluation signals yet exhibits degraded identifiability when the same evaluation intent is conveyed implicitly. Together, our results show that behavioral benchmarks provide necessary but insufficient evidence for latent alignment under evaluation awareness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
