大型语言模型中的预填充感知
Prefill Awareness in Large Language Models
AI总结：
研究大型语言模型能否识别并响应其助手消息被预填充或篡改，发现前沿模型具有显著预填充感知能力，可能影响安全评估方法。
摘要：
语言模型的安全相关研究，包括对齐和越狱评估以及AI控制协议，通常依赖于预填充模型输出。如果AI模型能够识别并利用其先前的助手消息被插入或编辑这一事实，这些方法的有效性和有效性可能会受到损害。我们调查了前沿语言模型是否能区分被篡改和未被篡改的助手侧上下文，我们将这种能力称为预填充感知。为此，我们构建了一个跨三种预填充机制的二元偏好基准，筛选出模型表现出一致立场的案例。我们发现前沿模型表现出显著的预填充感知：Claude Opus 4.5在9-35%的案例中检测到与其偏好相反的预填充，且在提示时假阳性率为0%；此外，模型通常会恢复到基线行为，而不会明确报告预填充是外来的。受控消融实验后来也表明，检测和抵抗依赖于不同的线索，其中风格不匹配主要影响模型是否将预填充标记为外来，而偏好不匹配主要影响模型是否恢复到其基线答案。我们还检查了更真实的智能体设置，如错位延续评估和SWE-bench轨迹，在这些设置中，前沿模型有时会否认预填充的助手轮次，其方式强烈依赖于数据集、任务成功和隐藏的格式伪影。我们的结果表明，预填充感知已经是一些基于预填充的方法的重要混淆因素。我们建议模型开发者在前沿系统中跟踪这种能力。
英文摘要：
Safety-relevant studies of language models, including alignment and jailbreaking evaluations and AI control protocols, often rely on prefilling model outputs. If AI models can recognize and act on the fact their prior assistant messages have been inserted or edited, the effectiveness and validity of these methods could be compromised. We investigate whether frontier language models can distinguish between tampered and untampered assistant-side context, a capability we call prefill awareness. To do so, we construct a binary preference benchmark across three prefill mechanisms, filtering for cases where models show consistent stances. We find that frontier models show substantial prefill awareness: Claude Opus 4.5 detects prefills opposing its preferences in 9-35% of cases with a 0% false positive rate when prompted; additionally, models often revert towards baseline behavior without explicitly reporting that the prefill was foreign. Controlled ablations later also show that detection and resistance rely on different cues, where stylistic mismatch mainly affects whether models flag a prefill as foreign, while preference mismatch mainly affects whether they revert toward their baseline answer. We also examine more realistic agentic settings such as misalignment-continuation evaluations and SWE-bench trajectories, where frontier models sometimes disavow prefilled assistant turns in ways that depend strongly on dataset, task success, and hidden formatting artifacts. Our results indicate that prefill awareness is already a substantial confound for some prefill-based methods. We recommend that model developers track this capability in frontier systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
