通过忠实性与详尽性衡量推理链的可监控性
Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity
AI总结：
本文通过结合忠实性与详尽性提出新的可监控性评分，评估模型推理链的透明度和可监控性，发现不同模型家族的可监控性差异显著。
摘要：
推理链（CoT）输出使我们能够阅读模型的逐步推理过程。由于任何长串的连续推理过程都必须经过这个文本轨迹，CoT的质量直接反映了模型在想什么。这种可见性有助于我们发现不安全或不一致的行为（可监控性），但只有当CoT对内部推理过程保持透明（忠实性）时才可能实现。完全衡量忠实性是困难的，因此研究者通常专注于检查在输入添加提示后模型改变答案的CoT情况。这种代理方法发现了一些不忠实的实例，但当模型保持答案时会丢失信息，并且不研究与提示无关的推理方面。我们通过引入详尽性（即CoT是否列出解决任务所需的所有因素）将这些结果扩展到更全面的可监控性概念中。我们将忠实性与详尽性结合成一个单一的可监控性评分，以显示CoT作为模型外部'工作记忆'的性能如何，这一属性是许多基于CoT监控的安全方案所依赖的。我们对指令调优和推理模型在BBH、GPQA和MMLU上进行了评估。我们的结果表明，模型可能在表面上忠实，但若遗漏关键因素则难以监控，且不同模型家族的可监控性差异显著。我们通过Inspect库发布了我们的评估代码，以支持可重复的未来工作。
英文摘要：
Chain-of-thought (CoT) outputs let us read a model's step-by-step reasoning. Since any long, serial reasoning process must pass through this textual trace, the quality of the CoT is a direct window into what the model is thinking. This visibility could help us spot unsafe or misaligned behavior (monitorability), but only if the CoT is transparent about its internal reasoning (faithfulness). Fully measuring faithfulness is difficult, so researchers often focus on examining the CoT in cases where the model changes its answer after adding a cue to the input. This proxy finds some instances of unfaithfulness but loses information when the model maintains its answer, and does not investigate aspects of reasoning not tied to the cue. We extend these results to a more holistic sense of monitorability by introducing verbosity: whether the CoT lists every factor needed to solve the task. We combine faithfulness and verbosity into a single monitorability score that shows how well the CoT serves as the model's external `working memory', a property that many safety schemes based on CoT monitoring depend on. We evaluate instruction-tuned and reasoning models on BBH, GPQA, and MMLU. Our results show that models can appear faithful yet remain hard to monitor when they leave out key factors, and that monitorability differs sharply across model families. We release our evaluation code using the Inspect library to support reproducible future work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
