架构决定Transformer的可观测性
Architecture Determines Observability of Transformers
AI总结:
研究探讨了Transformer架构如何影响模型的可观测性,通过控制输出置信度消除了60.3%的激活信号,揭示了架构和训练对可观测性的影响。
摘要:
自回归Transformer在输出置信度监控无法捕捉的自信错误上表现自信。激活监控只能在训练留下决策质量信号时才能捕捉到这些错误。这种信号是训练后的模型固有的架构属性,位于任何监控器之前。在Pythia的受控训练中,两种匹配宽度的配置在早期形成信号。一个在收敛时保留信号,另一个在困惑度继续提高时消除信号。能力和可观测性并不本质上矛盾。在独立训练的家族中,这种模式持续存在,即使崩溃点发生变化。信号存活时,监控可以捕捉置信度无法捕捉的内容。在下游QA任务中,一个未经过任务特定微调的WikiText训练探测器能够捕捉到约1/8的自信错误,误报率约为20%。这些结果确立了信号工程作为训练时间设计轴,与损失和能力并列。架构设定了可观测性的条件,而训练决定了什么仍然可读。
英文摘要:
Autoregressive transformers make confident errors that output-confidence monitoring cannot catch. Activation monitors catch them only when training leaves a decision-quality signal beyond what the output already exposes. This signal is an architectural property of the trained model, fixed upstream of any monitor. Controlling for output confidence removes 60.3% of the raw activation-probe signal on average across 14 models. Raw probe signal is mostly output confidence, and output-side readouts cannot recover the residual. What remains depends on architecture and training. In Pythia's controlled training, both matched-width configurations form the signal early. One preserves it through convergence while another erases it as perplexity continues to improve. Capability and observability are not inherently in tension. Across independently trained families this pattern persists, even as the collapse point shifts. Where the signal survives, monitoring catches what confidence cannot. On downstream QA, a WikiText-trained probe with no task-specific tuning catches about one in eight confident errors that output-confidence monitoring misses, at a 20% flag rate. These results establish signal engineering as a training-time design axis alongside loss and capability. Architecture sets the conditions for observability, and training determines what remains readable.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。