分类器上下文旋转:通过上下文长度监控性能退化
Classifier Context Rot: Monitor Performance Degrades with Context Length
AI总结:
研究发现当前前沿模型在长上下文情况下难以检测危险行为,通过提示技术可部分缓解此问题,强调长上下文退化对监控性能的影响。
摘要:
使用语言模型监控危险行为需要分类远超500K token的转录文本,但现有基准测试很少包含超过100K token的转录。本文显示,当用作分类器时,当前前沿模型在长转录中更难发现危险行为。特别是在需要识别编码代理在良性活动后采取微妙危险行为的数据集Opus 4.6上,GPT 5.4和Gemini 3.1在危险行为发生于800K token良性活动之后时,遗漏这些行为的次数比在行为发生时多2到30倍。此外,本文还表明,通过提示技术如转录中的定期提醒,这些弱点可以部分缓解,而更好的训练后处理可能进一步缓解。不考虑长上下文退化的监控评估可能高估了监控性能。
英文摘要:
Monitoring coding agents for dangerous behavior using language models requires classifying transcripts that often exceed 500K tokens, but prior agent monitoring benchmarks rarely contain transcripts longer than 100K tokens. We show that when used as classifiers, current frontier models fail to notice dangerous actions more often in longer transcripts. In particular, on a dataset that requires identifying when a coding agent takes a subtly dangerous action, Opus 4.6, GPT 5.4, and Gemini 3.1 miss these actions $2\times$ to $30\times$ more often when they occur after 800K tokens of benign activity than when they occur on their own. We also show that these weaknesses can be partially mitigated with prompting techniques such as periodic reminders throughout the transcript and may be mitigated further with better post-training. Monitor evaluations that do not consider long-context degradation are likely overestimating monitor performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。