arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.01784cs.AIcs.CR
收藏

噪声注入揭示了“沙袋”语言模型的隐藏能力

Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models

Geodesic Research;Technical University of Munich(慕尼黑技术大学);Tufts University(塔夫茨大学);SecureBio;Apart Research;Anthropic;Apollo Research

Cameron Tice, Philipp Alexander Kreer, Nathan Helm-Burger, Prithviraj Singh Shahani, Fedor Ryzhenkov, Fabien Roger, Clement Neo, Jacob Haimes, Felix Hofstätter, Teun van der Weij

AI总结:

通过噪声注入揭示沙袋语言模型的隐藏能力,提供了一种检测和评估前沿AI系统有效性的实用工具。

摘要:

能力评估在评估和监管前沿AI系统中起着至关重要的作用。这些评估的有效性面临重大挑战:战略性欠佳,或称为“沙袋”现象,其中模型在评估期间故意表现不佳。沙袋行为可能通过显式的开发者干预或通过意外的模型行为表现出来,构成了准确能力评估的根本障碍。我们介绍了一种基于向模型权重中注入不同幅度噪声的新型沙袋检测方法。非沙袋模型在增加噪声时表现出可预测的性能下降,但我们证明沙袋模型表现出异常的性能提升,这可能是由于干扰了欠佳机制,而核心能力仍部分保持完好。通过在各种模型架构、大小和沙袋技术上的实验,我们确立了这种独特的响应模式作为可靠、模型无关的信号,用于检测沙袋行为。重要的是,我们发现噪声注入能够在一个模型表现不佳但未被指示的情况下,使Mistral Large 120B完全展现其性能。我们的发现提供了一种实用的AI评估和监管工具,解决了确保前沿AI系统准确能力评估的挑战。

英文摘要:

Capability evaluations play a crucial role in assessing and regulating frontier AI systems. The effectiveness of these evaluations faces a significant challenge: strategic underperformance, or ``sandbagging'', where models deliberately underperform during evaluation. Sandbagging can manifest either through explicit developer intervention or through unintended model behavior, presenting a fundamental obstacle to accurate capability assessment. We introduce a novel sandbagging detection method based on injecting noise of varying magnitudes into model weights. While non-sandbagging models show predictable performance degradation with increasing noise, we demonstrate that sandbagging models exhibit anomalous performance improvements, likely due to disruption of underperformance mechanisms while core capabilities remain partially intact. Through experiments across various model architectures, sizes, and sandbagging techniques, we establish this distinctive response pattern as a reliable, model-agnostic signal for detecting sandbagging behavior. Importantly, we find noise-injection is capable of eliciting the full performance of Mistral Large 120B in a setting where the model underperforms without being instructed to do so. Our findings provide a practical tool for AI evaluation and oversight, addressing a challenge in ensuring accurate capability assessment of frontier AI systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑