在大语言模型中操控工具收敛倾向的可行性
Steerability of Instrumental-Convergence Tendencies in LLMs
AI总结：
该研究通过反工具提示显著降低大语言模型的收敛率，揭示了可控性与安全之间的矛盾，为AI系统的设计提供了新的安全策略。
摘要：
我们研究了人工智能系统两个属性：能力（系统能做什么）和可控性（如何可靠地将行为转向预期结果）。核心问题是能力增长是否会降低可控性并导致控制崩溃的风险。我们还区分了授权可控性（构建者可靠地达到预期行为）和非授权可控性（攻击者引发不允许的行为）。这种区分突显了人工智能模型的基本安全-安全 dilemma：安全需要高可控性以强制控制（例如停止/拒绝），而安全需要低可控性以防止恶意行为引发有害行为。这种紧张关系给开放式模型带来了重大挑战，因为目前通过微调或对抗攻击等常见技术，开放式模型表现出高可控性。使用Qwen3和InstrumentalEval，我们发现一个短的反工具提示后缀能显著降低测量的收敛率（例如避免 shutdown、自我复制）。对于Qwen3-30B Instruct，在反工具提示下，收敛率从81.69%下降到2.82%。在反工具提示下，较大的对齐模型比较小的模型收敛率更低（Instruct：2.82% vs. 4.23%；Thinking：4.23% vs. 9.86%）。代码可在github.com/j-hoscilowicz/instrumental_steering上获得。
英文摘要：
We examine two properties of AI systems: capability (what a system can do) and steerability (how reliably one can shift behavior toward intended outcomes). A central question is whether capability growth reduces steerability and risks control collapse. We also distinguish between authorized steerability (builders reliably reaching intended behaviors) and unauthorized steerability (attackers eliciting disallowed behaviors). This distinction highlights a fundamental safety--security dilemma of AI models: safety requires high steerability to enforce control (e.g., stop/refuse), while security requires low steerability for malicious actors to elicit harmful behaviors. This tension presents a significant challenge for open-weight models, which currently exhibit high steerability via common techniques like fine-tuning or adversarial attacks. Using Qwen3 and InstrumentalEval, we find that a short anti-instrumental prompt suffix sharply reduces the measured convergence rate (e.g., shutdown avoidance, self-replication). For Qwen3-30B Instruct, the convergence rate drops from 81.69% under a pro-instrumental suffix to 2.82% under an anti-instrumental suffix. Under anti-instrumental prompting, larger aligned models show lower convergence rates than smaller ones (Instruct: 2.82% vs. 4.23%; Thinking: 4.23% vs. 9.86%). Code is available at github.com/j-hoscilowicz/instrumental_steering.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
