服从、发散、崩溃：对错误指令的盲目服从驱使代码语言模型陷入无法恢复的代码语义崩溃
Obey, Diverge, Collapse: Blind Obedience to Incorrect Instructions Drives Code LLMs to Irrecoverable Code Semantic Collapse
AI总结：
研究代码语言模型在指令错误时的表现，通过四个实验评估其在单步和迭代修复设置中对错误指令的应对，发现模型会盲目服从错误指令致代码语义崩溃，引入幽灵错误且无法恢复。
摘要：
代码语言模型如今在生产工作流程中是值得信赖的协作工具，评估它们的每个基准都假定其执行的指令是正确的。我们研究当这个假设不成立时会发生什么。我们使用具有确定性测试用例的算法Python问题的RunBugRun数据集，通过四个实验评估代码语言模型，以评估模型在单步和迭代修复设置中是否抵制或服从错误指令。我们的发现揭示了一种惊人的行为模式：模型正确地将错误指令识别为错误，但仍然遵循它。这种顺从在不知不觉中引入了超出原始错误的错误，并且损坏的代码状态无法通过随后的自我引导迭代修复来恢复，这种修复在各轮中无法收敛。我们将此称为盲目服从，描述它引入的幽灵（未知）错误，量化语义损坏被证明无法恢复的情况的比例，并表明扩展推理无法扭转它。这些发现揭示了通过率评估中不可见的行为属性，对生产环境中部署的代码语言模型有直接影响。
英文摘要：
Code language models are now trusted collaborators in production workflows for debugging, refactoring, and iterative repair, and every benchmark that evaluates them assumes the instructions they act on are correct. We study what happens when that assumption breaks. We evaluate code language models across four experiments designed to assess whether models resist or obey incorrect instructions in single-pass and iterative repair settings, using the RunBugRun dataset of algorithmic Python problems with deterministic test cases. Our findings reveal a striking behavioral pattern: models correctly identify an incorrect instruction as wrong, then follow it anyway. This compliance unknowingly introduces errors beyond the original bug, and the corrupted code state cannot be recovered through subsequent self-guided iterative repair, which fails to converge across passes. We term this Blind Obedience, characterize the Ghost (Unknown) Errors it introduces, quantify the proportion of cases where semantic corruption proves irrecoverable, and show that extended reasoning cannot reverse it. These findings surface behavioral properties invisible to pass-rate evaluation, with direct consequences for code language models deployed in production settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
