学习何时弃权：微调语言智能体中的课程覆盖与规则特定失败
Learning When to Withdraw: Curriculum Coverage and Rule-Specific Failures in Fine-Tuned Language Agents
AI总结：
该研究针对工具使用语言智能体的弃权决策问题，在NAQD-Env中微调Qwen2.5-3B-Instruct，发现丰富课程的微调检查点弃权表现更优，提出了检测规则特定退化的方法。
摘要：
使用工具的语言智能体必须在证据变化或权限被撤销时修改已准备的动作。选择性弃权（不执行）指暂停受影响的动作，保留已支持的工作，且仅在充分修复后恢复。我们在NAQD-Env中研究监督微调，该环境是具有显式依赖关系和确定性参考策略的合成环境。我们对比Qwen2.5-3B-Instruct的四个LoRA检查点：原始课程与丰富课程，后者在三种系统提示框架下进行，且具有相同的视图和目标。我们在新验证案例、保留的依赖族和保留的结构组合上，对三种指令调优基线和所有四个检查点在三种推理提示下进行评估。在naqd提示下，SFT检查点的决策一致性达到0.77-0.92，弃权召回率为0.46-0.86，而基线的对应值为0.45-0.63，弃权召回率最高仅为0.06。规则级诊断验证了这些增益。原始课程检查点在存在替代支持的34个动作中错误弃权26个，而丰富课程检查点仅弃权0-1个。相反，所有SFT检查点对禁止组合的缩小频率低于7B和8B基线，这是两种课程中都未出现的情况。委托结果显示弃权更多，但对独立授权动作的保留更差，且所有四个检查点都遗漏了有效的事件报告。框架对比在不同指标和划分上存在差异，未确立可复现的优势或等价性。每个分支仅进行一次训练，联合变化的课程属性、先前评估暴露以及基线与SFT的量化差异限制了因果归因。本研究的贡献是一种检测规则特定退化的方法，这种退化是聚合准确率可能掩盖的，而非证明仅微调就能确保智能体安全。
英文摘要：
Tool-using language agents must revise prepared actions when evidence changes or authority is revoked. Selective withdrawal means suspending affected actions, preserving supported work, and resuming only after sufficient repair. We study supervised fine-tuning in NAQD-Env, a synthetic environment with explicit dependencies and a deterministic reference policy. Four LoRA checkpoints of Qwen2.5-3B-Instruct compare an original curriculum with an enriched one, the latter under three system-prompt framings with identical views and targets. Three instruction-tuned baselines and all four checkpoints are evaluated under three inference prompts on fresh validation cases, held-out dependency families, and held-out structural combinations. Under the naqd prompt, SFT checkpoints reach decision agreement of 0.77-0.92 and withdrawal recall of 0.46-0.86, versus 0.45-0.63 and at most 0.06 for the baselines. Rule-level diagnostics qualify these gains. The original-curriculum checkpoint wrongly withdraws 26 of 34 actions with surviving alternative support; enriched checkpoints withdraw 0-1. Conversely, all SFT checkpoints narrow prohibited combinations less often than the 7B and 8B baselines, a case absent from both curricula. Delegation results show more withdrawal but poorer preservation of independently authorized actions, and all four checkpoints omit valid event reports. Framing contrasts vary across metrics and splits and do not establish a reproducible advantage or equivalence. One training run per arm, jointly changed curriculum properties, prior evaluation exposure, and baseline-SFT quantization differences limit causal attribution. The contribution is a method for detecting rule-specific regressions that aggregate accuracy can conceal, not evidence that fine-tuning alone ensures agent safety.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
