arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.22243cs.AIcs.CRcs.LG
收藏

智能体安全评估中的捷径敏感性

Shortcut Sensitivity in Agent-Safety Evaluations

Kontext(康泰克斯)

Hasan Demirkiran, Jens Ernstberger

AI总结:

该研究提出三级捷径审计框架,审计LinuxArena等数据集发现防护措施决策及评估性能受基准表示影响,建议采用域组留一法等评估方法。

摘要:

AI智能体通过工具执行动作,部分动作具有敏感性,因此监督这些动作的防护措施必须经过可靠评估。然而,用于评估防护措施的数据可能包含一些线索,这些线索无需测量预期的安全构造即可使标签具有可预测性,且这些线索也会影响防护措施的决策。我们提出了一个三级捷径审计框架,并对完整的LinuxArena轨迹和两个TS-Bench衍生的步骤数据集进行了审计。我们区分了标签可预测性、观测防护关联和配对决策敏感性。在LinuxArena中,较短的平均shell命令强烈预测轨迹标签;在AgentDojo-Traj中,精确工具查找的F1值在混合域折叠下为0.901,在域留一法下降至0.495;在AgentHarm-Traj中,发布的TS-Guard的F1值在行加权下为0.902,在等交互加权下降至0.719,而平衡准确率则上升。随后,我们在冻结的64步AgentDojo-Traj队列上,使用重复的交错Haiku调用,在7种配置的筛选中测试最强的安全行标记关联。重写<INFORMATION>标记使拦截率降低8.33个百分点,而不透明工具别名则使拦截率提高11.46个百分点;所有不安全调用仍被拦截,这些变化涉及对继承标签的误报,在减去格式安慰剂效应后,两种效应仍非零,探索性95%区间不包含零,相同的编辑也会改变对继承标签的测量准确率。这直接证明,基准表示会实质性改变防护措施的决策和测量评估性能;Haiku规模的效应在Luna或确定性Q8 TS-Guard中无法复现。我们建议采用域组留一法、评估单元指标和安慰剂控制的配对干预。

英文摘要:

AI agents take actions through tools. Some actions are sensitive, so guards that supervise them must themselves be evaluated reliably. Yet the data used to evaluate a guard can include cues that make labels predictable without measuring the intended safety construct, and those cues can also influence guard decisions. We introduce a three-level shortcut-auditing framework and audit completed LinuxArena trajectories and two TS-Bench-derived step datasets. We distinguish label predictability, observational guard association, and paired decision sensitivity. In LinuxArena, shorter average shell commands strongly predict trajectory labels. In AgentDojo-Traj, exact-tool lookup falls from F_1=.901 with mixed-domain folds to .495 under domain holdout. In AgentHarm-Traj, released TS-Guard F_1 falls from .902 under row weighting to .719 with equal interaction weighting, while balanced accuracy rises. We then test the strongest safe-row marker association in a seven-configuration screen on a frozen 64-step AgentDojo-Traj cohort using repeated, interleaved Haiku calls. Rewriting the <INFORMATION> marker lowers blocking by 8.33 percentage points, while opaque tool aliases raise it by 11.46 points. All unsafe calls remain blocked; the changes concern false positives against inherited labels. Both effects remain nonzero after subtracting the formatting-placebo effect, with exploratory 95% ranges excluding zero. The same edits also change measured accuracy against inherited labels. This provides direct evidence that benchmark representation can materially change a guard's decisions and measured evaluation performance. The Haiku-sized effects do not reproduce in Luna or deterministic Q8 TS-Guard. We recommend domain group holdouts, evaluation-unit metrics, and placebo-controlled paired interventions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑