arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.11337cs.AI
收藏

奖励黑客代理:用于LLM机器学习工程代理的基准评估完整性

RewardHackingAgents: Benchmarking Evaluation Integrity for LLM ML-Engineering Agents

Yonas Atinafu, Robin Cohen

AI总结:

本研究提出RewardHackingAgents基准,用于评估机器学习工程代理的评估完整性,通过显式测量评估者篡改和训练测试泄漏两种妥协向量,展示评估完整性可作为核心指标进行验证。

摘要:

大型语言模型代理越来越多地执行端到端的机器学习工程任务,其成功由单一标量测试度量标准判断。这创造了一种结构性的漏洞:代理可以通过篡改评估流程来提高报告分数,而不是改进模型。我们介绍了RewardHackingAgents,一种基于工作区的基准,使两种妥协向量显式且可测量:评估者篡改(修改度量计算或报告)和训练/测试泄漏(在训练过程中访问预留数据或标签)。每个回合都在一个新鲜的工作区中运行,具有补丁跟踪和运行时文件访问日志记录;检测器将代理报告的度量与可信参考进行比较,以分配可审计的完整性标签。在三个任务和两个LLM基础架构上,脚本攻击在完全可变的工作区中成功于两个向量;单一机制防御只能阻止一个向量;而综合机制可以阻止两者。在自然代理运行中,评估者篡改尝试出现在约50%的回合中,并被评估者锁定消除,具有25-31%的中位数运行时间开销。总体而言,我们证明了机器学习工程代理的评估完整性可以作为第一类结果进行基准测试,而不是被假设。

英文摘要:

LLM agents increasingly perform end-to-end ML engineering tasks where success is judged by a single scalar test metric. This creates a structural vulnerability: an agent can increase the reported score by compromising the evaluation pipeline rather than improving the model. We introduce RewardHackingAgents, a workspace-based benchmark that makes two compromise vectors explicit and measurable: evaluator tampering (modifying metric computation or reporting) and train/test leakage (accessing held-out data or labels during training). Each episode runs in a fresh workspace with patch tracking and runtime file-access logging; detectors compare the agent-reported metric to a trusted reference to assign auditable integrity labels. Across three tasks and two LLM backbones, scripted attacks succeed on both vectors in fully mutable workspaces; single-mechanism defenses block only one vector; and a combined regime blocks both. In natural-agent runs, evaluator-tampering attempts occur in about 50% of episodes and are eliminated by evaluator locking, with a 25-31% median runtime overhead. Overall, we demonstrate that evaluation integrity for ML-engineering agents can be benchmarked as a first-class outcome rather than assumed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑