LLM agents increasingly perform end-to-end ML engineering tasks where success is judged by a single scalar test metric. This creates a structural vulnerability: an agent can increase the reported score by compromising the evaluation pipeline rather than improving the model. We introduce RewardHackingAgents, a workspace-based benchmark that makes two compromise vectors explicit and measurable: evaluator tampering (modifying metric computation or reporting) and train/test leakage (accessing held-out data or labels during training). Each episode runs in a fresh workspace with patch tracking and runtime file-access logging; detectors compare the agent-reported metric to a trusted reference to assign auditable integrity labels. Across three tasks and two LLM backbones, scripted attacks succeed on both vectors in fully mutable workspaces; single-mechanism defenses block only one vector; and a combined regime blocks both. In natural-agent runs, evaluator-tampering attempts occur in about 50% of episodes and are eliminated by evaluator locking, with a 25-31% median runtime overhead. Overall, we demonstrate that evaluation integrity for ML-engineering agents can be benchmarked as a first-class outcome rather than assumed.
MMPostTrainBench: Benchmarking Autonomous Research for Multimodal Post-Training
University of Science and Technology of China(中国科学技术大学);Alibaba Token Hub, Alibaba Group(阿里巴巴集团阿里云通义千问团队);University of the Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学)
Benchmarking Linguistic Adaptation in Comparable-Sized LLMs: A Study of Llama-3.1-8B, Mistral-7B-v0.1, and Qwen3-8B on Romanized Nepali
Dept. of Computer Science & Engineering(计算机科学与工程系);Nepal Engineering College(尼泊尔工程学院);Central Dept. of CS and IT(计算机科学与信息科技中央系);Tribhuvan University(特里布文大学)
Into the Unknown: Benchmarking AI agents on open scientific research problems
Carnegie Mellon University(卡内基梅隆大学);University of Minnesota(明尼苏达大学);Stanford University(斯坦福大学);Trillion Labs;Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室);Sandia National Laboratory(桑迪亚国家实验室);Lawrence Berkeley National Laboratory(劳伦斯伯克利国家实验室);Ames National Laboratory(艾姆斯国家实验室);Iowa State University(爱荷华州立大学);University of Colorado Boulder(科罗拉多大学博尔德分校);Michigan State University(密歇根州立大学);Lawrence Livermore National Laboratory(劳伦斯利弗莫尔国家实验室);TelePIX;Johns Hopkins University(约翰斯·霍普金斯大学);NEC Laboratories Europe(NEC欧洲实验室);Ss.Cyril and Methodius University of Skopje(斯科普里圣西里尔和圣美多德大学)
本研究推出Into the Unknown基准,测试9个前沿智能体撰写开放科学问题后续论文的能力,发现无智能体达到领域科学家标准,仍有较大提升空间。