LLM agents are increasingly deployed in long-horizon, complex environments to solve challenging problems, but this expansion exposes them to long-horizon attacks that exploit multi-turn user-agent-environment interactions to achieve objectives infeasible in single-turn settings. To measure agent vulnerabilities to such risks, we present AgentLAB, the first benchmark dedicated to evaluating LLM agent susceptibility to adaptive, long-horizon attacks. Currently, AgentLAB supports five novel attack types including intent hijacking, tool chaining, task injection, objective drifting, and memory poisoning, spanning 28 realistic agentic environments, and 644 security test cases. Leveraging AgentLAB, we evaluate representative LLM agents and find that they remain highly susceptible to long-horizon attacks; moreover, defenses designed for single-turn interactions fail to reliably mitigate long-horizon threats. We anticipate that AgentLAB will serve as a valuable benchmark for tracking progress on securing LLM agents in practical settings. The benchmark is publicly available at https://tanqiujiang.github.io/AgentLAB_main.
Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environment
Georgia Institute of Technology（佐治亚理工学院）；The Fin AI；Stevens Institute of Technology（史蒂文斯理工学院）；Columbia University（哥伦比亚大学）；George Mason University（乔治·马歇尔大学）；McGill University（麦吉尔大学）；Mohamed bin Zayed University of Artificial Intelligence（莫扎大学人工智能学院）；California State University, Monterey Bay（加州州立大学蒙特雷湾分校）；University of Manchester（曼彻斯特大学）；Mila – Quebec Artificial Intelligence Institute（魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）
Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Tsinghua University（清华大学）；Peking University（北京大学）；Southeast University（东南大学）
MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations
MemTensor (Shanghai) Technology Co., Ltd.（墨天轮（上海）科技有限公司）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Renmin University of China（中国人民大学）
Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所信息处理实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Kuaishou Technology（快手科技）
LabOSBench: Benchmarking Computer Use Agents for Scientific Instrument Control
Shenzhen Loop Area Institute（深圳河套学院）；Dalian University of Technology（大连理工大学）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong University of Science and Technology（香港科技大学）
Words to World: Benchmarking LLMs in Interactive Physical Environments
Michigan State University（密歇根州立大学）；Zhejiang University（浙江大学）；Microsoft（微软）；University of Rochester（罗切斯特大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）
Claw-Anything: Benchmarking Always-On Personal Assistants with Broader Access to User's Digital World
Beijing Institute of Technology（北京理工大学）；Huawei Technologies Co., Ltd（华为技术有限公司）；Peking University（北京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；MIT-IBM Watson AI Lab, IBM Research（MIT-IBM沃森人工智能实验室，IBM研究）；Stony Brook University（石溪大学）；Brookhaven National Laboratory（布鲁赫斯国家实验室）
MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environments
Department of Computer Science & Technology, Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint Center for ML（计算机科学与技术系、人工智能研究院、BNRist中心、THBI实验室、清华-博世联合机器学习中心）；Shanghai Key Lab. of Multidimensional Info. Processing（上海多维信息处理重点实验室）
Socratic-PRMBench: Benchmarking Process Reward Models with Systematic Reasoning Patterns
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Tongyi Lab, Alibaba Group（阿里云实验室）
IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents
School of Computing, National University of Singapore（新加坡国立大学计算机学院）；School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）智能科学与工程学院）；Zhejiang University（浙江大学）；State Key Lab of CAD&CG, Zhejiang University（浙江大学计算机辅助设计与图形学国家重点实验室）
SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
Tsinghua University（清华大学）；Chongqing University（重庆大学）；Peking University（北京大学）；ZenoMind AI；Xi’an Jiaotong University（西安交通大学）；Beijing Institute of Technology（北京理工大学）；Southeast University（东南大学）；Shanghai Jiao Tong University（上海交通大学）；Joy Future Academy（京东探索研究院）；The University of Hong Kong（香港大学）
SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios
FPT Software AI Center（FPT软件人工智能中心）；School of Computing and Information Systems（计算与信息系统学院）；University of Melbourne（墨尔本大学）；Center of AI Research（人工智能研究中心）；VinUniversity（文大学）
HSSBench: Benchmarking Humanities and Social Sciences Ability for Multimodal Large Language Models
National Engineering Research Center for Software Engineering, Peking University（北京大学软件工程国家工程研究中心）；Institute of Artificial Intelligence, China Telecom (TeleAI)（中国电信人工智能研究院）；Tsinghua University（清华大学）；Chinese Academy of Sciences（中国科学院）；University of British Columbia（不列颠哥伦比亚大学）；Renmin University of China（中国人民大学）
