General-purpose agents perform tasks in unfamiliar environments without domain-specific manual customization. Yet no study has systematically measured how agent architecture shapes performance across heterogeneous protocols and diverse unfamiliar environments. This is the first systematic study, comparing tool-calling, MCP, code-generation, and CLI agents on the same benchmarks with the same models. Two gaps blocked such a study: existing harnesses require per-benchmark wiring or fixed protocol classes (web for BrowserGym, CLI for Harbor), and benchmarks themselves expect human-authored prompts, context, and integration glue. To enable this study, we contribute (1) a unifying protocol that bridges existing benchmark and agent protocols; (2) an evaluation harness that surfaces any benchmark to any general-purpose agent and backbone model; and (3) the first Open General Agent Leaderboard of agent configurations, a full factorial over 5 agent architectures x 5 backbone LLMs (three closed-source, two open-weight) x 6 benchmarks spanning software engineering, customer service, deep research, and personal assistance. We find that (i) general agents adapt to every tested domain without per-domain customization; (ii) agent architecture choice swings results by up to 12pp within a single model, yet backbone model choice dominates overall performance; (iii) on 4 of 6 tested benchmarks, top general agents are indistinguishable from the leading heavily-customized domain-specific agents; (iv) open-weight models tested exhibit "generality sinks" absent from frontier closed-source models: they consistently collapse on specific agent architectures or benchmarks; (v) a behavioral failure analysis reveals architecture-distinctive error signatures that aggregate scoring cannot discriminate. Code, harness, leaderboard, and traces are at https://www.exgentic.ai.
ASA: Backbone-Training-Free Representation Engineering for Tool-Calling Agents
Renmin University of China(中国人民大学);University of Macau(澳门大学);Central South University(中南大学);Jiangxi Normal University(江西师范大学);Macau Millennium College(澳门 millennium 学院);Peking University(北京大学)
Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents
Qwen Large Model Application Team, Alibaba(阿里云大模型应用团队);Alibaba Zhejiang University(阿里巴巴浙江大学);University of Waterloo(多伦多大学);Vector Institute(向量研究所);Nanjing University(南京大学);Binjiang Institute of Zhejiang University(浙江大学滨江学院)
MemRouter: Memory-as-Embedding Routing for Long-Term Conversational Agents
University of Central Florida(佛罗里达大学中央分校);University of Rochester(罗切斯特大学);University of Illinois at Chicago(伊利诺伊大学香槟分校);Case Western Reserve University(凯斯西储大学)
Benchmarking LLM Judges for Mobile Agent Evaluation
Mila – Québec AI Institute(米拉-魁北克人工智能研究所);Concordia University(康考迪亚大学);University of Toronto(多伦多大学);Shanghai University(上海大学);McMaster University(麦克马斯特大学)
From Soliloquy to Agora: Memory-Enhanced LLM Agents with Decentralized Debate for Optimization Modeling
Antai College of Economics and Management(上海交通大学安泰经济管理学院);University of Chicago Booth School of Business(芝加哥大学布斯商学院);The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)(香港中文大学(深圳));School of Economics and Management(清华大学经济管理学院)
BioMedArena: An Open-source Toolkit for Building and Evaluating Biomedical Deep Research Agents
University of Oxford(牛津大学);University College London(伦敦大学学院);Technical University of Munich(慕尼黑技术大学);Oxford-Suzhou Centre for Advanced Research, China(牛津-苏州先进研究中心)
College of Computer Science and Software Engineering, Shenzhen University, China(深圳大学计算机科学与软件工程学院);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), China(广东省人工智能与数字经济发展实验室(深圳));School of Information Technology, Carleton University, Canada(卡尔顿大学信息技术学院)
MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphs
University of New South Wales(新南威尔士大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
TimeEvo: Failure-Driven Self-Evolution of a Time Series Agent
University of Illinois at Chicago(伊利诺伊大学芝加哥分校);Visa Research(Visa研究院);University of Florida(佛罗里达大学);The Ohio State University(俄亥俄州立大学);Case Western Reserve University(凯斯西储大学)