We introduce Prediction Arena, a benchmark for evaluating AI models' predictive accuracy and decision-making by enabling them to trade autonomously on live prediction markets with real capital. Unlike synthetic benchmarks, Prediction Arena tests models in environments where trades execute on actual exchanges (Kalshi and Polymarket), providing objective ground truth that cannot be gamed or overfitted. Each model operates as an independent agent starting with $10,000, making autonomous decisions every 15-45 minutes. Over a 57-day longitudinal evaluation (January 12 to March 9, 2026), we track two cohorts: six frontier models in live trading (Cohort 1, full period) and four next-generation models in paper trading (Cohort 2, 3-day preliminary). For Cohort 1, final Kalshi returns range from -16.0% to -30.8%. Our analysis identifies a clear performance hierarchy: initial prediction accuracy and the ability to capitalize on correct predictions are the main drivers, while research volume shows no correlation with outcomes. A striking cross-platform contrast emerges from parallel Polymarket live trading: Cohort 1 models averaged only -1.1% on Polymarket vs. -22.6% on Kalshi, with grok-4-20-checkpoint achieving a 71.4% settlement win rate - the highest across any platform or cohort. gemini-3.1-pro-preview (Cohort 2), which executed zero trades on Kalshi, achieved +6.02% on Polymarket in 3 days - the best return of any model across either cohort - demonstrating that platform design has a profound effect on which models succeed. Beyond performance, we analyze computational efficiency (token usage, cycle time), settlement accuracy, exit patterns, and market preferences, providing a comprehensive view of how frontier models behave under real financial pressure.
CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs
MATS;Mila – Quebec AI Institute(Mila – 魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Astra Fellowship;ELLIS Institute Tübingen, MPI for Intelligent Systems & Tübingen AI Center;LawZero;Google DeepMind(谷歌DeepMind)
Energy-Arena: A Dynamic Benchmark for Operational Energy Forecasting
Karlsruhe Institute of Technology, Institute for Industrial Production(卡尔斯鲁厄理工学院,工业生产研究所);University of Duisburg-Essen, House of Energy Markets and Finance(杜伊斯堡-埃森大学,能源市场与金融中心);Karlsruhe Institute of Technology, Institute for Automation and Applied Informatics(卡尔斯鲁厄理工学院,自动化与应用信息研究所);Karlsruhe Institute of Technology, Institute of Statistics(卡尔斯鲁厄理工学院,统计研究所);Heidelberg Institute for Theoretical Studies(海德堡理论研究所);Marburg University, Department of Mathematics and Computer Science(马尔堡大学,数学与计算机科学系);Volatile GmbH(Volatile GmbH公司);Forschungszentrum Jülich GmbH, Institute of Climate and Energy Systems – Jülich Systems Analysis(焦耳研究中心 GmbH,气候与能源系统研究所——焦耳系统分析)
Benchmarking Robust Aggregation in Decentralized Gradient Marketplaces
Department of Computer Science and Engineering(计算机科学与工程系);Pennsylvania State University(宾夕法尼亚州立大学);Department of Computer Science(计算机科学系);Cornell University(康奈尔大学)
S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models
Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology(深圳先进技术研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Nanjing University(南京大学);Shenzhen Loop Area Institute(深圳河套学院);CentraleSupélec, Université Paris-Saclay(巴黎萨克雷大学CentraleSupélec分校);National University of Singapore(新加坡国立大学)
Cohere Labs(Cohere 实验室);Cohere(Cohere 公司);Princeton University(普林斯顿大学);Stanford University(斯坦福大学);University of Waterloo(滑铁卢大学);Massachusetts Institute of Technology(麻省理工学院);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of Washington(华盛顿大学)
Benchmarking for Deep Uplift Modeling in Online Marketing
Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东省人工智能与数字经济实验室(深圳));Tencent(腾讯);Shenzhen University(深圳大学);Renmin University of China(中国人民大学);Shenzhen Technology University(深圳技术大学)
Job-SDF: A Multi-Granularity Dataset for Job Skill Demand Forecasting and Benchmarking
University of Science and Technology of China(中国科学技术大学);Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);Baidu Inc.(百度公司);Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院);AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)AI 分支)
Benchmarking is Broken -- Don't Let AI be its Own Judge
Princeton University(普林斯顿大学);CISPA Helmholtz Center for Information Security(CISPA海德堡信息安全中心);Michigan State University(密歇根州立大学);Ohio State University(俄亥俄州立大学);Brown University(布朗大学);Massachusetts Institute of Technology(麻省理工学院);University of California, Los Angeles(加州大学洛杉矶分校);University of Tübingen(图宾根大学);Old Dominion University(旧 Dominion 大学);Technical University of Munich(慕尼黑技术大学);Cornell University(康奈尔大学);Forest AI(森林AI)
Agent Trading Arena: A Study on Numerical Understanding in LLM-Based Agents
Wuhan University of Technology(武汉理工大学);Agency for Science, Technology and Research, Singapore(新加坡科技研究局);Hubei University(湖北大学);University of Science and Technology of China(中国科学技术大学)
KernelCraft: Benchmarking for Agentic Close-to-Metal Kernel Generation on Emerging Hardware
Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom(计算机科学与技术系,剑桥大学,剑桥,英国);Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom(电气与电子工程系,伦敦帝国理工学院,伦敦,英国);School of Informatics, University of Edinburgh, Edinburgh, United Kingdom(信息学院,爱丁堡大学,爱丁堡,英国)
Multi-Agent Reinforcement Learning for Dynamic Pricing in Supply Chains: Benchmarking Strategic Agent Behaviours under Realistically Simulated Market Conditions
Informatics Institute, University of Amsterdam(阿姆斯特丹大学信息研究所);Department of Computer Science and Technology, SUNY Empire State University(SUNY帝国州大学计算机科学与技术系);KPMG N.V(KPMG荷兰公司)
Evaluating Uplift Modeling under Structural Biases: Insights into Metric Stability and Model Robustness
College of Computer Science and Software Engineering(计算机科学与软件工程学院);Shenzhen University(深圳大学);School of Computing and Information Technology(计算与信息科技学院);Great Bay University(大湾大学)