Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
Peking University（北京大学）；University of Wisconsin - Madison（威斯康星大学麦迪逊分校）；Shanghai Jiao Tong University（上海交通大学）；The University of Hong Kong（香港大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Alibaba Group（阿里巴巴集团）；University of Waterloo（滑铁卢大学）；Zhongguancun Laboratory（中关村实验室）；Engineering Research Center of Information Networks（信息网络工程研究中心）
Recent advancements in large language models (LLMs) have led to significant breakthroughs in mathematical reasoning capabilities. However, existing benchmarks like GSM8K or MATH are now being solved with high accuracy (e.g., OpenAI o1 achieves 94.8\% on MATH dataset), indicating their inadequacy for truly challenging these models. To bridge this gap, we propose a comprehensive and challenging benchmark specifically designed to assess LLMs' mathematical reasoning at the Olympiad level. Unlike existing Olympiad-related benchmarks, our dataset focuses exclusively on mathematics and comprises a vast collection of 4428 competition-level problems with rigorous human annotation. These problems are meticulously categorized into over 33 sub-domains and span more than 10 distinct difficulty levels, enabling a holistic assessment of model performance in Olympiad-mathematical reasoning. Furthermore, we conducted an in-depth analysis based on this benchmark. Our experimental results show that even the most advanced models, OpenAI o1-mini and OpenAI o1-preview, struggle with highly challenging Olympiad-level problems, with 60.54\% and 52.55\% accuracy, highlighting significant challenges in Olympiad-level mathematical reasoning.
LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning
Fudan University（复旦大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Hong Kong Polytechnic University（香港理工大学）；University of New South Wales（新南威尔士大学）；Shanghai Jiao Tong University（上海交通大学）；Stanford University（斯坦福大学）；University of California, Merced（加州大学默塞德分校）
CogAtom: From Cognitive Atoms to Olympiad-level Mathematical Reasoning in Large Language Models
School of Computer Science and Engineering, Beihang University, China（北京航空航天大学计算机科学与工程学院）；Meituan Inc.（美团公司）；Beijing Institute for General Artificial Intelligence, China（北京通用人工智能研究院）
Intern-S1-MO: Long-horizon Reasoning Agent for Olympiad?Level Mathematical Problem Solving
Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；MMLab；The Chinese University of Hong Kong（香港中文大学）；ICMAT Spanish National Research Council（西班牙国家科研 council）；The High School Affiliated to Renmin University of China（中国人民大学附属高中）；Ren Hui Academy of Beijing（北京润辉学院）
FMC: Formalization of Natural Language Mathematical Competition Problems
State Key Laboratory for Multimedia Information Processing, School of Computer Science, PKU-Anker LLM Lab, Peking University, Beijing, China（多媒体信息处理国家重点实验室，计算机科学学院，PKU-Anker LLM实验室，北京大学，北京，中国）；Paul G. Allen School of Computer Science（保罗·G·阿伦计算机科学学校）；Engineering, University of Washington, U.S.A.（工程，华盛顿大学，美国）
Georgia Institute of Technology（佐治亚理工学院）；Seoul National University（首尔国立大学）；Microsoft（微软）；Massachusetts Institute of Technology（麻省理工学院）；Brown University（布朗大学）
Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling
Shanghai AI Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；Peking University（北京大学）
Electronic Circuit Principles of Large Language Models
Harbin Institute of Technology（哈尔滨工业大学）；Central South University（中南大学）；The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；ByteDance Seed (China)（字节跳动种子（中国））
An Investigation of Robustness of LLMs in Mathematical Reasoning: Benchmarking with Mathematically-Equivalent Transformation of Advanced Mathematical Problems
Department of Computer Science University of Illinois Urbana–Champaign（计算机科学系伊利诺伊大学厄巴纳-香槟分校）；Department of Computer Science Stanford University（计算机科学系斯坦福大学）
OMIBench: Benchmarking Olympiad-Level Multi-Image Reasoning in Large Vision-Language Model
Research Center for Social Computing and Interactive Robotics（社会计算与交互机器人研究室）；Harbin Institute of Technology（哈尔滨工业大学）；Central South University（中南大学）；Fudan University（复旦大学）；The University of Hong Kong（香港大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Text Computing and Cognitive Intelligence Ministry of Education Engineering Research Center（教育部文本计算与认知智能工程研究中心）；Guizhou University（贵州大学）
UC Berkeley（伯克利大学）；Brown University（布朗大学）；Yonsei University（延世大学）；Concordia University（Concordia 大学）；Korea Institute for Advanced Study（韩国高级研究院）；UC San Diego（圣地亚哥大学）；Caltech（加州理工学院）
MATH-PT: A Math Reasoning Benchmark for European and Brazilian Portuguese
Instituto Superior Técnico, Universidade de Lisboa（里斯本大学理工学院）；Fundação Getulio Vargas（古特曼基金会）；Instituto de Telecomunicações（电信研究所）；Universidade de Coimbra, CISUC/LASI, DEI（科英布拉大学，CISUC/LASI，DEI）；Basque Center for Applied Mathematics（巴斯克应用数学中心）
