Enabling Large Language Models (LLMs) to handle a wider range of complex tasks (e.g., coding, math) has drawn great attention from many researchers. As LLMs continue to evolve, merely increasing the number of model parameters yields diminishing performance improvements and heavy computational costs. Recently, OpenAI's o1 model has shown that inference strategies (i.e., Test-time Compute methods) can also significantly enhance the reasoning capabilities of LLMs. However, the mechanisms behind these methods are still unexplored. In our work, to investigate the reasoning patterns of o1, we compare o1 with existing Test-time Compute methods (BoN, Step-wise BoN, Agent Workflow, and Self-Refine) by using OpenAI's GPT-4o as a backbone on general reasoning benchmarks in three domains (i.e., math, coding, commonsense reasoning). Specifically, first, our experiments show that the o1 model has achieved the best performance on most datasets. Second, as for the methods of searching diverse responses (e.g., BoN), we find the reward models' capability and the search space both limit the upper boundary of these methods. Third, as for the methods that break the problem into many sub-problems, the Agent Workflow has achieved better performance than Step-wise BoN due to the domain-specific system prompt for planning better reasoning processes. Fourth, it is worth mentioning that we have summarized six reasoning patterns of o1, and provided a detailed analysis on several reasoning benchmarks.
H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking
Duke University（杜克大学）；Accenture（埃森哲）；Accenture Security（埃森哲安全）
Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Guanghua Law School, Zhejiang University（浙江大学光华法学院）；School of Software Technology, Zhejiang University（浙江大学软件学院）；Law & AI Lab, Zhejiang University（浙江大学法律与人工智能实验室）；Shanghai AI Laboratory（上海人工智能实验室）
Can OpenAI o1 Reason Well in Ophthalmology? A 6,990-Question Head-to-Head Evaluation Study
National University of Singapore（新加坡国立大学）；Singapore Eye Research Institute（新加坡眼科研究所）；Yale University（耶鲁大学）；Cleveland Clinic（克利夫兰诊所）；Zhejiang University（浙江大学）
DeepSeek-R1 Outperforms Gemini 2.0 Pro, OpenAI o1, and o3-mini in Bilingual Complex Ophthalmology Reasoning
School of Optometry, The Hong Kong Polytechnic University（香港理工大学视光学院）；Zhejiang University（浙江大学）；Research Centre for SHARP Vision (RCSV), The Hong Kong Polytechnic University（香港理工大学SHARP视觉研究中心）；Centre for Eye and Vision Research (CEVR)（眼科与视觉研究中心）
Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
Peking University（北京大学）；University of Wisconsin - Madison（威斯康星大学麦迪逊分校）；Shanghai Jiao Tong University（上海交通大学）；The University of Hong Kong（香港大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Alibaba Group（阿里巴巴集团）；University of Waterloo（滑铁卢大学）；Zhongguancun Laboratory（中关村实验室）；Engineering Research Center of Information Networks（信息网络工程研究中心）
Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judge
City University of Hong Kong（香港城市大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；McGill University（麦吉尔大学）；MILA
Evaluation empirique de la sécurisation et de l'alignement de ChatGPT et Gemini: analyse comparative des vulnérabilités par expérimentations de jailbreaks
Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Models
AI Agent Lab, Vokram Group（AI代理实验室，Vokram集团）；Purdue University（普渡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Minnesota（明尼苏达大学）；Emory University（埃默里大学）；Imperial College London（伦敦帝国学院）
OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
University College London（伦敦大学学院）；University of Liverpool（利物浦大学）；Shanghai Jiao Tong University（上海交通大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Westlake University（西湖大学）
Evaluating Text Summaries Generated by Large Language Models Using OpenAI's GPT
University of Colorado - Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；University of Kansas（堪萨斯大学）；DataCrunch Lab, LLC（DataCrunch实验室有限责任公司）；University of Florida（佛罗里达大学）
MMATH: A Multilingual Benchmark for Mathematical Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学首都人工智能学院）；iFLYTEK Research (Central China), iFLYTEK Co., Ltd.（iFLYTEK中央研究院、iFLYTEK有限公司）
Advances in LLM Reasoning Enable Flexibility in Clinical Problem-Solving
Weill Institute of Neurology and Neurosciences（威尔神经与神经科学研究所）；Carle Illinois College of Medicine（卡乐伊利诺伊医学院）；University of California, San Francisco（加州大学旧金山分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Stanford University（斯坦福大学）；University of Dundee（邓迪大学）；Stevens Institute of Technology（史蒂文斯理工学院）；University of California Berkeley（加州大学伯克利分校）
Evaluating Large Language Models on the Spanish Medical Intern Resident (MIR) Examination 2024/2025:A Comparative Analysis of Clinical Reasoning and Knowledge Application
Can Multi-modal (reasoning) LLMs detect document manipulation?
Duke University（杜克大学）；Indian Institute of Technology, Roorkee（印度理工学院，罗尔基）；New York University（纽约大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Wisconsin Madison（威斯康星大学麦迪逊分校）；Columnbia University（哥伦比亚大学）
