OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
University College London(伦敦大学学院);University of Liverpool(利物浦大学);Shanghai Jiao Tong University(上海交通大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Westlake University(西湖大学)
Jun Wang, Meng Fang, Ziyu Wan, Muning Wen, Jiachen Zhu, Anjie Liu, Ziqin Gong, Yan Song, Lei Chen, Lionel M. Ni, Linyi Yang, Ying Wen, Weinan Zhang
In this technical report, we introduce OpenR, an open-source framework designed to integrate key components for enhancing the reasoning capabilities of large language models (LLMs). OpenR unifies data acquisition, reinforcement learning training (both online and offline), and non-autoregressive decoding into a cohesive software platform. Our goal is to establish an open-source platform and community to accelerate the development of LLM reasoning. Inspired by the success of OpenAI's o1 model, which demonstrated improved reasoning abilities through step-by-step reasoning and reinforcement learning, OpenR integrates test-time compute, reinforcement learning, and process supervision to improve reasoning in LLMs. Our work is the first to provide an open-source framework that explores the core techniques of OpenAI's o1 model with reinforcement learning, achieving advanced reasoning capabilities beyond traditional autoregressive methods. We demonstrate the efficacy of OpenR by evaluating it on the MATH dataset, utilising publicly available data and search methods. Our initial experiments confirm substantial gains, with relative improvements in reasoning and performance driven by test-time computation and reinforcement learning through process reward models. The OpenR framework, including code, models, and datasets, is accessible at https://openreasoner.github.io.
Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning
DataArc Tech Ltd.(DataArc科技有限公司);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));IDEA Research, International Digital Economy Academy(IDEA研究所,国际数字经济学院);Hithink RoyalFlush Information Network Co., Ltd(Hithink RoyalFlush信息网络有限公司)
OpenRL: A Unified Reinforcement Learning Framework
Paradigm Inc.(第四范式);Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所);School of Data Science, Fudan University(复旦大学大数据学院);Beijing University of Posts and Telecommunications(北京邮电大学)
Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubric
Qwen Large Model Application Team, Alibaba(阿里巴巴文勤大模型应用团队);Beijing University Of Posts and Telecommunications(北京邮电大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)
Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoning
Shanghai AI Lab(上海人工智能实验室);Xi’an Jiaotong University(西安交通大学);The University of Hong Kong(香港大学);Peking University(北京大学);Hong Kong University of Science and Technology(香港科技大学)
A Survey of Task-Oriented Knowledge Graph Reasoning: Status, Applications, and Prospects
School of Artificial Intelligence, Beihang University(北航人工智能学院);School of Information Science and Technology, Beijing Forestry University(北京林业大学信息科学与技术学院)
Open-RAG: Enhanced Retrieval-Augmented Reasoning with Open-Source Large Language Models
Bangladesh University of Engineering and Technology(孟加拉工程技术大学);Cohere For AI(Cohere AI研究团队);University of North Texas(北得克萨斯大学);York University(约克大学);Salesforce Research(Salesforce研究院);Qatar Computing Research Institute (QCRI)(卡塔尔计算研究所)
Georgia Institute of Technology(佐治亚理工学院);Seoul National University(首尔国立大学);Microsoft(微软);Massachusetts Institute of Technology(麻省理工学院);Brown University(布朗大学)
Investigating Advanced Reasoning of Large Language Models via Black-Box Interaction
College of Artificial Intelligence, Nanjing University of Aeronautics(人工智能学院,南京航空航天大学);School of Electronics Engineering(电子工程学院);Computer Science, Peking University(计算机科学,北京大学);Theta Health Inc.(Theta健康公司)
LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning
Fudan University(复旦大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Hong Kong Polytechnic University(香港理工大学);University of New South Wales(新南威尔士大学);Shanghai Jiao Tong University(上海交通大学);Stanford University(斯坦福大学);University of California, Merced(加州大学默塞德分校)
An Investigation of Robustness of LLMs in Mathematical Reasoning: Benchmarking with Mathematically-Equivalent Transformation of Advanced Mathematical Problems
Department of Computer Science University of Illinois Urbana–Champaign(计算机科学系伊利诺伊大学厄巴纳-香槟分校);Department of Computer Science Stanford University(计算机科学系斯坦福大学)
DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models
Shanghai Jiao Tong University(上海交通大学);SII;SPIRAL Lab(SPIRAL实验室);Generative AI Research Lab (GAIR)(生成式AI研究实验室);Shanghai Chest Hospital(上海胸科医院);Beijing Anzhen Hospital, Capital Medical University(北京安贞医院,首都医科大学)
Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning
Zhejiang University(浙江大学);Westlake University(西湖大学);Westlake Institute for Advanced Study(西湖高等研究院);Johns Hopkins University(约翰斯·霍普金斯大学);Imperial College London(帝国理工学院)
RIMO: An Easy-to-Evaluate, Hard-to-Solve Olympiad Benchmark for Advanced Mathematical Reasoning
Artificial Intelligence Thrust, Information Hub, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China(人工智能 thrust,信息中心,香港科学与技术大学(广州),广州,中国)
MMATH: A Multilingual Benchmark for Mathematical Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学首都人工智能学院);iFLYTEK Research (Central China), iFLYTEK Co., Ltd.(iFLYTEK中央研究院、iFLYTEK有限公司)