大模型推理能力

Discriminative Policy Optimization for Token-Level Reward Models

Hongzhan Chen, Tao Yang, Shiping Gao, Ruijun Chen, Xiaojun Quan, Hongtao Tian, Ting Yao

机构 * Hongzhan Chen（洪湛 Chen） ； Tao Yang（杨 Tao） ； Shiping Gao（高世平） ； Ruijun Chen（陈瑞俊） ； Xiaojun Quan（钱小军） ； Hongtao Tian（田洪涛） ； Ting Yao（姚婷） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Comments ICML 2025

The Climb Carves Wisdom Deeper Than the Summit: On the Noisy Rewards in Learning to Reason

Ang Lv, Ruobing Xie, Xingwu Sun, Zhanhui Kang, Rui Yan

机构 * GSAI, Renmin University of China（清华大学） ； Large Language Model Department, Tencent（腾讯大语言模型部门） ； University of Macau（澳门大学） ； School of Computer Science, Wuhan University（武汉大学计算机学院） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Comments Preprint

AI Mathematician: Towards Fully Automated Frontier Mathematical Research

Yuanhang Liu, Yanxing Huang, Yanqiao Wang, Peng Li, Yang Liu

机构 * Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China（计算机科学与技术系，人工智能研究院，清华大学，北京，中国） ； Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China（人工智能产业研究院（AIR），清华大学，北京，中国） ； Department of Mathematical Sciences, Tsinghua University（数学科学系，清华大学） ； Qiuzhen College, Tsinghua University（齐臻学院，清华大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI

Comments 95 pages, 1 figure

Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Data

Yunhao Tang, Sid Wang, Lovish Madaan, Rémi Munos

专题命中 数学推理 ：chain-of-thought(abstract)；分类 cs.LG

PATS: Process-Level Adaptive Thinking Mode Switching

Yi Wang, Junxiao Liu, Shimao Zhang, Jiajun Chen, Shujian Huang

机构 * National Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家实验室，南京大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Unearthing Large Scale Domain-Specific Knowledge from Public Corpora

Zhaoye Fei, Yunfan Shao, Linyang Li, Zhiyuan Zeng, Conghui He, Qipeng Guo, Hang Yan, Dahua Lin, Xipeng Qiu

机构 * School of Computer Science, Fudan University（复旦大学计算机科学学院） ； Shanghai AI Laboratory（上海人工智能实验室） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Comments We have released the full data (total of 735GB) in https://huggingface.co/datasets/Query-of-CC/Retrieve-Pile and partial data (about 40GB) in https://huggingface.co/datasets/Query-of-CC/knowledge_pile

1bit-Merging: Dynamic Quantized Merging for Large Language Models

Shuqi Liu, Yuxuan Yao, Bowei He, Zehua Liu, Xiongwei Han, Mingxuan Yuan, Han Wu, Linqi Song

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

OSoRA: Output-Dimension and Singular-Value Initialized Low-Rank Adaptation

Jialong Han, Si Zhang, Ke Zhang

机构 * SIST, ShanghaiTech University（上海科技大学信息科学与技术学院） ； SKLP, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection

Yibo Yan, Shen Wang, Jiahao Huo, Philip S. Yu, Xuming Hu, Qingsong Wen

机构 * Squirrel Ai Learning ； The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）） ； The Hong Kong University of Science and Technology（香港科学与技术大学） ； University of Illinois at Chicago（伊利诺伊大学香槟分校） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by The 63rd Annual Meeting of the Association for Computational Linguistics (ACL Industry 2025, Oral Presentation)

InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model

Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Ziyu Liu, Shengyuan Ding, Shenxi Wu, Yubo Ma, Haodong Duan, Wenwei Zhang, Kai Chen, Dahua Lin, Jiaqi Wang

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； The Chinese University of Hong Kong（香港中文大学） ； Shanghai Jiao Tong University（上海交通大学） ； Nanjing University（南京大学） ； Fudan University（复旦大学） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Comments ACL 2025 Findings

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training

Zheheng Luo, Xin Zhang, Xiao Liu, Haoling Li, Yeyun Gong, Chen Qi, Peng Cheng

机构 * The University of Manchester（曼彻斯特大学） ； Microsoft（微软） ； Tsinghua University（清华大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to ACL 2025

Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics

Yaniv Nikankin, Anja Reusch, Aaron Mueller, Yonatan Belinkov

机构 * Technion – Israel Institute of Technology（技术离子-以色列理工学院） ； Northeastern University（东北大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem

Fan Liu, Zherui Yang, Cancheng Liu, Tianrui Song, Xiaofeng Gao, Hao Liu

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI

The Hallucination Tax of Reinforcement Finetuning

Linxin Song, Taiwei Shi, Jieyu Zhao

机构 * University of Southern California（南加州大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

MoL for LLMs: Dual-Loss Optimization to Enhance Domain Expertise While Preserving General Capabilities

Jingxue Chen, Qingkun Tang, Qianchun Lu, Siyuan Fang

机构 * Wired Product Operation Division, ZTE Corporation（中兴通讯无线产品运营部） ； Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Multi2: Multi-Agent Test-Time Scalable Framework for Multi-Document Processing

Juntai Cao, Xiang Zhang, Raymond Li, Chuyuan Li, Chenyu You, Shafiq Joty, Giuseppe Carenini

机构 * University of British Columbia（不列颠哥伦比亚大学） ； Stony Brook University（石溪大学） ； Salesforce Research（Salesforce研究） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimization

Minghan Chen, Guikun Chen, Wenguan Wang, Yi Yang

机构 * ReLER Lab, CCAI, Zhejiang University（ReLER实验室、中国人工智能学会、浙江大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI

Comments On going project

Toward Evaluative Thinking: Meta Policy Optimization with Evolving Reward Models

Zae Myung Kim, Chanwoo Park, Vipul Raheja, Suin Kim, Dongyeop Kang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Comments Code and data: https://github.com/minnesotanlp/mpo

Agent-as-a-Service based on Agent Network

Yuhan Zhu, Haojie Liu, Jian Wang, Bing Li, Zikang Yin, Yefei Liao

机构 * School of Computer Science Wuhan University（武汉大学计算机学院） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI

Comments work in progress

Preliminary Explorations with GPT-4o(mni) Native Image Generation

Pu Cao, Feng Zhou, Junyi Ji, Qingye Kong, Zhixiang Lv, Mingjian Zhang, Xuekun Zhao, Siqi Wu, Yinghui Lin, Qing Song, Lu Yang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI

SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning

Tianjian Li, Daniel Khashabi

机构 * Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Comments To appear in ICML 2025

PICO: Secure Transformers via Robust Prompt Isolation and Cybersecurity Oversight

Ben Goertzel, Paulos Yibelo

机构 * Ben Goertzel ； Paulos Yibelo 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI

Parameter-Efficient Checkpoint Merging via Metrics-Weighted Averaging

Shi Jie Yu, Sehyun Choi

专题命中 数学推理 ：reasoning(abstract)；分类 cs.LG

SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM

Xiaojiang Zhang, Jinghui Wang, Zifei Cheng, Wenhao Zhuang, Zheng Lin, Minglei Zhang, Shaojie Wang, Yinghan Cui, Chao Wang, Junyi Peng, Shimiao Jiang, Shiqi Kuang, Shouyu Yin, Chaohang Wen, Haotian Zhang, Bin Chen, Bing Yu

机构 * Kuaishou Kwaipilot Team（快手Kwaipilot团队） ； HuggingFace 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.LG

Benchmarking Large Language Models for Calculus Problem-Solving: A Comparative Analysis

In Hak Moon

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B

Xue Zhang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI

Comments 6 pages, 2 figures. Conducted as part of research on alignment techniques for language models

UNDO: Understanding Distillation as Optimization

Kushal Jain, Piyushi Goyal, Kumar Shridhar

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback

Wei Shen, Guanlin Liu, Zheng Wu, Ruofei Zhu, Qingping Yang, Chao Xin, Yu Yue, Lin Yan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.LG

Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains

Yi Su, Dian Yu, Linfeng Song, Juntao Li, Haitao Mi, Zhaopeng Tu, Min Zhang, Dong Yu

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL

Entropy-Based Adaptive Weighting for Self-Training

Xiaoxuan Wang, Yihe Deng, Mingyu Derek Ma, Wei Wang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL