大模型推理能力

CURE: Critical-Token-Guided Re-Concatenation for Entropy-Collapse Prevention

Qingbin Li, Rongkun Xue, Jie Wang, Ming Zhou, Zhi Li, Xiaofeng Ji, Yongqi Wang, Miao Liu, Zheming Yang, Minghui Qiu, Jing Yang

机构 * ByteDance（字节跳动） 纠错

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.AI、cs.LG

Post-Training Large Language Models via Reinforcement Learning from Self-Feedback

Carel van Niekerk, Renato Vukovic, Benjamin Matthias Ruppik, Hsien-chin Lin, Milica Gašić

机构 * Heinrich Heine Universität（海因里希·海因大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?

Yizhe Zhang, Richard Bai, Zijin Gu, Ruixiang Zhang, Jiatao Gu, Emmanuel Abbe, Samy Bengio, Navdeep Jaitly

机构 * Apple（苹果公司） 纠错

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.LG

Comments 10 pages for the main text

Thinkless: LLM Learns When to Think

Gongfan Fang, Xinyin Ma, Xinchao Wang

机构 * National University of Singapore（新加坡国立大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Perspective Transition of Large Language Models for Solving Subjective Tasks

Xiaolong Wang, Yuanchi Zhang, Ziyue Wang, Yuzhuang Xu, Fuwen Luo, Yile Wang, Peng Li, Yang Liu

机构 * Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China（计算机科学与技术系，人工智能研究院，清华大学，北京，中国） ； Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China（人工智能产业研究院（AIR），清华大学，北京，中国） ； Jiuquan Satellite Launch Center (JSLC), Gansu, China（酒泉卫星发射中心（JSLC），甘肃，中国） ； Harbin Institute of Technology, Harbin, China（哈尔滨工业大学，哈尔滨，中国） ； College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China（计算机科学与软件工程学院，深圳大学，深圳，中国） ； Jiangsu Collaborative Innovation Center for Language Competence, Jiangsu, China（江苏省语言能力协同创新中心，江苏，中国） ； Tencent Inc, China（腾讯公司，中国） 纠错

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 Findings

Beyond path selection: Better LLMs for Scientific Information Extraction with MimicSFT and Relevance and Rule-induced(R$^2$)GRPO

Ran Li, Shimin Di, Yuchen Liu, Chen Jing, Yu Qiu, Lei Chen

机构 * HKUST Hong Kong SAR, China（香港科技大学） ； SEU Jiangsu, China（江苏大学） ； HKUST(GZ) Guangzhou, China（香港科技大学（广州）） ； Zhipu AI Beijing, China（智谱AI） ； HKUST(GZ), HKUST Guangzhou, China（香港科技大学（广州）、香港科技大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Pause Tokens Strictly Increase the Expressivity of Constant-Depth Transformers

Charles London, Varun Kanade

机构 * Department of Computer Science University of Oxford（计算机科学系牛津大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.LG

On the Effect of Negative Gradient in Group Relative Deep Reinforcement Optimization

Wenlong Deng, Yi Ren, Muchen Li, Danica J. Sutherland, Xiaoxiao Li, Christos Thrampoulidis

机构 * University of British Columbia（不列颠哥伦比亚大学） ； Vector Institute（向量研究所） 纠错

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL、cs.LG

Anchored Diffusion Language Model

Litu Rout, Constantine Caramanis, Sanjay Shakkottai

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.LG

Comments Preprint

Scaling Synthetic Data Creation with 1,000,000,000 Personas

Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, Dong Yu

机构 * Tencent AI Lab（腾讯AI实验室） 纠错

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.LG

Comments Work in progress

Subset Selection for Fine-Tuning: A Utility-Diversity Balanced Approach for Mathematical Domain Adaptation

Madhav Kotecha, Vijendra Kumar Vaishya, Smita Gautam, Suraj Racha

机构 * Department of Computer Science and Engineering IIT Bombay（印度理工学院班加罗尔计算机科学与工程系） ； Koita Centre for Digital Health IIT Bombay（IIT班加罗尔数字健康中心） 纠错

专题命中 数学推理 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.AI、cs.LG

Comments 9 pages

Sleep-time Compute: Beyond Inference Scaling at Test-time

Kevin Lin, Charlie Snell, Yu Wang, Charles Packer, Sarah Wooders, Ion Stoica, Joseph E. Gonzalez

专题命中 数学推理 ：reasoning(abstract)；test-time compute(abstract)；分类 cs.CL、cs.AI

Comments Code and data released at: https://github.com/letta-ai/sleep-time-compute

ReTool: Reinforcement Learning for Strategic Tool Use in LLMs

Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, Wanjun Zhong

专题命中 数学推理 ：reasoning(abstract)；self-correction(abstract)；分类 cs.CL、cs.AI

Comments fix typos

Breaking the Data Barrier -- Building GUI Agents Through Task Generalization

Junlei Zhang, Zichen Ding, Chang Ma, Zijie Chen, Qiushi Sun, Zhenzhong Lan, Junxian He

专题命中 数学推理 ：reasoning(abstract)；planning(abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 11 figures

Eliminating Position Bias of Language Models: A Mechanistic Approach

Ziqi Wang, Hanlin Zhang, Xiner Li, Kuan-Hao Huang, Chi Han, Shuiwang Ji, Sham M. Kakade, Hao Peng, Heng Ji

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL、cs.LG

Comments 26 pages, 6 figures, 15 tables

The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models

Matthew Renze, Erhan Guven

专题命中 数学推理 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Journal ref 2nd International Conference on Foundation and Large Language Models (FLLM 2024), pp.476-483

Number Cookbook: Number Understanding of Language Models and How to Improve It

Haotong Yang, Yi Hu, Shijia Kang, Zhouchen Lin, Muhan Zhang

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2025 poster

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

Xingwu Sun, Yanfeng Chen, Yiqing Huang, Ruobing Xie, Jiaqi Zhu, Kai Zhang, Shuaipeng Li, Zhen Yang, Jonny Han, Xiaobo Shu, Jiahao Bu, Zhongzhi Chen, Xuemeng Huang, Fengzong Lian, Saiyong Yang, Jianfeng Yan, Yuyuan Zeng, Xiaoqin Ren, Chao Yu, Lulu Wu, Yue Mao, Jun Xia, Tao Yang, Suncong Zheng, Kan Wu, Dian Jiao, Jinbao Xue, Xipeng Zhang, Decheng Wu, Kai Liu, Dengpeng Wu, Guanghui Xu, Shaohua Chen, Shuang Chen, Xiao Feng, Yigeng Hong, Junqiang Zheng, Chengcheng Xu, Zongwei Li, Xiong Kuang, Jianglu Hu, Yiqi Chen, Yuchi Deng, Guiyang Li, Ao Liu, Chenchen Zhang, Shihui Hu, Zilong Zhao, Zifan Wu, Yao Ding, Weichao Wang, Han Liu, Roberts Wang, Hao Fei, Peijie Yu, Ze Zhao, Xun Cao, Hai Wang, Fusheng Xiang, Mengyuan Huang, Zhiyuan Xiong, Bin Hu, Xuebin Hou, Lei Jiang, Jianqiang Ma, Jiajia Wu, Yaping Deng, Yi Shen, Qian Wang, Weijie Liu, Jie Liu, Meng Chen, Liang Dong, Weiwen Jia, Hu Chen, Feifei Liu, Rui Yuan, Huilin Xu, Zhenxiang Yan, Tengfei Cao, Zhichao Hu, Xinhua Feng, Dong Du, Tinghao Yu, Yangyu Tao, Feng Zhang, Jianchen Zhu, Chengzhong Xu, Xirui Li, Chong Zha, Wen Ouyang, Yinben Xia, Xiang Li, Zekun He, Rongpeng Chen, Jiawei Song, Ruibin Chen, Fan Jiang, Chongqing Zhao, Bo Wang, Hao Gong, Rong Gan, Winston Hu, Zhanhui Kang, Yong Yang, Yuhong Liu, Di Wang, Jie Jiang

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 4 Figures

Library Learning Doesn't: The Curious Case of the Single-Use "Library"

Ian Berlot-Attwell, Frank Rudzicz, Xujie Si

专题命中 数学推理 ：reasoning(abstract)；self-correction(abstract)；分类 cs.CL、cs.LG

Comments 24 pages, 7 figures. Accepted to the 4th MATH-AI Workshop at NeurIPS'24

Can LLMs plan paths with extra hints from solvers?

Erik Wu, Sayan Mitra

专题命中 数学推理 ：reasoning(abstract)；planning(abstract)；分类 cs.CL、cs.AI

CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions

Jun Rao, Xuebo Liu, Lian Lian, Shengjun Cheng, Yunjie Liao, Min Zhang

专题命中 数学推理 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024

PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning

Qibin Wang, Xiaolin Hu, Weikai Xu, Wei Liu, Jian Luan, Bin Wang

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL、cs.LG

MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems

Bin Lei, Yi Zhang, Shan Zuo, Ali Payani, Caiwen Ding

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

CharED: Character-wise Ensemble Decoding for Large Language Models

Kevin Gu, Eva Tuecke, Dmitriy Katz, Raya Horesh, David Alvarez-Melis, Mikhail Yurochkin

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.LG

Comments 9 pages, 4 figures

The neural correlates of logical-mathematical symbol systems processing resemble that of spatial cognition more than natural language processing

Yuannan Li, Shan Xu, Jia Liu

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks

Mahdi Sabbaghi, George Pappas, Hamed Hassani, Surbhi Goel

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.LG

Comments 32 pages, 16 figures

Investigating Symbolic Capabilities of Large Language Models

Neisarg Dave, Daniel Kifer, C. Lee Giles, Ankur Mali

专题命中 数学推理 ：reasoning(abstract)；planning(abstract)；分类 cs.CL、cs.LG

Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving

Aniket Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy Lillicrap, Danilo Rezende, Yoshua Bengio, Michael Mozer, Sanjeev Arora

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.AI、cs.LG

Comments Preprint. Under review

VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context

Yunxin Li, Baotian Hu, Haoyuan Shi, Wei Wang, Longyue Wang, Min Zhang

专题命中 数学推理 ：reasoning(abstract)；planning(abstract)；分类 cs.CL、cs.AI

Comments 17 pages; Accepted by ICML 2024

Learning From Mistakes Makes LLM Better Reasoner

Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, Weizhu Chen

专题命中 数学推理 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Comments 23 pages, 13 figures, 6 tables