大模型推理能力

Pause Tokens Strictly Increase the Expressivity of Constant-Depth Transformers

Charles London, Varun Kanade

机构 * Department of Computer Science University of Oxford（计算机科学系牛津大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.LG

On the Effect of Negative Gradient in Group Relative Deep Reinforcement Optimization

Wenlong Deng, Yi Ren, Muchen Li, Danica J. Sutherland, Xiaoxiao Li, Christos Thrampoulidis

机构 * University of British Columbia（不列颠哥伦比亚大学） ； Vector Institute（向量研究所） 纠错

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL、cs.LG

Anchored Diffusion Language Model

Litu Rout, Constantine Caramanis, Sanjay Shakkottai

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.LG

Comments Preprint

Scaling Synthetic Data Creation with 1,000,000,000 Personas

Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, Dong Yu

机构 * Tencent AI Lab（腾讯AI实验室） 纠错

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.LG

Comments Work in progress

Subset Selection for Fine-Tuning: A Utility-Diversity Balanced Approach for Mathematical Domain Adaptation

Madhav Kotecha, Vijendra Kumar Vaishya, Smita Gautam, Suraj Racha

机构 * Department of Computer Science and Engineering IIT Bombay（印度理工学院班加罗尔计算机科学与工程系） ； Koita Centre for Digital Health IIT Bombay（IIT班加罗尔数字健康中心） 纠错

专题命中 数学推理 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.AI、cs.LG

Comments 9 pages

Sleep-time Compute: Beyond Inference Scaling at Test-time

Kevin Lin, Charlie Snell, Yu Wang, Charles Packer, Sarah Wooders, Ion Stoica, Joseph E. Gonzalez

专题命中 数学推理 ：reasoning(abstract)；test-time compute(abstract)；分类 cs.CL、cs.AI

Comments Code and data released at: https://github.com/letta-ai/sleep-time-compute

ReTool: Reinforcement Learning for Strategic Tool Use in LLMs

Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, Wanjun Zhong

专题命中 数学推理 ：reasoning(abstract)；self-correction(abstract)；分类 cs.CL、cs.AI

Comments fix typos

Breaking the Data Barrier -- Building GUI Agents Through Task Generalization

Junlei Zhang, Zichen Ding, Chang Ma, Zijie Chen, Qiushi Sun, Zhenzhong Lan, Junxian He

专题命中 数学推理 ：reasoning(abstract)；planning(abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 11 figures

Eliminating Position Bias of Language Models: A Mechanistic Approach

Ziqi Wang, Hanlin Zhang, Xiner Li, Kuan-Hao Huang, Chi Han, Shuiwang Ji, Sham M. Kakade, Hao Peng, Heng Ji

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL、cs.LG

Comments 26 pages, 6 figures, 15 tables

The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models

Matthew Renze, Erhan Guven

专题命中 数学推理 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Journal ref 2nd International Conference on Foundation and Large Language Models (FLLM 2024), pp.476-483

Number Cookbook: Number Understanding of Language Models and How to Improve It

Haotong Yang, Yi Hu, Shijia Kang, Zhouchen Lin, Muhan Zhang

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2025 poster

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

Xingwu Sun, Yanfeng Chen, Yiqing Huang, Ruobing Xie, Jiaqi Zhu, Kai Zhang, Shuaipeng Li, Zhen Yang, Jonny Han, Xiaobo Shu, Jiahao Bu, Zhongzhi Chen, Xuemeng Huang, Fengzong Lian, Saiyong Yang, Jianfeng Yan, Yuyuan Zeng, Xiaoqin Ren, Chao Yu, Lulu Wu, Yue Mao, Jun Xia, Tao Yang, Suncong Zheng, Kan Wu, Dian Jiao, Jinbao Xue, Xipeng Zhang, Decheng Wu, Kai Liu, Dengpeng Wu, Guanghui Xu, Shaohua Chen, Shuang Chen, Xiao Feng, Yigeng Hong, Junqiang Zheng, Chengcheng Xu, Zongwei Li, Xiong Kuang, Jianglu Hu, Yiqi Chen, Yuchi Deng, Guiyang Li, Ao Liu, Chenchen Zhang, Shihui Hu, Zilong Zhao, Zifan Wu, Yao Ding, Weichao Wang, Han Liu, Roberts Wang, Hao Fei, Peijie Yu, Ze Zhao, Xun Cao, Hai Wang, Fusheng Xiang, Mengyuan Huang, Zhiyuan Xiong, Bin Hu, Xuebin Hou, Lei Jiang, Jianqiang Ma, Jiajia Wu, Yaping Deng, Yi Shen, Qian Wang, Weijie Liu, Jie Liu, Meng Chen, Liang Dong, Weiwen Jia, Hu Chen, Feifei Liu, Rui Yuan, Huilin Xu, Zhenxiang Yan, Tengfei Cao, Zhichao Hu, Xinhua Feng, Dong Du, Tinghao Yu, Yangyu Tao, Feng Zhang, Jianchen Zhu, Chengzhong Xu, Xirui Li, Chong Zha, Wen Ouyang, Yinben Xia, Xiang Li, Zekun He, Rongpeng Chen, Jiawei Song, Ruibin Chen, Fan Jiang, Chongqing Zhao, Bo Wang, Hao Gong, Rong Gan, Winston Hu, Zhanhui Kang, Yong Yang, Yuhong Liu, Di Wang, Jie Jiang

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 4 Figures

Library Learning Doesn't: The Curious Case of the Single-Use "Library"

Ian Berlot-Attwell, Frank Rudzicz, Xujie Si

专题命中 数学推理 ：reasoning(abstract)；self-correction(abstract)；分类 cs.CL、cs.LG

Comments 24 pages, 7 figures. Accepted to the 4th MATH-AI Workshop at NeurIPS'24

Can LLMs plan paths with extra hints from solvers?

Erik Wu, Sayan Mitra

专题命中 数学推理 ：reasoning(abstract)；planning(abstract)；分类 cs.CL、cs.AI

CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions

Jun Rao, Xuebo Liu, Lian Lian, Shengjun Cheng, Yunjie Liao, Min Zhang

专题命中 数学推理 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024

PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning

Qibin Wang, Xiaolin Hu, Weikai Xu, Wei Liu, Jian Luan, Bin Wang

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL、cs.LG

MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems

Bin Lei, Yi Zhang, Shan Zuo, Ali Payani, Caiwen Ding

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

CharED: Character-wise Ensemble Decoding for Large Language Models

Kevin Gu, Eva Tuecke, Dmitriy Katz, Raya Horesh, David Alvarez-Melis, Mikhail Yurochkin

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.LG

Comments 9 pages, 4 figures

The neural correlates of logical-mathematical symbol systems processing resemble that of spatial cognition more than natural language processing

Yuannan Li, Shan Xu, Jia Liu

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks

Mahdi Sabbaghi, George Pappas, Hamed Hassani, Surbhi Goel

专题命中 数学推理 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.LG

Comments 32 pages, 16 figures

Investigating Symbolic Capabilities of Large Language Models

Neisarg Dave, Daniel Kifer, C. Lee Giles, Ankur Mali

专题命中 数学推理 ：reasoning(abstract)；planning(abstract)；分类 cs.CL、cs.LG

Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving

Aniket Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy Lillicrap, Danilo Rezende, Yoshua Bengio, Michael Mozer, Sanjeev Arora

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.AI、cs.LG

Comments Preprint. Under review

VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context

Yunxin Li, Baotian Hu, Haoyuan Shi, Wei Wang, Longyue Wang, Min Zhang

专题命中 数学推理 ：reasoning(abstract)；planning(abstract)；分类 cs.CL、cs.AI

Comments 17 pages; Accepted by ICML 2024

Learning From Mistakes Makes LLM Better Reasoner

Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, Weizhu Chen

专题命中 数学推理 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Comments 23 pages, 13 figures, 6 tables

Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM

Sainbayar Sukhbaatar, Olga Golovneva, Vasu Sharma, Hu Xu, Xi Victoria Lin, Baptiste Rozière, Jacob Kahn, Daniel Li, Wen-tau Yih, Jason Weston, Xian Li

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL、cs.AI

Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs

Aaditya K. Singh, DJ Strouse

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.LG

Comments 21 pages, 18 figures

TinyGSM: achieving >80% on GSM8k with small language models

Bingbin Liu, Sebastien Bubeck, Ronen Eldan, Janardhan Kulkarni, Yuanzhi Li, Anh Nguyen, Rachel Ward, Yi Zhang

专题命中 数学推理 ：reasoning(abstract)；verifier(abstract)；分类 cs.CL、cs.LG

Prompt Sketching for Large Language Models

Luca Beurer-Kellner, Mark Niklas Müller, Marc Fischer, Martin Vechev

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Beyond Words: A Mathematical Framework for Interpreting Large Language Models

Javier González, Aditya V. Nori

专题命中 数学推理 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.AI、cs.LG

Comments 4 figures, 18 pages

MoT: Memory-of-Thought Enables ChatGPT to Self-Improve

Xiaonan Li, Xipeng Qiu

专题命中 数学推理 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Comments Accepted to appear at EMNLP 2023