大模型推理能力

MegaMath: Pushing the Limits of Open Math Corpora

Fan Zhou, Zengzhi Wang, Nikhil Ranjan, Zhoujun Cheng, Liping Tang, Guowei He, Zhengzhong Liu, Eric P. Xing

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 26 pages, 15 figures, 22 tables

Teaching LLMs for Step-Level Automatic Math Correction via Reinforcement Learning

Junsong Li, Jie Zhou, Yutao Yang, Bihao Zhan, Qianjun Pan, Yuyang Ding, Qin Chen, Jiang Bo, Xin Lin, Liang He

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

AgentRxiv: Towards Collaborative Autonomous Research

Samuel Schmidgall, Michael Moor

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?

Zhengbo Wang, Jian Liang, Ran He, Zilei Wang, Tieniu Tan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Camera-Ready Version for ICLR 2025; technical corrections to previous version

xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference

Maximilian Beck, Korbinian Pöppel, Phillip Lippe, Richard Kurle, Patrick M. Blies, Günter Klambauer, Sebastian Böck, Sepp Hochreiter

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Code available at: https://github.com/NX-AI/xlstm and https://github.com/NX-AI/xlstm-jax

Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs

Microsoft, :, Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi-ling Chen, Qi Dai, Xiyang Dai, Ruchao Fan, Mei Gao, Min Gao, Amit Garg, Abhishek Goswami, Junheng Hao, Amr Hendy, Yuxuan Hu, Xin Jin, Mahmoud Khademi, Dongwoo Kim, Young Jin Kim, Gina Lee, Jinyu Li, Yunsheng Li, Chen Liang, Xihui Lin, Zeqi Lin, Mengchen Liu, Yang Liu, Gilsinia Lopez, Chong Luo, Piyush Madan, Vadim Mazalov, Arindam Mitra, Ali Mousavi, Anh Nguyen, Jing Pan, Daniel Perez-Becker, Jacob Platin, Thomas Portet, Kai Qiu, Bo Ren, Liliang Ren, Sambuddha Roy, Ning Shang, Yelong Shen, Saksham Singhal, Subhojit Som, Xia Song, Tetyana Sych, Praneetha Vaddamanu, Shuohang Wang, Yiming Wang, Zhenghao Wang, Haibin Wu, Haoran Xu, Weijian Xu, Yifan Yang, Ziyi Yang, Donghan Yu, Ishmam Zabir, Jianwen Zhang, Li Lyna Zhang, Yunan Zhang, Xiren Zhou

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 39 pages

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Y. X. Wei, Lean Wang, Zhiping Xiao, Yuqing Wang, Chong Ruan, Ming Zhang, Wenfeng Liang, Wangding Zeng

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?

Zhenheng Tang, Xiang Liu, Qian Wang, Peijie Dong, Bingsheng He, Xiaowen Chu, Bo Li

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs

Hao Sun, Yunyi Shen, Jean-Francois Ton, Mihaela van der Schaar

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Language Models Use Trigonometry to Do Addition

Subhash Kantamneni, Max Tegmark

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

LLMs can see and hear without any training

Kumar Ashutosh, Yossi Gandelsman, Xinlei Chen, Ishan Misra, Rohit Girdhar

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Code: https://github.com/facebookresearch/MILS

Evolutionary Optimization of Model Merging Recipes

Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, David Ha

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)

Comments Authors' submitted version before final edits. Published in Nature Machine Intelligence on January 27, 2025: https://www.nature.com/articles/s42256-024-00975-8

Journal ref Nat Mach Intell (2025)

Stream Aligner: Efficient Sentence-Level Alignment via Distribution Induction

Hantao Lou, Jiaming Ji, Kaile Wang, Yaodong Yang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments AAAI Alignment Track 2025 Poster

GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection

Xutao Liao, Shaohui Li, Yuhui Xu, Zhi Li, Yu Liu, You He

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Channel Merging: Preserving Specialization for Merged Experts

Mingyang Zhang, Jing Liu, Ganggui Ding, Xinyi Yu, Linlin Ou, Bohan Zhuang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted by AAAI 2025

Augmenting Math Word Problems via Iterative Question Composing

Haoxiong Liu, Yifan Zhang, Yifan Luo, Andrew Chi-Chih Yao

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision

Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, Chuang Gan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024

Evaluating Large Vision-and-Language Models on Children's Mathematical Olympiads

Anoop Cherian, Kuan-Chuan Peng, Suhas Lohit, Joanna Matthiesen, Kevin Smith, Joshua B. Tenenbaum

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024 (Datasets and Benchmarks Track)

A Careful Examination of Large Language Model Performance on Grade School Arithmetic

Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Charlotte Zhuang, Dylan Slack, Qin Lyu, Sean Hendryx, Russell Kaplan, Michele Lunati, Summer Yue

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 2024 NeurIPS Camera Ready (Datasets and Benchmarks Track)

Solving for X and Beyond: Can Large Language Models Solve Complex Math Problems with More-Than-Two Unknowns?

Kuei-Chun Kao, Ruochen Wang, Cho-Jui Hsieh

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Journal ref https://aclanthology.org/2024.findings-emnlp.980/

3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability

Baohao Liao, Christof Monz

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to NeurIPS 2024. Code: https://github.com/BaohaoLiao/road

Implicit Personalization in Language Models: A Systematic Study

Zhijing Jin, Nils Heil, Jiarui Liu, Shehzaad Dhuliawala, Yahang Qi, Bernhard Schölkopf, Rada Mihalcea, Mrinmaya Sachan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2024 Findings

MathPile: A Billion-Token-Scale Pretraining Corpus for Math

Zengzhi Wang, Xuefeng Li, Rui Xia, Pengfei Liu

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 43 pages. Accepted by NeurIPS 2024. https://github.com/GAIR-NLP/MathPile/

ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models

Jing Liu, Ruihao Gong, Mingyang Zhang, Yefei He, Jianfei Cai, Bohan Zhuang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Tech report

NVLM: Open Frontier-Class Multimodal LLMs

Wenliang Dai, Nayeon Lee, Boxin Wang, Zhuolin Yang, Zihan Liu, Jon Barker, Tuomas Rintamaki, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Fixed the typos. For more information, please visit our project page at: https://research.nvidia.com/labs/adlr/NVLM-1

SBoRA: Low-Rank Adaptation with Regional Weight Updates

Lai-Man Po, Yuyang Liu, Haoxuan Wu, Tianqi Zhang, Wing-Yin Yu, Zhuohan Wang, Zeyu Jiang, Kun Li

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 16 pages, 4 figures

Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together

Dilara Soylu, Christopher Potts, Omar Khattab

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2024

Revisiting the Superficial Alignment Hypothesis

Mohit Raghavendra, Vaskar Nath, Sean Hendryx

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs

Ruoyu Wang, Xiaoxuan Li, Lina Yao

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

The Perfect Blend: Redefining RLHF with Mixture of Judges

Tengyu Xu, Eryk Helenowski, Karthik Abinav Sankararaman, Di Jin, Kaiyan Peng, Eric Han, Shaoliang Nie, Chen Zhu, Hejia Zhang, Wenxuan Zhou, Zhouhao Zeng, Yun He, Karishma Mandyam, Arya Talabzadeh, Madian Khabsa, Gabriel Cohen, Yuandong Tian, Hao Ma, Sinong Wang, Han Fang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments submitted to conference