大模型推理能力

TreeCut: A Synthetic Unanswerable Math Word Problem Dataset for LLM Hallucination Evaluation

Jialin Ouyang

机构 * Columbia University（哥伦比亚大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL 2025 Main Conference

InductionBench: LLMs Fail in the Simplest Complexity Class

Wenyue Hua, Tyler Wong, Sun Fei, Liangming Pan, Adam Jardine, William Yang Wang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 25 pages, 10 figures, more details including examples and prompts are added

DeepCritic: Deliberate Critique with Large Language Models

Wenkai Yang, Jingwen Chen, Yankai Lin, Ji-Rong Wen

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学首都人工智能学院） ； School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Work in progress. Data and models are available at https://github.com/RUCBM/DeepCritic

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models

Michael Noukhovitch, Shengyi Huang, Sophie Xhonneux, Arian Hosseini, Rishabh Agarwal, Aaron Courville

机构 * Mila Quebec AI Institute（魁北克AI研究所） ； Université de Montréal（蒙特利尔大学） ； Allen Institute for AI（人工智能研究院） ； Google Deepmind（谷歌DeepMind） ； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted at ICLR 2025, code at https://github.com/mnoukhov/async_rlhf, integrated into the open-instruct library https://github.com/allenai/open-instruct

A Dual-Space Framework for General Knowledge Distillation of Large Language Models

Xue Zhang, Songming Zhang, Yunlong Liang, Fandong Meng, Yufeng Chen, Jinan Xu, Jie Zhou

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 19 pages, 9 figures, 11 tables, under review. Code is available at: https://github.com/songmzhang/DSKDv2. arXiv admin note: text overlap with arXiv:2406.17328

MegaMath: Pushing the Limits of Open Math Corpora

Fan Zhou, Zengzhi Wang, Nikhil Ranjan, Zhoujun Cheng, Liping Tang, Guowei He, Zhengzhong Liu, Eric P. Xing

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 26 pages, 15 figures, 22 tables

Teaching LLMs for Step-Level Automatic Math Correction via Reinforcement Learning

Junsong Li, Jie Zhou, Yutao Yang, Bihao Zhan, Qianjun Pan, Yuyang Ding, Qin Chen, Jiang Bo, Xin Lin, Liang He

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

AgentRxiv: Towards Collaborative Autonomous Research

Samuel Schmidgall, Michael Moor

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?

Zhengbo Wang, Jian Liang, Ran He, Zilei Wang, Tieniu Tan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Camera-Ready Version for ICLR 2025; technical corrections to previous version

xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference

Maximilian Beck, Korbinian Pöppel, Phillip Lippe, Richard Kurle, Patrick M. Blies, Günter Klambauer, Sebastian Böck, Sepp Hochreiter

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Code available at: https://github.com/NX-AI/xlstm and https://github.com/NX-AI/xlstm-jax

Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs

Microsoft, :, Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi-ling Chen, Qi Dai, Xiyang Dai, Ruchao Fan, Mei Gao, Min Gao, Amit Garg, Abhishek Goswami, Junheng Hao, Amr Hendy, Yuxuan Hu, Xin Jin, Mahmoud Khademi, Dongwoo Kim, Young Jin Kim, Gina Lee, Jinyu Li, Yunsheng Li, Chen Liang, Xihui Lin, Zeqi Lin, Mengchen Liu, Yang Liu, Gilsinia Lopez, Chong Luo, Piyush Madan, Vadim Mazalov, Arindam Mitra, Ali Mousavi, Anh Nguyen, Jing Pan, Daniel Perez-Becker, Jacob Platin, Thomas Portet, Kai Qiu, Bo Ren, Liliang Ren, Sambuddha Roy, Ning Shang, Yelong Shen, Saksham Singhal, Subhojit Som, Xia Song, Tetyana Sych, Praneetha Vaddamanu, Shuohang Wang, Yiming Wang, Zhenghao Wang, Haibin Wu, Haoran Xu, Weijian Xu, Yifan Yang, Ziyi Yang, Donghan Yu, Ishmam Zabir, Jianwen Zhang, Li Lyna Zhang, Yunan Zhang, Xiren Zhou

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 39 pages

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Y. X. Wei, Lean Wang, Zhiping Xiao, Yuqing Wang, Chong Ruan, Ming Zhang, Wenfeng Liang, Wangding Zeng

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?

Zhenheng Tang, Xiang Liu, Qian Wang, Peijie Dong, Bingsheng He, Xiaowen Chu, Bo Li

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs

Hao Sun, Yunyi Shen, Jean-Francois Ton, Mihaela van der Schaar

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Language Models Use Trigonometry to Do Addition

Subhash Kantamneni, Max Tegmark

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

LLMs can see and hear without any training

Kumar Ashutosh, Yossi Gandelsman, Xinlei Chen, Ishan Misra, Rohit Girdhar

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Code: https://github.com/facebookresearch/MILS

Evolutionary Optimization of Model Merging Recipes

Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, David Ha

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)

Comments Authors' submitted version before final edits. Published in Nature Machine Intelligence on January 27, 2025: https://www.nature.com/articles/s42256-024-00975-8

Journal ref Nat Mach Intell (2025)

Stream Aligner: Efficient Sentence-Level Alignment via Distribution Induction

Hantao Lou, Jiaming Ji, Kaile Wang, Yaodong Yang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments AAAI Alignment Track 2025 Poster

GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection

Xutao Liao, Shaohui Li, Yuhui Xu, Zhi Li, Yu Liu, You He

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Channel Merging: Preserving Specialization for Merged Experts

Mingyang Zhang, Jing Liu, Ganggui Ding, Xinyi Yu, Linlin Ou, Bohan Zhuang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted by AAAI 2025

Augmenting Math Word Problems via Iterative Question Composing

Haoxiong Liu, Yifan Zhang, Yifan Luo, Andrew Chi-Chih Yao

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision

Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, Chuang Gan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024

Evaluating Large Vision-and-Language Models on Children's Mathematical Olympiads

Anoop Cherian, Kuan-Chuan Peng, Suhas Lohit, Joanna Matthiesen, Kevin Smith, Joshua B. Tenenbaum

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024 (Datasets and Benchmarks Track)

A Careful Examination of Large Language Model Performance on Grade School Arithmetic

Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Charlotte Zhuang, Dylan Slack, Qin Lyu, Sean Hendryx, Russell Kaplan, Michele Lunati, Summer Yue

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 2024 NeurIPS Camera Ready (Datasets and Benchmarks Track)

Solving for X and Beyond: Can Large Language Models Solve Complex Math Problems with More-Than-Two Unknowns?

Kuei-Chun Kao, Ruochen Wang, Cho-Jui Hsieh

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Journal ref https://aclanthology.org/2024.findings-emnlp.980/

3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability

Baohao Liao, Christof Monz

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to NeurIPS 2024. Code: https://github.com/BaohaoLiao/road

Implicit Personalization in Language Models: A Systematic Study

Zhijing Jin, Nils Heil, Jiarui Liu, Shehzaad Dhuliawala, Yahang Qi, Bernhard Schölkopf, Rada Mihalcea, Mrinmaya Sachan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2024 Findings

MathPile: A Billion-Token-Scale Pretraining Corpus for Math

Zengzhi Wang, Xuefeng Li, Rui Xia, Pengfei Liu

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 43 pages. Accepted by NeurIPS 2024. https://github.com/GAIR-NLP/MathPile/

ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models

Jing Liu, Ruihao Gong, Mingyang Zhang, Yefei He, Jianfei Cai, Bohan Zhuang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Tech report

NVLM: Open Frontier-Class Multimodal LLMs

Wenliang Dai, Nayeon Lee, Boxin Wang, Zhuolin Yang, Zihan Liu, Jon Barker, Tuomas Rintamaki, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Fixed the typos. For more information, please visit our project page at: https://research.nvidia.com/labs/adlr/NVLM-1