大模型推理能力

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartłomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, César Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodola, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocoń, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Şenel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramírez Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michał Swędrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Miłkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima, Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, Ziyi Wu

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages, 17 figures + references and appendices, repo: https://github.com/google/BIG-bench

Journal ref Transactions on Machine Learning Research, May/2022, https://openreview.net/forum?id=uyTL5Bvosj

Turning large language models into cognitive models

Marcel Binz, Eric Schulz

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Self-Refine: Iterative Refinement with Self-Feedback

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, Peter Clark

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Code, data, and demo at https://selfrefine.info/

Learning Non-linguistic Skills without Sacrificing Linguistic Proficiency

Mandar Sharma, Nikhil Muralidhar, Naren Ramakrishnan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL 2023's main conference

A Categorical Archive of ChatGPT Failures

Ali Borji

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Learning to Reason With Relational Abstractions

Andrew J. Nam, Mengye Ren, Chelsea Finn, James L. McClelland

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Solving math word problems with process- and outcome-based feedback

Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, Irina Higgins

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Overcoming Barriers to Skill Injection in Language Modeling: Case Study in Arithmetic

Mandar Sharma, Nikhil Muralidhar, Naren Ramakrishnan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2022: Math-AI Workshop

Measuring Mathematical Problem Solving With the MATH Dataset

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, Jacob Steinhardt

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2021. Code and the MATH dataset is available at https://github.com/hendrycks/math/

Investigating the Limitations of Transformers with Simple Arithmetic Tasks

Rodrigo Nogueira, Zhiying Jiang, Jimmy Lin

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

h1: Bootstrapping LLMs to Reason over Longer Horizons via Reinforcement Learning

Sumeet Ramesh Motwani, Alesia Ivanova, Ziyang Cai, Philip Torr, Riashat Islam, Shital Shah, Christian Schroeder de Witt, Charles London

机构 * University of Oxford（牛津大学） ； Princeton University（普林斯顿大学） ； Microsoft AI Frontiers（微软人工智能前沿） 纠错

专题命中 数学推理 ：reasoning(abstract,comments)；分类 cs.AI、cs.LG

Comments Preprint, 31 pages, 8 figures, long-horizon reasoning

MizAR 40 for Mizar 40

Cezary Kaliszyk, Josef Urban

专题命中 数学推理 ：reasoning(abstract,journal_ref)；分类 cs.AI、cs.LG

Journal ref J. Automated Reasoning 55(3): 245-256 (2015)

Cutting Through the Noise: Boosting LLM Performance on Math Word Problems

Ujjwala Anantheswaran, Himanshu Gupta, Kevin Scaria, Shreyas Verma, Chitta Baral, Swaroop Mishra

机构 * Arizona State University（亚利桑那州立大学） ； Georgia Institute of Technology（佐治亚理工学院） 纠错

专题命中 数学推理 ：reasoning(abstract,comments)；分类 cs.CL；planning(comments)

Comments Published at ICLR 2025 Workshop on Reasoning and Planning for LLMs

TailSFT: Filtered Fine-Tuning Improves Post-Training Performance

TailSFT：过滤式微调提升后训练性能

Sadhika Malladi, Samy Jelassi, Dylan Foster, Jordan T. Ash, Akshay Krishnamurthy

机构 * University of California San Diego（加州大学圣迭戈分校） ； Microsoft Research NYC（微软研究院纽约分部） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 本文提出TailSFT过滤式微调算法，可提升模型后强化学习性能，在OLMo-3 7B上数学与代码评估pass@16最高提17%，后续GRPO的pass@1最高提4%，还引入轻量诊断方法识别适用场景。

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models

StoSignSGD：无偏结构随机性修正SignSGD用于训练大语言模型

Dingzhi Yu, Rui Pan, Yuxing Liu, Difan Zou, Tong Zhang

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 本文提出StoSignSGD算法，通过在sign操作中注入结构随机性，解决SignSGD在非光滑目标下的发散问题，理论和实验均证明其在凸和非凸优化中的优越性。

Ban&Pick: Enhancing Performance and Efficiency of MoE-LLMs via Smarter Routing

Ban&Pick：通过更智能的路由提升MoE-LLM的性能与效率

Yuanteng Chen, Peisong Wang, Yuantian Shao, Nanxin Zeng, Chang Xu, Jian Cheng

机构 * Nanjing University of Science and Technology（南京理工大学） ； C 2 \text{C}^{2} DL, Institute of Automation, Chinese Academy of Sciences（C2 DL，自动化研究所，中国科学院） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； School of Computer Science, University of Sydney（悉尼大学计算机学院） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 本研究针对MoE-LLM预训练路由的问题，提出即插即用的Ban&Pick策略，通过Pick强化关键专家、Ban修剪冗余专家，在不重新训练或改架构的情况下提升MoE-LLM性能并加速推理。

Comments 26 pages, 13 figures

Journal ref The 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026)

Contrastive Branch Policy Optimization

对比分支策略优化

Ying Wang, Changlin Qiu, Bang Lin, Linbo Jin, Wen Jiang, Zhe Sun, Jingli Yang

机构 * Alibaba Group（阿里巴巴集团） ； Harbin Institute of Technology（哈尔滨工业大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 针对工具集成智能体训练的细粒度信用分配问题，提出CBPO解耦分支采样的预算分配与信用转化，在十个基准上优于现有方法，获两个领域及两种模型规模下最高宏平均准确率。

Comments 10 pages, 5 figures, 3 tables

LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding

LOCKS：用于高效长上下文解码的页面局部紧凑键摘要

Junsung Hwang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 研究针对长上下文解码中KV缓存瓶颈问题，提出LOCKS方法，为页面提供局部紧凑键摘要，通过重建对数its、估计注意力质量等操作，仅关注顶部页面，在多种任务中表现出色，显著降低解码延迟。

Measuring Stability and Failure Behavior in Language Models Under Structured Perturbations

结构化扰动下语言模型的稳定性与失效行为测量

Samira Golsefid

机构 * NeuroLoft（神经 loft（或：神经实验室）） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI

AI总结 本研究提出分级多类别失效感知框架，在四类模型上测试七种结构化扰动，发现模型失效层级因类别而异，冲突指令与不可能前提问题是所有模型共有的弱点，此类失效无法被标准准确率检测。

Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Models

先给出答案，后进行推理：扩散大语言模型中的承诺顺序

Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Hwiyeong Lee, Taesup Kim

机构 * Graduate School of Data Science, Seoul National University（首尔国立大学数据科学研究生院） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI

AI总结 本研究发现扩散大语言模型（dLLMs）的任意顺序提交机制会导致推理失败，通过前沿门控承诺干预可恢复推理性能，同时保留并行解码优势。

How Should LLMs Consume High-Quality Data? Optimal Data Scheduling via Quality-Aware Functional Scaling Laws

LLM应如何消费高质量数据？通过质量感知的功能缩放定律实现最优数据调度

Zhitao Zhu, Xili Wang, Shizhe Wu, Jiawei Fu, Xiaoqing Liu

机构 * Peking University（北京大学） ； Meituan（美团） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 本文通过引入数据质量维度扩展功能缩放定律，解析求解了联合数据质量和批次大小调度问题，揭示了高质量数据的双重角色，并提出了Drop-Stable-Rampup调度策略，在15B MoE模型上相比WSD和余弦衰减分别提升平均准确率+1.70和+2.98。

Approximate Speculative Decoding

近似投机解码

Yuannuo Feng, Zegang Peng, Yuxin Xie, Yubing Ye, Yizhe Chen, Wenshuai Yao, Wenyong Zhou, Wang Kang

专题命中 数学推理 ：verifier(abstract)；分类 cs.AI、cs.LG

AI总结 本文提出无需训练的Approximate Speculative Decoding（ASD），通过带预算的最长前缀选择优化投机解码，提升了生成吞吐量与验证器接受率，且无需新草稿模型或微调。

SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

SLAI T-Rex：在升腾超级集群上对DeepSeek-V4系列进行全参数训练后优化

Dongfang Li, Xiaodong Luo, Ruoyu Sun, Xuhui Chen, Linyuan Qiu, Jian Meng, Zhengxuan Lu, Yiting Wang, Yucheng Xie, Tao Guo, Tianxiang Fang, Jing Li, Sihang Chen, Shihao Hong, Chang Liu, Weihua Dai, Zirong Zeng, Ziwei Zhu, Zhuohan Wang, Zhengjun Yue, Igor Vasilyev, Min Liu, Weijian Sun, Xin Chen, Yingmeng Gao, Jinhua Zhou, Taolue Chen, Chenwei Wu, Dong Zhang, Wenlong Jin, Jinmin Xiang, Barkova Maria, Ushakov Anton, Xianfei Jin, Tian Ding, Zhihang Lin, Qian Chen, Linxin Yang, Mingzhe Yang, Bingwei Zhang, Hongzhang Yang, Fangxue Zhang, Shijun Qin, Jie Yu, Cuihua Hu, Tolstykh Vasiliy, Nosov Ivan, Abdullin Amir, Zhicheng Zhou, Xin Zhang, Zhixiong Ning, Xutong Zhao, Junjie Huang, Jiajun Liu, Weiyan Kong, Zheng Zhang, Wenhan Luo, Lin Hu, Yangbo Guo, Li Zeng, Shihao Zhang, Baotian Hu, Min Zhang, Haizhou Li, Zhiquan Luo

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI

AI总结 研究针对万亿参数规模MoE模型全参数训练后优化的系统挑战，在升腾NPU超级集群上以DeepSeek-V4为目标工作负载开发分层优化框架，建立CPT和SFT工作流程，提升模型性能，推动复杂推理前沿模型系统发展。

Comments 73 pages, 22 figures, 20 tables

SOD: Step-wise On-policy Distillation for Small Language Model Agents

SOD：分步式在线蒸馏用于小型语言模型代理

Qiyong Zhong, Mao Zheng, Mingyang Song, Xin Lin, Jie Sun, Houcheng Jiang, Xiang Wang, Junfeng Fang

机构 * Zhejiang University（浙江大学） ； Large Language Model Department, Tencent（腾讯大语言模型部门） ； University of Science and Technology of China（中国科学技术大学） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI

AI总结 针对小型语言模型中工具集成推理的稳定性问题，提出SOD分步式在线蒸馏框架，通过动态调整蒸馏强度缓解教师信号误导，提升推理性能。

Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization

学习剩余内容，而非已掌握内容：面向多奖励策略优化的饱和感知优势重加权方法

Yixuan Wang, Yifei Chen, Haichao Zhang, Haozheng Luo, Xander Wu, Jie Ni, Yun Fu, Nuno Vasconcelos, Yijiang Li

机构 * University of Florida（佛罗里达大学） ； UC San Diego（加州大学圣迭戈分校） ； Northeastern University（东北大学） ； Northwestern University（西北大学） ； Stanford University（斯坦福大学） ； Universität Innsbruck（因斯布鲁克大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 针对多奖励策略优化中现有方法的缺陷，提出SA-MRPO方法，动态分配优化资源，在数学推理、自适应推理、编码任务中均实现性能提升。

Comments 14 pages, 2 figures

SAPE: Sandwich Adapters for Parameter Efficiency in Large Language Model Fine-Tuning

SAPE：用于大语言模型微调参数效率的三明治适配器

Mohammad Aref Jafari-Raddani, Morteza Mohajjel Kafshdooz

机构 * Qom University of Technology（库姆理工大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 SAPE是一种基于三明治式硬权重共享拓扑的PEFT框架，通过隔离边界变换降低内存与计算开销，在低参数约束下，于RoBERTa-large、LLaMA-3.2等模型的多项任务中取得优于现有方法的性能。

Comments 16 pages, 4 figures, 10 tables, includes appendix

Measuring Semantic Abstractness of SAE Features via Nonlocality

通过非局部性测量SAE特征的语义抽象性

Chuqiao Lin, Shivaji Sondhi, Xiao-Liang Qi

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 该研究提出特征非局部性（FNL）指标，可区分SAE特征的抽象层级，用于审计越狱缓解特征及引导DeepSeek-R1模型特征提升数学推理准确率。

Comments 18 pages, 9 figures

Fusion Training for Mathematical Generalization in Large Language Models

大型语言模型中数学泛化的融合训练

Congfeng Cao, Pengyu Zhang, Jelke Bloem

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI

AI总结 本研究针对思维模式融合（TMF）训练动态开展系统性研究，揭示两种模式间的非对称相互作用与负相关关系，为设计TMF训练设置提供指导。

Comments ACL SRW 2026

Mismatch Matters: On-Policy Distillation Beyond Token Agreement

失配很重要：超越token一致性的在线蒸馏

Zichao Yu, Chengzhi Yu, Shengze Xu, Yujin Han, Bingqing Jiang, Xu Wang, Difan Zou

机构 * The University of Hong Kong（香港大学） ； University of Science and Technology of China（中国科学技术大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI

AI总结 本文针对在线蒸馏存在的退化一致性失效问题，提出TIDE方法修正师生失配，在数学推理基准上显著提升性能、缩短响应长度并减少格式错误。

SR-OPSD: Self-Referenced On-Policy Self-Distillation

SR-OPSD：自引用同策略自蒸馏

Zhuo Sun, Entong Li, Yanlong Zhao, Xiaoyuan Cheng, Wenxuan Yuan, Kaiyu Li, Che Liu, Huihang Liu, Harrison Bo Hua Zhu, Li Zeng

机构 * Shanghai University of Finance and Economics（上海财经大学） ； Imperial College London（伦敦帝国学院） ； University of Science and Technology of China（中国科学技术大学） ； University College London（伦敦大学学院） ； Nanyang Technological University（南洋理工大学） ； Technical University of Denmark（丹麦技术大学） ； University of Copenhagen（哥本哈根大学） ； Peking University（北京大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.AI、cs.LG

AI总结 该研究针对同策略自蒸馏的优化不稳定问题，提出SR-OPSD方法，通过几何插值结合Rényi散度优化蒸馏目标，在多任务大语言模型实验中取得最优或竞争性能。