An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Shanghai Innovation Center for Processor Technologies（上海处理器技术创新中心）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学与技术学院）
Zui Chen, Yezeng Chen, Jiaqi Han, Zhijie Huang, Ji Qi, Yi Zhou
大语言模型（LLM）在数学推理任务上正展现出涌现能力，通过监督微调（SFT）增强开源LLM能力也受到越来越多关注。本文旨在探索一种面向监督数据的通用数据策略，以帮助优化和扩展数学推理能力。首先，我们通过识别推理路径的最小最优集合，确定推理路径增强的能力边界。其次，我们验证了模型的不同能力可通过混合相应类型数据的最小最优集合（Mix of Minimal Optimal Sets）得到累积增强；同时，我们的模型MMOS在低得多的构建成本下，于一系列基座模型上取得SOTA性能。此外，我们指出GSM-HARD并非真正困难，如今的LLM也不再缺乏数值鲁棒性。我们还提供了用于鲁棒性测试和教育应用的Auto Problem Generator。代码和数据已公开发布于https://github.com/cyzhh/MMOS。
英文摘要：
Large language models (LLMs) are displaying emergent abilities for math reasoning tasks,and there is a growing attention on enhancing the ability of open-source LLMs through supervised fine-tuning (SFT).In this paper, we aim to explore a general data strategy for supervised data to help optimize and expand math reasoning ability.Firstly, we determine the ability boundary of reasoning paths augmentation by identifying these paths' minimal optimal set.Secondly, we validate that different abilities of the model can be cumulatively enhanced by Mix of Minimal Optimal Sets of corresponding types of data, while our models MMOS achieve SOTA performance on series base models under much lower construction costs.Besides, we point out GSM-HARD is not really hard and today's LLMs no longer lack numerical robustness.Also, we provide an Auto Problem Generator for robustness testing and educational applications.Our code and data are publicly available at https://github.com/cyzhh/MMOS.
How Likely Do LLMs with CoT Mimic Human Reasoning?
Zhejiang University（浙江大学）；School of Engineering, Westlake University（西湖大学工学院）；Institute of Advanced Technology, Westlake Institute for Advanced Study（西湖高等研究院先进技术研究院）
Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
Intelligent Game and Decision Lab（智能游戏与决策实验室）；National University of Defense Technology（国防科技大学）；University of Melbourne（墨尔本大学）；University of Sydney（悉尼大学）；Hong Kong Baptist University（香港 Baptist 大学）
How Do LLMs Understand Legal Philosophy? Diagnosing Legal-Philosophical Reasoning in LLMs
The University of Osaka（大阪大学）；UC Davis（加州大学戴维斯分校）；UC Berkeley（加州大学伯克利分校）；Peking University（北京大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Shanghai Jiao Tong University（上海交通大学）；NII（信息学研究所（日本））；City University of Hong Kong（香港城市大学）
Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving
The Hong Kong University of Science and Technology（香港科学与技术大学）；Huawei Noah’s Ark Lab（华为诺亚实验室）；Zhejiang University（浙江大学）；Peking University（北京大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Surrey（萨里大学）
InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles
Nankai University（南开大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai AI Laboratory（上海人工智能实验室）；Fudan University（复旦大学）；Johns Hopkins University（约翰霍普金斯大学）；Wuhan University（武汉大学）；University of Science and Technology of China（中国科学技术大学）
Acting Flatterers via LLMs Sycophancy: Combating Clickbait with LLMs Opposing-Stance Reasoning
Yangzhou University（扬州大学）；Auburn University（亚伯拉罕大学）；University of South Dakota（南达科他大学）；Institute for Clarity in Documentation（文档清晰研究所）；Inria Paris-Rocquencourt（巴黎-罗quentourt研究所）；Rajiv Gandhi University（拉贾夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）
Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation
Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University（上海数据科学关键实验室，计算机科学学院，复旦大学）；School of Data Science, Fudan University（数据科学学院，复旦大学）；Ant Group（蚂蚁集团）
Reasoning or Not? A Comprehensive Evaluation of Reasoning LLMs for Dialogue Summarization
Faculty of Applied Sciences, Macao Polytechnic University（澳门理工大学应用科学学院）；Computer Science and Communication Research Center, School of Technology and Management, Polytechnic of Leiria（莱里亚理工学院技术与管理学院计算机科学与通信研究中心）；Institute of International Language Services Studies, Macau Millennium College（澳门 millennium 学院国际语言服务研究学院）；CISUC, Department of Informatics Engineering, University of Coimbra（科英布拉大学信息工程系 CISUC）
Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
Tsinghua University（清华大学）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；The University of Tokyo（东京大学）；Peking University（北京大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；HKUST（香港科技大学）
