Numerical reasoning is often important to accurately understand the world. Recently, several format-specific datasets have been proposed, such as numerical reasoning in the settings of Natural Language Inference (NLI), Reading Comprehension (RC), and Question Answering (QA). Several format-specific models and architectures in response to those datasets have also been proposed. However, there exists a strong need for a benchmark which can evaluate the abilities of models, in performing question format independent numerical reasoning, as (i) the numerical reasoning capabilities we want to teach are not controlled by question formats, (ii) for numerical reasoning technology to have the best possible application, it must be able to process language and reason in a way that is not exclusive to a single format, task, dataset or domain. In pursuit of this goal, we introduce NUMBERGAME, a multifaceted benchmark to evaluate model performance across numerical reasoning tasks of eight diverse formats. We add four existing question types in our compilation. Two of the new types we add are about questions that require external numerical knowledge, commonsense knowledge and domain knowledge. For building a more practical numerical reasoning system, NUMBERGAME demands four capabilities beyond numerical reasoning: (i) detecting question format directly from data (ii) finding intermediate common format to which every format can be converted (iii) incorporating commonsense knowledge (iv) handling data imbalance across formats. We build several baselines, including a new model based on knowledge hunting using a cheatsheet. However, all baselines perform poorly in contrast to the human baselines, indicating the hardness of our benchmark. Our work takes forward the recent progress in generic system development, demonstrating the scope of these under-explored tasks.
Atomic Skills are the Prerequisite: When Reinforcement Learning Synthesizes Compositional Reasoning, and When It Only Amplifies
University of Waterloo（滑铁卢大学）；Duke University（杜克大学）；National University of Singapore（新加坡国立大学）；Princeton University（普林斯顿大学）；Peking University（北京大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
JudgeRLVR: Judge First, Generate Second for Efficient Reasoning
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）；CFCS, School of Computer Science, Peking University（计算机学院，北京大学）；LLM-Core Xiaomi（小智LLM核心）
Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
University of Science and Technology of China（中国科学技术大学）；State Key Laboratory of Cognitive Intelligence（认知智能国家重点实验室）；iFLYTEK AI Research（iFLYTEK人工智能研究院）；Hong Kong University of Science and Technology（香港科学与技术大学）
From Scientific Crop Imagery to Agronomic Reasoning: A Multimodal LLM Benchmark for Plant Phenotyping
University of Memphis（孟菲斯大学）；University of Missouri（密苏里大学）；University of Arkansas Division of Agriculture（阿肯色大学农业分部）；Mississippi State University（密西西比州立大学）
SATQuest: A Verifier for Logical Reasoning Evaluation and Reinforcement Fine-Tuning of LLMs
Chengdu Institute of Computer Applications, Chinese Academy of Sciences（中国科学院成都计算机应用研究所）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）
Reasoning With a Star: A Heliophysics Dataset and Benchmark for Agentic Scientific Reasoning
Frontier Development Lab（前沿发展实验室）；Department of Mechanical and Aerospace Engineering, UCLA（机械与航空航天工程系，加州大学洛杉矶分校）；Trillium Technologies Inc.（Trillium技术公司）；Department of Engineering, University of Cambridge（工程系，剑桥大学）
RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format
Institute of Image Processing and Pattern Recoginition, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（图像处理与模式识别研究所，自动化与智能感知学院，上海交通大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；MoE Key Laboratory of System Control and Information Processing (Shanghai)（系统控制与信息处理国家重点实验室（上海））
