大模型推理能力

MORABLES: A Benchmark for Assessing Abstract Moral Reasoning in LLMs with Fables

Matteo Marcuzzo, Alessandro Zangari, Andrea Albarelli, Jose Camacho-Collados, Mohammad Taher Pilehvar

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2025 Main Conference

EthicsMH: A Pilot Benchmark for Ethical Reasoning in Mental Health AI

Sai Kartheek Reddy Kasu

机构 * IIIT Dharwad（德瓦德理工学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Automated MCQA Benchmarking at Scale: Evaluating Reasoning Traces as Retrieval Sources for Domain Adaptation of Small Language Models

Ozan Gokdemir, Neil Getty, Robert Underwood, Sandeep Madireddy, Franck Cappello, Arvind Ramanathan, Ian T. Foster, Rick L. Stevens

机构 * Argonne National Laboratory（阿贡国家实验室） ； The University of Chicago（芝加哥大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments This manuscript has been accepted for publication at the Supercomputing 25 (SC '25) Conference (Frontiers in Generative AI for HPC Science and Engineering: Foundations, Challenges, and Opportunities Workshop) in St. Louis, MO, USA on November 16th, 2025. It will appear in the SC25 Workshop Proceedings after that date

Optimizing Length Compression in Large Reasoning Models

Zhengxiang Cheng, Dongping Chen, Mingyang Fu, Tianyi Zhou

机构 * umd（马里兰大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 7 figures, 4 tables

MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapes

Nilay Pande, Sahiti Yerramilli, Jayant Sravan Tamarapalli, Rynaa Grover

机构 * Waymo ； Google（谷歌） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Reasoning Language Model for Personalized Lung Cancer Screening

Chuang Niu, Ge Wang

机构 * Department of Biomedical Engineering, School of Engineering, Center for Computational Innovations, Center for Biotechnology & Interdisciplinary Studies（生物医学工程系、工程学院、计算创新中心、生物技术与跨学科研究所以及中心） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Reverse-Engineered Reasoning for Open-Ended Generation

Haozhe Wang, Haoran Que, Qixin Xu, Minghao Liu, Wangchunshu Zhou, Jiazhan Feng, Wanjun Zhong, Wei Ye, Tong Yang, Wenhao Huang, Ge Zhang, Fangzhen Lin

机构 * Hong Kong University of Science and Technology（香港科技大学） ； Peking University（北京大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Preprint

CVPD at QIAS 2025 Shared Task: An Efficient Encoder-Based Approach for Islamic Inheritance Reasoning

Salah Eddine Bekhouche, Abdellah Zakaria Sellam, Hichem Telli, Cosimo Distante, Abdenour Hadid

机构 * University of the Basque Country UPV/EHU（巴斯克大学UPV/EHU） ； Institute of Applied Sciences and Intelligent Systems – CNR（应用科学与智能系统研究所 – CNR） ； Laboratory of LESIA, University of Biskra（LESIA实验室，比斯克拉大学） ； Sorbonne University Abu Dhabi, UAE（阿布扎赫尔索邦大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Study

Xuan Yao, Qianteng Wang, Xinbo Liu, Ke-Wei Huang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

FRIDA to the Rescue! Analyzing Synthetic Data Effectiveness in Object-Based Common Sense Reasoning for Disaster Response

Mollie Shichman, Claire Bonial, Austin Blodgett, Taylor Hudson, Francis Ferraro, Rachel Rudinger

机构 * University of Maryland, College Park（马里兰大学学院市分校） ； Army Research Lab（陆军研究实验室） ； Oak Ridge Associated Universities（橡树岭关联大学） ； University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 3 figures, 5 tables

LongReasonArena: A Long Reasoning Benchmark for Large Language Models

Jiayu Ding, Shuming Ma, Lei Cui, Nanning Zheng, Furu Wei

机构 * IAIR, Xi’an Jiaotong University（IAIR，西安交通大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models

Ziyao Shangguan, Chuhan Li, Yuxuan Ding, Yanan Zheng, Yilun Zhao, Tesca Fitzgerald, Arman Cohan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

GLARE: Agentic Reasoning for Legal Judgment Prediction

Xinyu Yang, Chenlong Deng, Zhicheng Dou

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学北京校区人工智能学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMs

Hongtao Liu, Zhicheng Du, Zihe Wang, Weiran Shen

机构 * Gaoling School of Artificial Intelligence（北京语言大学人工智能学院） ； Renmin University of China（中国人民大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

InfoCausalQA:Can Models Perform Non-explicit Causal Reasoning Based on Infographic?

Keummin Ka, Junhyeong Park, Jaehyun Jeon, Youngjae Yu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 14 pages, 9 figures

CrossWordBench: Evaluating the Reasoning Capabilities of LLMs and LVLMs with Controllable Puzzle Generation

Jixuan Leng, Chengsong Huang, Langlin Huang, Bill Yuchen Lin, William W. Cohen, Haohan Wang, Jiaxin Huang

机构 * CMU（卡内基梅隆大学） ； WUSTL（华盛顿大学） ； UIUC（伊利诺伊大学香槟分校） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Leaps Beyond the Seen: Reinforced Reasoning Augmented Generation for Clinical Notes

Lo Pang-Yun Ting, Chengshuai Zhao, Yu-Hua Zeng, Yuan Jee Lim, Kun-Ta Chuang, Huan Liu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Are Your LLMs Capable of Stable Reasoning?

Junnan Liu, Hongwei Liu, Linchen Xiao, Ziyi Wang, Kuikun Liu, Songyang Gao, Wenwei Zhang, Songyang Zhang, Kai Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 Camera, Benchmark: https://huggingface.co/datasets/opencompass/LiveMathBench, Code: https://github.com/open-compass/GPassK

Unveiling the Landscape of Clinical Depression Assessment: From Behavioral Signatures to Psychiatric Reasoning

Zhuang Chen, Guanqun Bi, Wen Zhang, Jiawei Hu, Aoyun Wang, Xiyao Xiao, Kun Feng, Minlie Huang

机构 * School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院） ； CoAI Group, DCST, IAI, BNRIST, Tsinghua University（清华大学） ； University of International Relations（国际关系大学） ； Central China Normal University（华中师范大学） ； Lingxin AI（灵心AI） ； Yuquan Hospital, Tsinghua University（清华大学友谊医院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

AgREE: Agentic Reasoning for Knowledge Graph Completion on Emerging Entities

Ruochen Zhao, Simone Conia, Eric Peng, Min Li, Saloni Potdar

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models

Wanqi Yang, Yanda Li, Yunchao Wei, Meng Fang, Ling Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Which LLMs Get the Joke? Probing Non-STEM Reasoning Abilities with HumorBench

Reuben Narad, Siddharth Suresh, Jiayi Chen, Pine S. L. Dysart-Bricken, Bob Mankoff, Robert Nowak, Jifan Zhang, Lalit Jain

机构 * University of Washington（华盛顿大学） ； University of Wisconsin-Madison（威斯康星大学麦迪逊分校） ； Air Mail and Cartoon Collections（航空邮报与漫画收藏） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Agentar-Fin-R1: Enhancing Financial Intelligence through Domain Expertise, Training Efficiency, and Advanced Reasoning

Yanjun Zheng, Xiyang Du, Longfei Liao, Xiaoke Zhao, Zhaowen Zhou, Jingze Song, Bo Zhang, Jiawei Liu, Xiang Qi, Zhe Li, Zhiqiang Zhang, Wei Wang, Peng Zhang

机构 * Ant Group（蚂蚁集团） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

AraTable: Benchmarking LLMs' Reasoning and Understanding of Arabic Tabular Data

Rana Alshaikh, Israa Alghanmi, Shelan Jeawak

机构 * Department of Information Systems, Faculty of Computing and Information Technology（信息系统系，计算与信息科技学院） ； King Abdulaziz University（国王阿卜杜勒阿齐兹大学） ； Department of Information Systems and Technology, College of Computer Science and Engineering（信息系统与技术系，计算机科学与工程学院） ； University of Jeddah（朱德赫大学） ； School of Computing and Creative Technologies（计算与创意科技学院） ； University of the West of England（西英格兰大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Why Braking? Scenario Extraction and Reasoning Utilizing LLM

Yin Wu, Daniel Slieter, Vivek Subramanian, Ahmed Abouelazm, Robin Bohn, J. Marius Zöllner

机构 * CARIAD SE（CARIAD公司） ； Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院） ； FZI Research Center for Information Technology（弗劳恩霍夫信息技术研究中心） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

BEnchmarking LLMs for Ophthalmology (BELO) for Ophthalmological Knowledge and Reasoning

Sahana Srinivasan, Xuguang Ai, Thaddaeus Wai Soon Lo, Aidan Gilson, Minjie Zou, Ke Zou, Hyunjae Kim, Mingjia Yang, Krithi Pushpanathan, Samantha Yew, Wan Ting Loke, Jocelyn Goh, Yibing Chen, Yiming Kong, Emily Yuelei Fu, Michelle Ongyong Hui, Kristen Nwanyanwu, Amisha Dave, Kelvin Zhenghao Li, Chen-Hsin Sun, Mark Chia, Gabriel Dawei Yang, Wendy Meihua Wong, David Ziyou Chen, Dianbo Liu, Maxwell Singer, Fares Antaki, Lucian V Del Priore, Jost Jonas, Ron Adelman, Qingyu Chen, Yih-Chung Tham

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Attend or Perish: Benchmarking Attention in Algorithmic Reasoning

Michal Spiegel, Michal Štefánik, Marek Kadlčík, Josef Kuchař

机构 * Kempelen Institute of Intelligent Technologies（智能技术研究所） ； Language Technology, University of Helsinki（语言技术，赫尔辛基大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Temporal reasoning for timeline summarisation in social media

Jiayu Song, Mahmud Elahi Akhter, Dana Atzil Slonim, Maria Liakata

机构 * Queen Mary University of London（伦敦女王学院） ； Bar-Ilan University（巴伊兰大学） ； The Alan Turing Institute（艾伦·图灵研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locating

Chao Deng, Jiale Yuan, Pi Bu, Peijie Wang, Zhong-Zhi Li, Jian Xu, Xiao-Hui Li, Yuan Gao, Jun Song, Bo Zheng, Cheng-Lin Liu

机构 * MAIS, Institute of Automation, Chinese Academy of Sciences（人工智能研究所，中国科学院） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学） ； Taobao & Tmall Group of Alibaba（阿里巴巴淘宝与天猫集团） ； Zhongguancun Academy, Beijing（中关村学院，北京） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Plancraft: an evaluation dataset for planning with LLM agents

Gautier Dagan, Frank Keller, Alex Lascarides

机构 * University of Edinburgh（爱丁堡大学） 纠错

专题命中 推理评测 ：planning(title,abstract)；分类 cs.CL、cs.AI