大模型推理能力

ALARB: An Arabic Legal Argument Reasoning Benchmark

Harethah Abu Shairah, Somayah AlHarbi, Abdulaziz AlHussein, Sameer Alsabea, Omar Shaqaqi, Hebah AlShamlan, Omar Knio, George Turkiyyah

机构 * King Abdullah University of Science and Technology（卡塔尔国王阿卜杜勒阿齐兹科学与技术大学） ； THIQAH 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted paper at ArabicNLP 2025

TASER: Translation Assessment via Systematic Evaluation and Reasoning

Monishwaran Maheswaran, Marco Carini, Christian Federmann, Tony Diaz

机构 * University of California, Berkeley（加州大学伯克利分校） ； Apple Inc.（苹果公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

EH-Benchmark Ophthalmic Hallucination Benchmark and Agent-Driven Top-Down Traceable Reasoning Workflow

Xiaoyu Pan, Yang Bai, Ke Zou, Yang Zhou, Jun Zhou, Huazhu Fu, Yih-Chung Tham, Yong Liu

机构 * Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR)（高性能计算研究所，科技研究局（A*STAR）） ； Centre for Innovation and Precision Eye Health（创新与精准眼健康中心） ； Department of Ophthalmology, NUHS Tower Block, Level 7, 1E Kent Ridge Road, Singapore, 119228（眼科部，NUHS塔楼7层，1E Kent Ridge Road，新加坡，119228） ； Singapore Eye Research Institute, Singapore National Eye Centre, 20 College Road, Singapore, 169856（新加坡眼研究 institute，新加坡国家眼科中心，20 College Road，新加坡，169856） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 9 figures, 5 tables. submit/6621751

Aligning Multilingual Reasoning with Verifiable Semantics from a High-Resource Expert Model

Fahim Faisal, Kaiqiang Song, Song Wang, Simin Ma, Shujian Liu, Haoyun Deng, Sathish Reddy Indurthi

机构 * Zoom 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model Merging

Xiaochong Lan, Yu Zheng, Shiteng Cao, Yong Li

机构 * Tsinghua University（清华大学） ； Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models

Jue Zhang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

机构 * Microsoft（微软） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP'25 (Main)

RCI: A Score for Evaluating Global and Local Reasoning in Multimodal Benchmarks

Amit Agarwal, Hitesh Laxmichand Patel, Srikant Panda, Hansa Meghwani, Jyotika Singh, Karan Dua, Paul Li, Tao Sheng, Sujith Ravi, Dan Roth

机构 * Oracle AI 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted in EMNLP 2025

Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs

Zhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； The Hong Kong University of Science and Technology（香港科技大学） ； Southeast University（东南大学） 纠错

专题命中 推理评测 ：chain-of-thought(title,abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2025 Main

One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning

Sualeha Farid, Jayden Lin, Zean Chen, Shivani Kumar, David Jurgens

机构 * University of Michigan（密歇根大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 11 figures, 6 tables

JUREX-4E: Juridical Expert-Annotated Four-Element Knowledge Base for Legal Reasoning

Huanghai Liu, Quzhe Huang, Qingjing Chen, Yiran Hu, Jiayu Ma, Yun Liu, Weixing Shen, Yansong Feng

机构 * School of Law, Tsinghua University（清华大学法学院） ； Wangxuan Institute of Computer Technology, Peking University（北京大学王璇计算机技术研究所） ； Department of Legal Studies, University of Bologna（博洛尼亚大学法律系） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World

Saeed Almheiri, Rania Hossam, Mena Attia, Chenxi Wang, Preslav Nakov, Timothy Baldwin, Fajri Koto

机构 * Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2025 - Findings

Brittleness and Promise: Knowledge Graph Based Reward Modeling for Diagnostic Reasoning

Saksham Khatwani, He Cheng, Majid Afshar, Dmitriy Dligach, Yanjun Gao

机构 * University of Colorado Boulder（科罗拉多大学博尔德分校） ； University of Colorado Anschutz（科罗拉多大学安舒茨分校） ； University of Wisconsin - Madison（威斯康星大学麦迪逊分校） ； Loyola University Chicago（芝加哥洛约拉大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

MSCoRe: A Benchmark for Multi-Stage Collaborative Reasoning in LLM Agents

Yuzhen Lei, Hongbin Xie, Jiaxing Zhao, Shuangxue Liu, Xuan Song

机构 * Jilin University（吉林大学） ； Southern University of Science and Technology（南方科技大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 5 figures

Agentic AI with Orchestrator-Agent Trust: A Modular Visual Classification Framework with Trust-Aware Orchestration and RAG-Based Reasoning

Konstantinos I. Roumeliotis, Ranjan Sapkota, Manoj Karkee, Nikolaos D. Tselikas

机构 * University of the Peloponnese, Department of Informatics and Telecommunications（希腊皮埃蒙特大学信息与电信系） ； Cornell University, Department of Biological and Environmental Engineering（康奈尔大学生物与环境工程系） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Seeing Culture: A Benchmark for Visual Reasoning and Grounding

Burak Satar, Zhixin Ma, Patrick A. Irawan, Wilfried A. Mulyawan, Jing Jiang, Ee-Peng Lim, Chong-Wah Ngo

机构 * Singapore Management University（新加坡管理大学） ； Bandung Institute of Technology（班达理工大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2025 Main Conference, https://seeingculture-benchmark.github.io/

GRID: Graph-based Reasoning for Intervention and Discovery in Built Environments

Taqiya Ehsan, Shuren Xia, Jorge Ortiz

机构 * Rutgers University（罗格斯大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

ASTRA: A Negotiation Agent with Adaptive and Strategic Reasoning via Tool-integrated Action for Dynamic Offer Optimization

Deuksin Kwon, Jiwon Hae, Emma Clift, Daniel Shamsoddini, Jonathan Gratch, Gale M. Lucas

机构 * University of Southern California（美国南加州大学） ； St. Olaf College（圣奥拉夫学院） ； Northwestern University（西北大学） ； USC for Institute of Creative Technologies（美国南加州大学创意技术研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2025 (Main Conference)

Spatial Audio Motion Understanding and Reasoning

Arvind Krishna Sridhar, Yinyi Guo, Erik Visser

机构 * Qualcomm Technologies Inc.（高通技术公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 5 pages, 2 figures, 3 tables

MovieCORE: COgnitive REasoning in Movies

Gueter Josmy Faure, Min-Hung Chen, Jia-Fong Yeh, Ying Cheng, Hung-Ting Su, Yung-Hao Tang, Shang-Hong Lai, Winston H. Hsu

机构 * National Taiwan University（国立台湾大学） ； NVIDIA ； National Tsing Hua University（国立清华大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted for EMNLP'2025 Main Conference (Oral Presentation). Project Page: https://joslefaure.github.io/assets/html/moviecore.html

Assessing Large Language Models on Islamic Legal Reasoning: Evidence from Inheritance Law Evaluation

Abdessalam Bouchekif, Samer Rashwani, Heba Sbahi, Shahd Gaben, Mutaz Al-Khatib, Mohammed Ghaly

机构 * Hamad Bin Khalifa University（哈马德·本·卡西姆大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 7 Tables, Code: https://github.com/bouchekif/inheritance_evaluation

FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning

Liang Hu, Jianpeng Jiao, Jiashuo Liu, Yanle Ren, Zhoufutu Wen, Kaiyuan Zhang, Xuanliang Zhang, Xiang Gao, Tianci He, Fei Hu, Yali Liao, Zaiyuan Wang, Chenghao Yang, Qianyu Yang, Mingren Yin, Zhiyuan Zeng, Ge Zhang, Xinyi Zhang, Xiying Zhao, Zhenwei Zhu, Hongseok Namkoong, Wenhao Huang, Yuwen Tang

机构 * Columbia Business School（哥伦比亚商学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments 29 pages

Zero-shot Graph Reasoning via Retrieval Augmented Framework with LLMs

Hanqing Li, Kiran Sheena Jyothi, Henry Liang, Sharika Mahadevan, Diego Klabjan

机构 * Northwestern University（西北大学） ； EXL Service（EXL服务） ； Vail Systems（Vail系统） ； Netflix 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

MORABLES: A Benchmark for Assessing Abstract Moral Reasoning in LLMs with Fables

Matteo Marcuzzo, Alessandro Zangari, Andrea Albarelli, Jose Camacho-Collados, Mohammad Taher Pilehvar

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2025 Main Conference

EthicsMH: A Pilot Benchmark for Ethical Reasoning in Mental Health AI

Sai Kartheek Reddy Kasu

机构 * IIIT Dharwad（德瓦德理工学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Automated MCQA Benchmarking at Scale: Evaluating Reasoning Traces as Retrieval Sources for Domain Adaptation of Small Language Models

Ozan Gokdemir, Neil Getty, Robert Underwood, Sandeep Madireddy, Franck Cappello, Arvind Ramanathan, Ian T. Foster, Rick L. Stevens

机构 * Argonne National Laboratory（阿贡国家实验室） ； The University of Chicago（芝加哥大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments This manuscript has been accepted for publication at the Supercomputing 25 (SC '25) Conference (Frontiers in Generative AI for HPC Science and Engineering: Foundations, Challenges, and Opportunities Workshop) in St. Louis, MO, USA on November 16th, 2025. It will appear in the SC25 Workshop Proceedings after that date

Optimizing Length Compression in Large Reasoning Models

Zhengxiang Cheng, Dongping Chen, Mingyang Fu, Tianyi Zhou

机构 * umd（马里兰大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 7 figures, 4 tables

MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapes

Nilay Pande, Sahiti Yerramilli, Jayant Sravan Tamarapalli, Rynaa Grover

机构 * Waymo ； Google（谷歌） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Reasoning Language Model for Personalized Lung Cancer Screening

Chuang Niu, Ge Wang

机构 * Department of Biomedical Engineering, School of Engineering, Center for Computational Innovations, Center for Biotechnology & Interdisciplinary Studies（生物医学工程系、工程学院、计算创新中心、生物技术与跨学科研究所以及中心） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Reverse-Engineered Reasoning for Open-Ended Generation

Haozhe Wang, Haoran Que, Qixin Xu, Minghao Liu, Wangchunshu Zhou, Jiazhan Feng, Wanjun Zhong, Wei Ye, Tong Yang, Wenhao Huang, Ge Zhang, Fangzhen Lin

机构 * Hong Kong University of Science and Technology（香港科技大学） ； Peking University（北京大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Preprint

CVPD at QIAS 2025 Shared Task: An Efficient Encoder-Based Approach for Islamic Inheritance Reasoning

Salah Eddine Bekhouche, Abdellah Zakaria Sellam, Hichem Telli, Cosimo Distante, Abdenour Hadid

机构 * University of the Basque Country UPV/EHU（巴斯克大学UPV/EHU） ； Institute of Applied Sciences and Intelligent Systems – CNR（应用科学与智能系统研究所 – CNR） ； Laboratory of LESIA, University of Biskra（LESIA实验室，比斯克拉大学） ； Sorbonne University Abu Dhabi, UAE（阿布扎赫尔索邦大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG