大模型推理能力

Scaling LLM Planning: NL2FLOW for Parametric Problem Generation and Rigorous Evaluation

Jungkoo Kang

机构 * IBM Research（IBM研究院） ； MIT-IBM Watson AI Lab（麻省理工-IBM沃森人工智能实验室） 纠错

专题命中 推理评测 ：planning(title,abstract)；reasoning(abstract)；分类 cs.AI

Comments 30 pages, 7 figures

BanglaMATH : A Bangla benchmark dataset for testing LLM mathematical reasoning at grades 6, 7, and 8

Tabia Tanzin Prama, Christopher M. Danforth, Peter Sheridan Dodds

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)

MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual Reasoning

Rajarshi Ghosh, Abhay Gupta, Hudson McBride, Anurag Vaidya, Faisal Mahmood

机构 * Lone Star College ； Algoverse AI Research ； Empire State University ； Brigham and Women’s Hospital, Harvard Medical School 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinking

Alireza S. Ziabari, Nona Ghazizadeh, Zhivar Sourati, Farzan Karimi-Malekabadi, Payam Piray, Morteza Dehghani

机构 * University of Southern California（南加州大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Benchmarking LLMs' Swarm intelligence

Kai Ruan, Mowen Huang, Ji-Rong Wen, Hao Sun

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（人工智能学院，中国人民大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.CL

Evaluating and Mitigating Social Bias for Large Language Models in Open-ended Settings

Zhao Liu, Tian Xie, Xueru Zhang

机构 * The Ohio State University（俄亥俄州立大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 15 pages

Unlocking Public Catalogues: Instruction-Tuning LLMs for ICD Coding of German Tumor Diagnoses

Stefan Lenz, Lakisha Ortiz Rosario, Georg Vollmar, Arsenij Ustjanzew, Fatma Alickovic, Thomas Kindler, Torsten Panholzer

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 19 pages, 4 figures

Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems

Karthik Avinash, Nikhil Pareek, Rishav Hada

机构 * FutureAGI Inc.（未来人工智能公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learning

Mahdi Cherakhloo, Arash Abbasi, Mohammad Saeid Sarafraz, Bijan Vosoughi Vahdat

机构 * Department of Medical Engineering（医学工程系） ； Electrical Engineering Department（电气工程系） ； Sharif University of Technology（谢里夫技术大学） ； Department of Electrical Engineering（电气工程系） ； University of Tehran（德黑兰大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs

Xin Gao, Ruiyi Zhang, Daniel Du, Saurabh Mahindre, Sai Ashish Somayajula, Pengtao Xie

机构 * UC San Diego（UC圣迭戈大学） ； SUNY Buffalo（纽约州立大学布法罗分校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Published at EMNLP 2025; Code and data available at https://github.com/gxx27/time_unlearn

Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations

A. Bochkov

机构 * Moscow Institute of Physics and Technology (MIPT)（莫斯科物理技术学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Published in Transactions on Machine Learning Research (10/2025). OpenReview: https://openreview.net/forum?id=Odh8IynO1o

Journal ref Transactions on Machine Learning Research (TMLR), 2025

MULTI: Multimodal Understanding Leaderboard with Text and Images

Zichen Zhu, Yang Xu, Lu Chen, Jingkai Yang, Yichuan Ma, Yiming Sun, Hailin Wen, Jiaqi Liu, Jinyu Cai, Yingzi Ma, Situo Zhang, Zihan Zhao, Liangtai Sun, Kai Yu

机构 * X-LANCE Lab, School of Computer Science, Key Laboratory of Artificial Intelligence\ of Education, Shanghai Jiao Tong University, Shanghai 200240 , China ； Jiangsu Key Lab of Language Computing, Suzhou 215123 , China ； College of Computing ； Data Science, Nanyang Technological University, Singapore 639798 , Singapore ； Suzhou Laboratory, Suzhou 215123 , China 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 19 figures, 10 tables. Details and access are available at: https://OpenDFM.github.io/MULTI-Benchmark/

Journal ref Sci. China Inf. Sci. 68, 200107 (2025)

Selective Adversarial Attacks on LLM Benchmarks

Ivan Dubrovsky, Anastasia Orlova, Illarion Iov, Nina Gubina, Irena Gureeva, Alexey Zaytsev

机构 * ITMO University（ITMO大学） ； Applied AI Institute（应用人工智能研究所） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinking

Sifan Li, Yujun Cai, Yiwei Wang

机构 * University of California, Merced（加州大学默塞德分校） ； University of Queensland（昆士兰大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

MSEarth: A Multimodal Scientific Dataset and Benchmark for Phenomena Uncovering in Earth Science

Xiangyu Zhao, Wanghan Xu, Bo Liu, Yuhao Zhou, Fenghua Ling, Ben Fei, Xiaoyu Yue, Lei Bai, Wenlong Zhang, Xiao-Ming Wu

机构 * The Hong Kong Polytechnic University（香港理工大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Shanghai Jiao Tong University（上海交通大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Human-MME: A Holistic Evaluation Benchmark for Human-Centric Multimodal Large Language Models

Yuansen Liu, Haiming Tang, Jinlong Peng, Jiangning Zhang, Xiaozhong Ji, Qingdong He, Wenbin Wu, Donghao Luo, Zhenye Gan, Junwei Zhu, Yunhang Shen, Chaoyou Fu, Chengjie Wang, Xiaobin Hu, Shuicheng Yan

机构 * Yuan-Hou（元侯） 纠错

专题命中 推理评测 ：reasoning(abstract)