大模型推理能力

Predicting Implicit Arguments in Procedural Video Instructions

Anil Batra, Laura Sevilla-Lara, Marcus Rohrbach, Frank Keller

机构 * University of Edinburgh（爱丁堡大学） ； TU Darmstadt（德累斯顿理工大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ACL 2025 Main

Large Language Models for IT Automation Tasks: Are We There Yet?

Md Mahadi Hassan, John Salvador, Akond Rahman, Santu Karmaker

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 8 pages

CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis

Ruixiang Feng, Shen Gao, Xiuying Chen, Lisi Chen, Shuo Shang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments accepted by ACL 2025

Enhancing the Comprehensibility of Text Explanations via Unsupervised Concept Discovery

Yifan Sun, Danding Wang, Qiang Sheng, Juan Cao, Jintao Li

机构 * University of Chinese Academy of Sciences（中国科学院大学） ； Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所） ； Media Synthesis and Forensics Lab（媒体合成与取证实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ACL 2025 Findings

DFIR-Metric: A Benchmark Dataset for Evaluating Large Language Models in Digital Forensics and Incident Response

Bilel Cherif, Tamas Bisztray, Richard A. Dubniczky, Aaesha Aldahmani, Saeed Alshehhi, Norbert Tihanyi

机构 * Technology Innovation Institute（技术创新研究所） ； University of Oslo（奥斯陆大学） ； Eötvös Loránd University（埃斯特哈兹洛朗大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs

Wafa Alghallabi, Ritesh Thawkar, Sara Ghaboura, Ketan More, Omkar Thawakar, Hisham Cholakkal, Salman Khan, Rao Muhammad Anwer

机构 * Mohamed bin Zayed University of AI（Mohamed bin Zayed人工智能大学） ； Australian National University（澳大利亚国立大学） ； Aalto University（艾尔沃斯大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Github:https://github.com/mbzuai-oryx/FannOrFlop, Dataset:https://huggingface.co/datasets/omkarthawakar/FannOrFlop

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation

Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation

Dimitris Gkoumas, Maria Liakata

机构 * Queen Mary University of London（伦敦女王学院） ； The Alan Turing Institute（艾伦·图灵研究所） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ACL25 Main

ExAnte: A Benchmark for Ex-Ante Inference in Large Language Models

Yachuan Liu, Xiaochun Wei, Lin Shi, Xinnuo Li, Bohan Zhang, Paramveer Dhillon, Qiaozhu Mei

机构 * University of Michigan（密歇根大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Causal Distillation: Transferring Structured Explanations from Large to Compact Language Models

Aggrey Muhebwa, Khalid K. Osman

机构 * Stanford University（斯坦福大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Retrieval-Augmented Generation for Service Discovery: Chunking Strategies and Benchmarking

Robin D. Pesl, Jerin G. Mathew, Massimo Mecella, Marco Aiello

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2411.19804

Misleading through Inconsistency: A Benchmark for Political Inconsistencies Detection

Nursulu Sagimbayeva, Ruveyda Betül Bahçeci, Ingmar Weber

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 8 pages, 6 figures. Accepted for publication in the Proceedings of 1st Workshop on Misinformation Detection in the Era of LLMs (MisD) at ICWSM-2025

Medical Large Vision Language Models with Multi-Image Visual Ability

Xikai Yang, Juzheng Miao, Yuchen Yuan, Jiaze Wang, Qi Dou, Jinpeng Li, Pheng-Ann Heng

机构 * Dept. of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China（计算机科学与工程系，香港中文大学，香港，中国） ； Institute of Medical Intelligence and XR, The Chinese University of Hong Kong, Hong Kong, China（医学智能与XR研究所，香港中文大学，香港，中国） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 10 pages, 4 figures

Response Uncertainty and Probe Modeling: Two Sides of the Same Coin in LLM Interpretability?

Yongjie Wang, Yibo Wang, Xin Zhou, Zhiqi Shen

机构 * Nanyang Technological University（南洋理工大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 18 Pages

UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning

Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Guanjing Xiong, Hongsheng Li

机构 * vivo AI Lab（vivo人工智能实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Updated UI-R1-E-3B

PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialogues

Fangxu Yu, Lai Jiang, Shenyi Huang, Zhen Wu, Xinyu Dai

机构 * National Key Laboratory for Novel Software Technology, Nanjing University, China（南京大学新型软件技术国家重点实验室） ； School of Artificial Intelligence, Nanjing University, China（南京大学人工智能学院） ； Department of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程系） ； University of California, San Diego, CA, USA（美国加州大学圣地亚哥分校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Efficient Lung Ultrasound Severity Scoring Using Dedicated Feature Extractor

Jiaqi Guo, Yunan Wu, Evangelos Kaimakamis, Georgios Petmezas, Vasileios E. Papageorgiou, Nicos Maglaveras, Aggelos K. Katsaggelos

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Accepted by IEEE ISBI 2025 (Selected for oral presentation); 2025/4/15 (v2): Corrected a notation error in Figure 2

Journal ref 2025 IEEE 22nd International Symposium on Biomedical Imaging (ISBI), Houston, TX, USA, 2025, pp. 1-5

AI-Augmented LLMs Achieve Therapist-Level Responses in Motivational Interviewing

Yinghui Huang, Yuxuan Jiang, Hui Liu, Yixin Cai, Weiqing Li, Xiangen Hu

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

Comments 21 pages, 5 figures

CHART-6: Human-Centered Evaluation of Data Visualization Understanding in Vision-Language Models

Arnav Verma, Kushin Mukherjee, Christopher Potts, Elisa Kreiss, Judith E. Fan

机构 * Department of Psychology at Stanford University（斯坦福大学心理学系） ； Department of Linguistics at Stanford University（斯坦福大学语言学系） ； Department of Communication at University of California, Los Angeles（加州大学洛杉矶分校传播系） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comparative Evaluation of Prompting and Fine-Tuning for Applying Large Language Models to Grid-Structured Geospatial Data

Akash Dhruv, Yangxinyu Xie, Jordan Branham, Tanwi Mallick

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

KCIF: Knowledge-Conditioned Instruction Following

Rudra Murthy, Praveen Venkateswaran, Prince Kumar, Danish Contractor

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Under Review

Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answering

Bowen Jiang, Runchuan Zhu, Jiang Wu, Zinco Jiang, Yifan He, Junyuan Gao, Jia Yu, Rui Min, Yinfan Wang, Haote Yang, Songyang Zhang, Dahua Lin, Lijun Wu, Conghui He

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Peking University（北京大学） ； University of Chinese Academy of Sciences（中国科学院大学） ； Shanghai University（上海大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Equal contribution: Bowen Jiang, Runchuan Zhu, Jiang Wu; Corresponding author: Conghui He

Advancing the Scientific Method with Large Language Models: From Hypothesis to Discovery

Yanbo Zhang, Sumeer A. Khan, Adnan Mahmud, Huck Yang, Alexander Lavin, Michael Levin, Jeremy Frey, Jared Dunnmon, James Evans, Alan Bundy, Saso Dzeroski, Jesper Tegner, Hector Zenil

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 45 pages

Journal ref npj Artificial Intelligence, 2025

Not All Correct Answers Are Equal: Why Your Distillation Source Matters

Xiaoyu Tian, Yunjie Ji, Haotian Wang, Shuaiting Chen, Sitong Zhao, Yiping Peng, Han Zhao, Xiangang Li

机构 * Beike (Ke.com)（贝克（凯.com）） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

ORION Grounded in Context: Retrieval-Based Method for Hallucination Detection

Assaf Gerner, Netta Madvil, Nadav Barak, Alex Zaikman, Jonatan Liberman, Liron Hamra, Rotem Brazilay, Shay Tsadok, Yaron Friedman, Neal Harow, Noam Bressler, Shir Chorev, Philip Tannor

机构 * Deepchecks 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation

Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs, Andrew Head, Mark Yatskar, Chris Callison-Burch, Ranjay Krishna, Aniruddha Kembhavi, Christopher Clark

机构 * University of Pennsylvania（宾夕法尼亚大学） ； Allen Institute for Artificial Intelligence（人工智能研究所） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Published in ACL 2025, project page: https://yueyang1996.github.io/cosyn/

X-WebAgentBench: A Multilingual Interactive Web Benchmark for Evaluating Global Agentic System

Peng Wang, Ruihan Tao, Qiguang Chen, Mengkang Hu, Libo Qin

专题命中 推理评测 ：planning(abstract)；分类 cs.CL

Comments Accepted by ACL 2025 Findings

SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models

Jing Yu, Yuqi Tang, Kehua Feng, Mingyang Rao, Lei Liang, Zhiqiang Zhang, Mengshu Sun, Wen Zhang, Qiang Zhang, Keyan Ding, Huajun Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 25 pages, 4 figures

A Comprehensive Evaluation of Large Language Models on Temporal Event Forecasting

He Chang, Chenchen Ye, Zhulin Tao, Jie Wu, Zhengmao Yang, Yunshan Ma, Xianglin Huang, Tat-Seng Chua

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

S2SBench: A Benchmark for Quantifying Intelligence Degradation in Speech-to-Speech Large Language Models

Yuanbo Fang, Haoze Sun, Jun Liu, Tao Zhang, Zenan Zhou, Weipeng Chen, Xiaofen Xing, Xiangmin Xu

机构 * South China University of Technology（华南理工大学） ； Baichuan Inc（百川科技） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL