大模型推理能力

Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering

Tianjun Yao, Haoxuan Li, Zhiqiang Shen, Pan Li, Tongliang Liu, Kun Zhang

机构 * Mohamed bin Zayed University of Artificial Intelligence（莫扎德·本·扎耶德人工智能大学） ； Peking university（北京大学） ； Georgia Institute of Technology（佐治亚理工学院） ； The University of Sydney（悉尼大学） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments 32 pages, 28 figures

Robust Noise Attenuation via Adaptive Pooling of Transformer Outputs

Greyson Brothers

机构 * Johns Hopkins University Applied Physics Laboratory（约翰霍普金斯大学应用物理实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments [ICML 2025 Spotlight Poster] To be published in the Forty-Second International Conference on Machine Learning (ICML) Proceedings

LIFEBench: Evaluating Length Instruction Following in Large Language Models

Wei Zhang, Zhenhong Zhou, Kun Wang, Junfeng Fang, Yuanhe Zhang, Rui Wang, Ge Zhang, Xavier Li, Li Sun, Lingjuan Lyu, Yang Liu, Sen Su

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 81 pages, 22 tables, 32 figures. Homepage: https://ydyjya.github.io/LIFEBench/

Understand User Opinions of Large Language Models via LLM-Powered In-the-Moment User Experience Interviews

Mengqiao Liu, Tevin Wang, Cassandra A. Cohen, Sarah Li, Chenyan Xiong

机构 * Amazon（亚马逊公司） ； School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院） ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Evaluating Large Language Models on the Frame and Symbol Grounding Problems: A Zero-shot Benchmark

Shoko Oka

机构 * Independent Researcher（独立研究者） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 52 pages, Additional resources available on GitHub repository

When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration

Quan Shi, Carlos E. Jimenez, Shunyu Yao, Nick Haber, Diyi Yang, Karthik Narasimhan

机构 * Princeton Language and Intelligence（普林斯顿语言与智能研究所） ； Stanford University（斯坦福大学） ； OpenAI（开放人工智能研究所） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments For code, data, visualizer, visit: https://kite-live.vercel.app

Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models

Guangzhi Xiong, Eric Xie, Corey Williams, Myles Kim, Amir Hassan Shariatmadari, Sikun Guo, Stefan Bekiranov, Aidong Zhang

机构 * University of Virginia（弗吉尼亚大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to IJCAI 2025

CORDIAL: Can Multimodal Large Language Models Effectively Understand Coherence Relationships?

Aashish Anantha Ramakrishnan, Aadarsh Anantha Ramakrishnan, Dongwon Lee

机构 * The Pennsylvania State University（宾夕法尼亚州立大学） ； National Institute of Technology, Tiruchirappalli（特里奇里帕利学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments To appear at the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), Vienna, Austria, July 2025, https://2025.aclweb.org/

NeoBERT: A Next-Generation BERT

Lola Le Breton, Quentin Fournier, Mariam El Mezouar, John X. Morris, Sarath Chandar

机构 * Chandar Research Lab（昌达研究实验室） ； Mila – Quebec AI Institute（魁北克人工智能研究院） ； Polytechnique Montréal（蒙特利尔理工学院） ； Cornell University（康奈尔大学） ； Royal Military College of Canada（加拿大皇家军事学院） ； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 5 figures, 9 tables. Submitted to TMLR

Movie Facts and Fibs (MF$^2$): A Benchmark for Long Movie Understanding

Emmanouil Zaranis, António Farinhas, Saul Santos, Beatriz Canaverde, Miguel Moura Ramos, Aditya K Surikuchi, André Viveiros, Baohao Liao, Elena Bueno-Benito, Nithin Sivakumaran, Pavlo Vasylenko, Shoubin Yu, Sonal Sannigrahi, Wafaa Mohammed, Ben Peters, Danae Sánchez Villegas, Elias Stengel-Eskin, Giuseppe Attanasio, Jaehong Yoon, Stella Frank, Alessandro Suglia, Chrysoula Zerva, Desmond Elliott, Mariella Dimiccoli, Mohit Bansal, Oswald Lanz, Raffaella Bernardi, Raquel Fernández, Sandro Pezzelle, Vlad Niculae, André F. T. Martins

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Under Review

Hey, That's My Data! Label-Only Dataset Inference in Large Language Models

Chen Xiong, Zihao Wang, Rui Zhu, Tsung-Yi Ho, Pin-Yu Chen, Jingwei Xiong, Haixu Tang, Lucila Ohno-Machado

机构 * The Chinese University of Hong Kong（香港中文大学） ； Indiana University Bloomington（印第安纳大学布卢明顿分校） ； Yale University, School of Medicine（耶鲁大学医学院） ； IBM Research（IBM研究院） ； University of California, Davis（加州大学戴维斯分校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

DynamicMind: A Tri-Mode Thinking System for Large Language Models

Wei Li, Yanbin Wei, Qiushi Huang, Jiangyue Yan, Yang Chen, James T. Kwok, Yu Zhang

机构 * Southern University of Science and Technology（南方科技大学） ； Hong Kong University of Science and Technology（香港科技大学） ； University of Surrey（萨里大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

IDA-Bench: Evaluating LLMs on Interactive Guided Data Analysis

Hanyu Li, Haoyu Liu, Tingyu Zhu, Tianyu Guo, Zeyu Zheng, Xiaotie Deng, Michael I. Jordan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

BYO-Eval: Build Your Own Dataset for Fine-Grained Visual Assessment of Multimodal Language Models

Ludovic Arnould, Salim Khazem, Hugues Ali Mehenni

机构 * R&D Center, Talan（Talan 研发中心） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Attention-based transformer models for image captioning across languages: An in-depth survey and evaluation

Israa A. Albadarneh, Bassam H. Hammo, Omar S. Al-Kadi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 31 pages, 15 figures, 6 tables

Journal ref Computer Science Review, Vol. 58, pp. 100766, 2025

MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal Large Language Models

Gio Paik, Geewook Kim, Jinbae Im

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments ACL Findings 2025

MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learning

Yiqing Liang, Jielin Qiu, Wenhao Ding, Zuxin Liu, James Tompkin, Mengdi Xu, Mengzhou Xia, Zhengzhong Tu, Laixi Shi, Jiacheng Zhu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Project Webpage: https://modomodo-rl.github.io/

Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Prior

Yue Gong, Raul Castro Fernandez

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Under Review

Data Laundering: Artificially Boosting Benchmark Results through Knowledge Distillation

Jonibek Mansurov, Akhmed Sakip, Alham Fikri Aji

机构 * Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed人工智能大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 14 pages

Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giant

Jemin Lee, Sihyeong Park, Jinse Kwon, Jihun Oh, Yongin Kwon

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted in IJCAI 2025, 21 pages, 2 figure

Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs

Wenxuan Wang, Xiaoyuan Liu, Kuiyi Gao, Jen-tse Huang, Youliang Yuan, Pinjia He, Shuai Wang, Zhaopeng Tu

机构 * Renmin University of China（中国人民大学） ； Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Chinese University of Hong Kong（香港中文大学） ； Johns Hopkins University（约翰霍普金斯大学） ； Hong Kong University of Science and Technology（香港科技大学） ； Tencent（腾讯） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2025

Contextual Paralinguistic Data Creation for Multi-Modal Speech-LLM: Data Condensation and Spoken QA Generation

Qiongqiong Wang, Hardik B. Sailor, Tianchi Liu, Ai Ti Aw

机构 * Agency for Science, Technology and Research (A ⋆ ⋆ \star ⋆ STAR)（科技研究局） ； Institute for Infocomm Research (I 2 R)（信息通信研究所） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at Interspeech 2025. [v2]: The dataset has been released, and the link is now updated

Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation

Mohammad Mahdi Abootorabi, Amirhosein Zobeiri, Mahdi Dehghani, Mohammadali Mohammadkhani, Bardia Mohammadi, Omid Ghahroodi, Mahdieh Soleymani Baghshah, Ehsaneddin Asgari

机构 * Computer Engineering Department, Sharif University of Technology（沙斐大学计算机工程系） ； College of Interdisciplinary Science and Technology, University of Tehran（德黑兰大学跨学科科学与技术学院） ； Computer Engineering Department, K.N. Toosi University of Technology（技术学院计算机工程系） ； Qatar Computing Research Institute（卡塔尔计算研究院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments GitHub repository: https://github.com/llm-lab-org/Multimodal-RAG-Survey

Abstractive Visual Understanding of Multi-modal Structured Knowledge: A New Perspective for MLLM Evaluation

Yichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu, Min Zhang, Wen Zhang, Huajun Chen

机构 * Zhejiang University（浙江大学） ； Tianjin University（天津大学） ； Harbin Institute of Technology（哈尔滨工业大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Work in progress

XAI-Units: Benchmarking Explainability Methods with Unit Tests

Jun Rui Lee, Sadegh Emami, Michael David Hollins, Timothy C. H. Wong, Carlos Ignacio Villalobos Sánchez, Francesca Toni, Dekai Zhang, Adam Dejl

机构 * Department of Computing（计算系） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments Accepted at FAccT 2025

Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?

Zhuojun Gu, Quan Wang, Shuchu Han

机构 * Department of Information Systems and Business Analytics（信息系统与商业分析系） ； University at Albany - State University of New York（阿尔巴尼大学 - 新 york 状态大学） ； Santa Clara, CA（圣克拉拉，加州） ； Princeton Junction, NJ（普林斯顿 junction，新泽西） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuning

Saad Hossain, Samanvay Vajpayee, Sirisha Rambhatla

机构 * Critical ML Lab（关键机器学习实验室） ； University of Toronto（多伦多大学） ； University of Waterloo（滑铁卢大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks

Suhana Bedi, Hejie Cui, Miguel Fuentes, Alyssa Unell, Michael Wornow, Juan M. Banda, Nikesh Kotecha, Timothy Keyes, Yifan Mai, Mert Oez, Hao Qiu, Shrey Jain, Leonardo Schettini, Mehr Kashyap, Jason Alan Fries, Akshay Swaminathan, Philip Chung, Fateme Nateghi, Asad Aali, Ashwin Nayak, Shivam Vedak, Sneha S. Jain, Birju Patel, Oluseyi Fayanju, Shreya Shah, Ethan Goh, Dong-han Yao, Brian Soetikno, Eduardo Reis, Sergios Gatidis, Vasu Divi, Robson Capasso, Rachna Saralkar, Chia-Chun Chiang, Jenelle Jindal, Tho Pham, Faraz Ghoddusi, Steven Lin, Albert S. Chiou, Christy Hong, Mohana Roy, Michael F. Gensheimer, Hinesh Patel, Kevin Schulman, Dev Dash, Danton Char, Lance Downing, Francois Grolleau, Kameron Black, Bethel Mieso, Aydin Zahedivash, Wen-wai Yim, Harshita Sharma, Tony Lee, Hannah Kirsch, Jennifer Lee, Nerissa Ambers, Carlene Lugtu, Aditya Sharma, Bilal Mawji, Alex Alekseyev, Vicky Zhou, Vikas Kakkar, Jarrod Helzer, Anurang Revri, Yair Bannett, Roxana Daneshjou, Jonathan Chen, Emily Alsentzer, Keith Morse, Nirmal Ravi, Nima Aghaeepour, Vanessa Kennedy, Akshay Chaudhari, Thomas Wang, Sanmi Koyejo, Matthew P. Lungren, Eric Horvitz, Percy Liang, Mike Pfeffer, Nigam H. Shah

机构 * Stanford University School of Medicine（斯坦福大学医学院） ； Stanford Health Care（斯坦福健康系统） ； Center for Research on Foundation Models (CRFM) & Department of Computer Science（基础模型研究中心（CRFM）及计算机科学系） ； Microsoft Corporation（微软公司） ； Stanford Institute for Human-Centered AI（斯坦福大学人本人工智能研究所） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems

Xinwu Ye, Chengfan Li, Siming Chen, Wei Wei, Xiangru Tang

机构 * Department of Computer Science, Yale University（耶鲁大学计算机科学系） ； Department of Computer Science, Brown University（布朗大学计算机科学系） ； School of Data Science, Fudan University（复旦大学数据科学学院） ； Shanghai Key Laboratory of Data Science（上海数据科学关键实验室） ； Datawiz LLC 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Accepted to the Findings of the Association for Computational Linguistics (ACL 2025)

MIR: Methodology Inspiration Retrieval for Scientific Research Problems

Aniketh Garikaparthi, Manasi Patwardhan, Aditya Sanjiv Kanade, Aman Hassan, Lovekesh Vig, Arman Cohan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025