大模型推理能力

2506.18501 2025-08-12 cs.CL cs.AI 73%

Comparative Evaluation of ChatGPT and DeepSeek Across Key NLP Tasks: Strengths, Weaknesses, and Domain-Specific Performance

Wael Etaiwi, Bushra Alhijawi

机构 * Princess Sumaya University for Technology（普里incess苏玛亚技术大学）

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

2508.07111 2025-08-12 cs.CL cs.AI 73%

Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution

Falaah Arif Khan, Nivedha Sivakumar, Yinong Oliver Wang, Katherine Metcalf, Cezanne Camacho, Barry-John Theobald, Luca Zappella, Nicholas Apostoloff

机构 * Apple（苹果公司）

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

2502.17810 2025-08-12 cs.CL eess.AS 70%

URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models

Ruiqi Yan, Xiquan Li, Wenxi Chen, Zhikang Niu, Chen Yang, Ziyang Ma, Kai Yu, Xie Chen

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

2508.06600 2025-08-12 cs.CL cs.IR 70%

BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Sahel Sharifymoghaddam, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, Jimmy Lin

机构 * University of Waterloo（滑铁卢大学）； CSIRO（澳大利亚联邦科学与工业研究组织）； Independent（独立研究者）； Carnegie Mellon University（卡内基梅隆大学）； The University of Queensland（昆士兰大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.CL

2508.07485 2025-08-12 cs.AI cs.CL cs.CY cs.LG 67%

Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy

Alexander Duffy, Samuel J Paech, Ishana Shastri, Elizabeth Karpinski, Baptiste Alloui-Cros, Tyler Marques, Matthew Lyle Olson

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2508.07022 2025-08-12 cs.AI cs.CL cs.LG cs.MM 67%

MultiMedEdit: A Scenario-Aware Benchmark for Evaluating Knowledge Editing in Medical VQA

Shengtao Wen, Haodong Chen, Yadong Wang, Zhongying Pan, Xiang Chen, Yu Tian, Bo Qian, Dong Liang, Sheng-Jun Huang

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Under Review

2508.06624 2025-08-12 cs.CV 67%

VL-MedGuide: A Visual-Linguistic Large Model for Intelligent and Explainable Skin Disease Auxiliary Diagnosis

Kexin Yu, Zihan Xu, Jialei Xie, Carter Adams

机构 * Jiangsu Ocean University（江苏海洋大学）； Federal University of Bahia（巴伊亚联邦大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)

2508.07284 2025-08-12 cs.CL cs.AI cs.CY 62%

"Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas

Junchen Ding, Penghao Jiang, Zihao Xu, Ziqi Ding, Yichen Zhu, Jiaojiao Jiang, Yuekang Li

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI

2508.07273 2025-08-12 cs.CL cs.AI eess.AS 62%

Incorporating Contextual Paralinguistic Understanding in Large Speech-Language Models

Qiongqiong Wang, Hardik B. Sailor, Jeremy H. M. Wong, Tianchi Liu, Shuo Sun, Wenyu Zhang, Muhammad Huzaifah, Nancy Chen, Ai Ti Aw

机构 * Institute for Infocomm Research (I 2 R), Agency for Science, Technology

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at (ASRU 2025) 2025 IEEE Automatic Speech Recognition and Understanding Workshop

2508.07179 2025-08-12 cs.CL cs.AI cs.DB 62%

Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks

Jiaqi Yin, Yi-Wei Chen, Meng-Lung Lee, Xiya Liu

机构 * Microsoft Redmond, WA（微软红木城）； Antra. Inc.（Antra公司）

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI

2508.06960 2025-08-12 cs.AI cs.CL 62%

DatasetResearch: Benchmarking Agent Systems for Demand-Driven Dataset Discovery

Keyu Li, Mohan Jiang, Dayuan Fu, Yunze Wu, Xiangkun Hu, Dequan Wang, Pengfei Liu

机构 * Shanghai Jiao Tong University（上海交通大学）； SII ； GAIR

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI

2508.07714 2025-08-12 cs.CV cs.AI cs.ET 57%

DoorDet: Semi-Automated Multi-Class Door Detection Dataset via Object Detection and Large Language Models

Licheng Zhang, Bach Le, Naveed Akhtar, Tuan Ngo

机构 * The University of Melbourne（墨尔本大学）

专题命中推理评测：reasoning(abstract)；分类 cs.AI

2508.07646 2025-08-12 cs.LG 57%

Multi-Turn Jailbreaks Are Simpler Than They Seem

Xiaoxue Yang, Jaeha Lee, Anna-Katharina Dick, Jasper Timm, Fei Xie, Diogo Cruz

机构 * Imperial College London（伦敦帝国学院）； California Institute of Technology（加州理工学院）

专题命中推理评测：reasoning(abstract)；分类 cs.LG

Comments 25 pages, 15 figures. Accepted at COLM 2025 SoLaR Workshop

2508.07479 2025-08-12 cs.CL 57%

Positional Biases Shift as Inputs Approach Context Window Limits

Blerta Veseli, Julian Chibane, Mariya Toneva, Alexander Koller

机构 * Saarland Informatics Campus, Saarland University, Germany（萨尔兰大学信息学校区）； Max Planck Institute for Informatics, Saarland Informatics Campus, Germany（马克斯·普朗克信息研究所）； Max Planck Institute for Software Systems, Saarland Informatics Campus, Germany（马克斯·普朗克软件系统研究所）

专题命中推理评测：reasoning(abstract)；分类 cs.CL

Journal ref Conference on Language Modeling (COLM) 2025

2409.12962 2025-08-12 cs.CL cs.SD eess.AS 57%

CLAIR-A: Leveraging Large Language Models to Judge Audio Captions

Tsung-Han Wu, Joseph E. Gonzalez, Trevor Darrell, David M. Chan

机构 * Department of Electrical Engineering and Computer Science (EECS)（电气工程与计算机科学系）

专题命中推理评测：reasoning(abstract)；分类 cs.CL

Comments Accepted to ASRU 2025; Code is publicly available at https://github.com/DavidMChan/clair-a

2508.06851 2025-08-12 cs.AI cs.CY 57%

MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams

Pengfei Zhou, Xiaopeng Peng, Fanrui Zhang, Zhaopan Xu, Jiaxin Ai, Yansheng Qiu, Chuanhao Li, Zhen Li, Ming Li, Yukang Feng, Jianwen Sun, Haoquan Zhang, Zizhen Li, Xiaofeng Mao, Zekai Li, Wangbo Zhao, Kai Wang, Xiaojun Chang, Wenqi Shao, Yang You, Kaipeng Zhang

机构 * Shanghai AI Laboratory（上海人工智能实验室）； Shanghai Innovation Institute（上海创新研究院）； USTC（中国科学技术大学）； RIT（罗切斯特理工学院）； HIT（哈尔滨工业大学）； WHU（武汉大学）； MBZUAI（马克斯·普朗克人工智能研究所）； NUS（新加坡国立大学）

专题命中推理评测：reasoning(abstract)；分类 cs.AI

Comments 35 pages, 33 figures

2508.07918 2025-08-12 cs.CV 50%

RSVLM-QA: A Benchmark Dataset for Remote Sensing Vision Language Model-based Question Answering

Xing Zi, Jinghao Xiao, Yunxiao Shi, Xian Tao, Jun Li, Ali Braytee, Mukesh Prasad

机构 * School of Computer Science, University of Technology Sydney（技术悉尼大学计算机科学学院）； SEDE, University of Technology Sydney（技术悉尼大学SEDE）； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）

专题命中推理评测：reasoning(abstract)

Comments This paper has been accepted to the proceedings of the 33rd ACM International Multimedia Conference (ACM Multimedia 2025)

2508.07596 2025-08-12 cs.CV 50%

From Prediction to Explanation: Multimodal, Explainable, and Interactive Deepfake Detection Framework for Non-Expert Users

Shahroz Tariq, Simon S. Woo, Priyanka Singh, Irena Irmalasari, Saakshi Gupta, Dev Gupta

机构 * Sungkyunkwan University, S. Korea（顺天大学）； University of Queensland, Australia（昆士兰大学）

专题命中推理评测：reasoning(abstract)

Comments 11 pages, 3 tables, 5 figures, accepted for publicaiton in the 33rd ACM International Conference on Multimedia (MM '25), October 27-31, 2025, Dublin, Ireland

2508.07554 2025-08-12 cs.MM 50%

FineBadminton: A Multi-Level Dataset for Fine-Grained Badminton Video Understanding

Xusheng He, Wei Liu, Shanshan Ma, Qian Liu, Chenghao Ma, Jianlong Wu

专题命中推理评测：reasoning(abstract)

2508.07401 2025-08-12 cs.CV 50%

LET-US: Long Event-Text Understanding of Scenes

Rui Chen, Xingyu Chen, Shaoan Wang, Shihan Kong, Junzhi Yu

专题命中推理评测：reasoning(abstract)

2502.20758 2025-08-12 stat.AP cs.AI cs.CL 81%

Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truth

Seyed Pouyan Mousavi Davoudi, Amin Gholami Davodi, Alireza Amiri-Margavi, Alireza Shafiee Fard, Mahdi Jafari

机构 * Independent Researcher in AI and Statistics（人工智能与统计学独立研究者）； Shahrood University of Technology（沙霍罗德大学）； University of Pittsburgh（匹兹堡大学）； Duquesne University（杜克森大学）

专题命中其他推理：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 6pages

2508.07342 2025-08-12 cs.IR cs.CL 79%

PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization

Kepu Zhang, Teng Shi, Weijie Yu, Jun Xu

机构 * Gaoling School of Artificial Intelligence\ University of China Beijing China ； School of Information Technology ； Gaoling School of Artificial Intelligence\ University of China

专题命中其他推理：reasoning(title,abstract)；分类 cs.CL

2506.03616 2025-08-12 cs.CL 79%

Learning to Insert [PAUSE] Tokens for Better Reasoning

Eunki Kim, Sangryul Kim, James Thorne

机构 * KAIST AI（韩国科学技术院人工智能）

专题命中其他推理：reasoning(title,abstract)；分类 cs.CL

Comments 18 pages, 5 figures, ACL findings

Journal ref https://aclanthology.org/2025.findings-acl.1217/

2408.08651 2025-08-12 cs.CL cs.AI 73%

Chain of Thought Still Thinks Fast: APriCoT Helps with Thinking Slow

Kyle Moore, Jesse Roberts, Thao Pham, Douglas Fisher

机构 * Vanderbilt University（范德比大学）； Tennessee Technological University（田纳西技术大学）； Berea College（贝拉学院）

专题命中其他推理：reasoning(abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Comments Final version. Published In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 47) 2025

2508.07127 2025-08-12 cs.LG q-bio.GN 70%

How Effectively Can Large Language Models Connect SNP Variants and ECG Phenotypes for Cardiovascular Risk Prediction?

Niranjana Arun Menon, Iqra Farooq, Yulong Li, Sara Ahmed, Yutong Xie, Muhammad Awais, Imran Razzak

机构 * University of New South Wales（新南威尔士大学）； University of Surrey（萨里大学）； Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）

专题命中其他推理：reasoning(abstract)；CoT(abstract)；分类 cs.LG

2506.00826 2025-08-12 cs.CL cs.AI cs.LG 67%

HERGC: Heterogeneous Experts Representation and Generative Completion for Multimodal Knowledge Graphs

Yongkang Xiao, Rui Zhang

机构 * University of Minnesota（明尼苏达大学）

专题命中其他推理：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2508.07648 2025-08-12 cs.RO cs.AI cs.LG 62%

Grasp-HGN: Grasping the Unexpected

Mehrshad Zandigohar, Mallesham Dasari, Gunar Schirner

专题命中其他推理：reasoning(abstract)；分类 cs.AI、cs.LG

Comments Paper accepted at ACM Transactions on Embedded Computing Systems

2506.21931 2025-08-12 cs.IR cs.AI cs.CL cs.MA 62%

ARAG: Agentic Retrieval Augmented Generation for Personalized Recommendation

Reza Yousefi Maragheh, Pratheek Vadla, Priyank Gupta, Kai Zhao, Aysenur Inan, Kehui Yao, Jianpeng Xu, Praveen Kanumala, Jason Cho, Sushant Kumar

机构 * Walmart Global Tech（沃尔玛全球科技）

专题命中其他推理：reasoning(abstract)；分类 cs.CL、cs.AI

2508.07598 2025-08-12 cs.CL 57%

Keyword-Centric Prompting for One-Shot Event Detection with Self-Generated Rationale Enhancements

Ziheng Li, Zhi-Hong Deng

机构 * State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（一般人工智能国家重点实验室，智能科学与技术学院，北京大学）

专题命中其他推理：chain-of-thought(abstract)；分类 cs.CL

Comments ECAI 2025

2508.07223 2025-08-12 cs.IR cs.AI 57%

Selection and Exploitation of High-Quality Knowledge from Large Language Models for Recommendation

Guanchen Wang, Mingming Ha, Tianbao Ma, Linxun Chen, Zhaojie Liu, Guorui Zhou, Kun Gai

专题命中其他推理：reasoning(abstract)；分类 cs.AI

1. 推理评测 23 篇

Comparative Evaluation of ChatGPT and DeepSeek Across Key NLP Tasks: Strengths, Weaknesses, and Domain-Specific Performance

Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution

URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models

BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy

MultiMedEdit: A Scenario-Aware Benchmark for Evaluating Knowledge Editing in Medical VQA

VL-MedGuide: A Visual-Linguistic Large Model for Intelligent and Explainable Skin Disease Auxiliary Diagnosis

"Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas

Incorporating Contextual Paralinguistic Understanding in Large Speech-Language Models

Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks

DatasetResearch: Benchmarking Agent Systems for Demand-Driven Dataset Discovery

DoorDet: Semi-Automated Multi-Class Door Detection Dataset via Object Detection and Large Language Models

Multi-Turn Jailbreaks Are Simpler Than They Seem

Positional Biases Shift as Inputs Approach Context Window Limits

CLAIR-A: Leveraging Large Language Models to Judge Audio Captions

MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams

RSVLM-QA: A Benchmark Dataset for Remote Sensing Vision Language Model-based Question Answering

From Prediction to Explanation: Multimodal, Explainable, and Interactive Deepfake Detection Framework for Non-Expert Users

FineBadminton: A Multi-Level Dataset for Fine-Grained Badminton Video Understanding

LET-US: Long Event-Text Understanding of Scenes

2. 其他推理 13 篇

Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truth

PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization

Learning to Insert [PAUSE] Tokens for Better Reasoning

Chain of Thought Still Thinks Fast: APriCoT Helps with Thinking Slow

How Effectively Can Large Language Models Connect SNP Variants and ECG Phenotypes for Cardiovascular Risk Prediction?

HERGC: Heterogeneous Experts Representation and Generative Completion for Multimodal Knowledge Graphs

Grasp-HGN: Grasping the Unexpected

ARAG: Agentic Retrieval Augmented Generation for Personalized Recommendation

Keyword-Centric Prompting for One-Shot Event Detection with Self-Generated Rationale Enhancements

Selection and Exploitation of High-Quality Knowledge from Large Language Models for Recommendation