大模型推理能力

GeoAnalystBench: A GeoAI benchmark for assessing large language models for spatial analysis workflow and code generation

Qianheng Zhang, Song Gao, Chen Wei, Yibo Zhao, Ying Nie, Ziru Chen, Shijie Chen, Yu Su, Huan Sun

机构 * University of Wisconsin-Madison（威斯康星大学麦迪逊分校） ； The Ohio State University（俄亥根州立大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 34 pages, 8 figures

Journal ref Transactions in GIS, 2025

Personality as a Probe for LLM Evaluation: Method Trade-offs and Downstream Effects

Gunmay Handa, Zekun Wu, Adriano Koshiyama, Philip Treleaven

机构 * University College London（伦敦大学学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?

Guibin Zhang, Junhao Wang, Junjie Chen, Wangchunshu Zhou, Kun Wang, Shuicheng Yan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala

Ashmari Pramodya, Nirasha Nelki, Heshan Shalinda, Chamila Liyanage, Yusuke Sakai, Randil Pushpananda, Ruvan Weerasinghe, Hidetaka Kamigaito, Taro Watanabe

机构 * Nara Institute of Science and Technology (NAIST)（奈良科学技术研究所） ； University of Colombo School of Computing (UCSC)（科伦坡大学计算学院） ； Informatics Institute of Technology (IIT)（信息技术研究所） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 19 pages, 11 figures

Are We SOLID Yet? An Empirical Study on Prompting LLMs to Detect Design Principle Violations

Fatih Pehlivan, Arçin Ülkü Ergüzen, Sahand Moslemi Yengejeh, Mayasah Lami, Anil Koyuncu

机构 * Bilkent University, Turkey（巴伊肯大学） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.AI

Comments Accepted to ASE2025

Advancing Dialectal Arabic to Modern Standard Arabic Machine Translation

Abdullah Alabdullah, Lifeng Han, Chenghua Lin

机构 * University of Manchester（曼彻斯特大学） ； School of Informatics, University of Edinburgh（爱丁堡大学信息学院） ； Leiden University（莱顿大学） ； LIACS ； LUMC 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

Mitigating Catastrophic Forgetting in Continual Learning through Model Growth

Ege Süalp, Mina Rezaei

机构 * Department of Statistics Ludwig-Maximilian University（统计系路德维希-马克西米利安大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Point, Detect, Count: Multi-Task Medical Image Understanding with Instruction-Tuned Vision-Language Models

Sushant Gautam, Michael A. Riegler, Pål Halvorsen

机构 * Simula Metropolitan Center for Digital Engineering (SimulaMet), Norway（Simula数字工程中心（SimulaMet）） ； Oslo Metropolitan University (OsloMet), Norway（奥斯陆 Metropolitan 大学（OsloMet）） ； Simula Research Laboratory, Norway（Simula研究实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Accepted as a full paper at the 38th IEEE International Symposium on Computer-Based Medical Systems (CBMS) 2025

FactGenius: Combining Zero-Shot Prompting and Fuzzy Relation Mining to Improve Fact Verification with Knowledge Graphs

Sushant Gautam

机构 * Simula Metropolitan Center for Digital Engineering（模拟元数字工程中心） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments accepted and presented at the 6th IN5550 Workshop on Neural Natural Language Processing (WNNLP 2024) at the University of Oslo, Norway

Why Not Transform Chat Large Language Models to Non-English?

Xiang Geng, Ming Zhu, Jiahuan Li, Zhejian Lai, Wei Zou, Shuaijie She, Jiaxin Guo, Xiaofeng Zhao, Yinglu Li, Yuang Li, Chang Su, Yanqing Zhao, Xinglin Lyu, Min Zhang, Jiajun Chen, Hao Yang, Shujian Huang

机构 * National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China（新型软件技术国家重点实验室，南京大学，南京） ； Translation Services Center, Huawei Inc., Beijing 100085, China（翻译服务部，华为公司，北京） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

Comments The article has been accepted by Frontiers of Computer Science (FCS), with the DOI: {10.1007/s11704-025-50646-z}

VideoRewardBench: Comprehensive Evaluation of Multimodal Reward Models for Video Understanding

Zhihong Zhang, Xiaojian Huang, Jin Xu, Zhuodong Luo, Xinzhi Wang, Jiansheng Wei, Xuejin Chen

机构 * University of Science and Technology of China（中国科学技术大学） ； Huawei Noah’s Ark Lab（华为诺亚实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments https://videorewardbench.github.io/

Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Models

Sihao Wu, Gaojie Jin, Wei Huang, Jianhong Wang, Xiaowei Huang

机构 * University of Liverpool（利物浦大学） ； University of Exeter（埃克塞特大学） ； Purple Mountain Laboratories（紫金山实验室） ； University of Bristol（布里斯托大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulation

Enyu Zhao, Vedant Raval, Hejia Zhang, Jiageng Mao, Zeyu Shangguan, Stefanos Nikolaidis, Yue Wang, Daniel Seita

机构 * Department of Computer Science, University of Southern California（计算机科学系，南加州大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Conference on Robot Learning (CoRL) 2025. 50 pages and 30 figures. v2 is the camera-ready and includes a few more new experiments compared to v1

Automated Clinical Problem Detection from SOAP Notes using a Collaborative Multi-Agent LLM Architecture

Yeawon Lee, Xiaoyang Wang, Christopher C. Yang

机构 * Drexel University（德雷塞尔大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Accepted to The 16th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB 2025)(Poster Paper)

CE-RS-SBCIT A Novel Channel Enhanced Hybrid CNN Transformer with Residual, Spatial, and Boundary-Aware Learning for Brain Tumor MRI Analysis

Mirza Mumtaz Zahoor, Saddam Hussain Khan

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Comments 37 Pages, 12 Figures

MSRS: Evaluating Multi-Source Retrieval-Augmented Generation

Rohan Phanse, Yijie Zhou, Kejian Shi, Wencai Zhang, Yixin Liu, Yilun Zhao, Arman Cohan

机构 * Yale University（耶鲁大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments COLM 2025; this article supersedes the preprint: arXiv:2309.08960

WideSearch: Benchmarking Agentic Broad Info-Seeking

Ryan Wong, Jiawei Wang, Junjie Zhao, Li Chen, Yan Gao, Long Zhang, Xuan Zhou, Zuo Wang, Kai Xiang, Ge Zhang, Wenhao Huang, Yang Wang, Ke Wang

机构 * ByteDance Seed（字节跳动种子） 纠错

专题命中 推理评测 ：planning(abstract)；分类 cs.CL

CAMB: A comprehensive industrial LLM benchmark on civil aviation maintenance

Feng Zhang, Chengjie Pang, Yuehan Zhang, Chenyu Luo

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Tracking World States with Language Models: State-Based Evaluation Using Chess

Romain Harang, Jason Naradowsky, Yaswitha Gujju, Yusuke Miyao

机构 * University of Tokyo（东京大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Spotlight presentation at ICML 2025 Workshop on Assessing World Models

Step-Audio 2 Technical Report

Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, Mingrui Chen, Peng Liu, Wang You, Xiangyu Tony Zhang, Xingyuan Li, Xuerui Yang, Yayue Deng, Yechang Huang, Yuxin Li, Yuxin Zhang, Zhao You, Brian Li, Changyi Wan, Hanpeng Hu, Jiangjie Zhen, Siyu Chen, Song Yuan, Xuelin Zhang, Yimin Jiang, Yu Zhou, Yuxiang Yang, Bingxin Li, Buyun Ma, Changhe Song, Dongqing Pang, Guoqiang Hu, Haiyang Sun, Kang An, Na Wang, Shuli Gao, Wei Ji, Wen Li, Wen Sun, Xuan Wen, Yong Ren, Yuankai Ma, Yufan Lu, Bin Wang, Bo Li, Changxin Miao, Che Liu, Chen Xu, Dapeng Shi, Dingyuan Hu, Donghang Wu, Enle Liu, Guanzhe Huang, Gulin Yan, Han Zhang, Hao Nie, Haonan Jia, Hongyu Zhou, Jianjian Sun, Jiaoren Wu, Jie Wu, Jie Yang, Jin Yang, Junzhe Lin, Kaixiang Li, Lei Yang, Liying Shi, Li Zhou, Longlong Gu, Ming Li, Mingliang Li, Mingxiao Li, Nan Wu, Qi Han, Qinyuan Tan, Shaoliang Pang, Shengjie Fan, Siqi Liu, Tiancheng Cao, Wanying Lu, Wenqing He, Wuxun Xie, Xu Zhao, Xueqi Li, Yanbo Yu, Yang Yang, Yi Liu, Yifan Lu, Yilei Wang, Yuanhao Ding, Yuanwei Liang, Yuanwei Lu, Yuchu Luo, Yuhe Yin, Yumeng Zhan, Yuxiang Zhang, Zidong Yang, Zixin Zhang, Binxing Jiao, Daxin Jiang, Heung-Yeung Shum, Jiansheng Chen, Jing Li, Xiangyu Zhang, Yibo Zhu

机构 * StepFun Audio Team（StepFun音频团队） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments v3: Added introduction and evaluation results of Step-Audio 2 mini

Efficient Response Generation Strategy Selection for Fine-Tuning Large Language Models Through Self-Aligned Perplexity

Xuan Ren, Qi Chen, Lingqiao Liu

机构 * University of Adelaide（阿德莱德大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

NPHardEval4V: Dynamic Evaluation of Large Vision-Language Models with Effects of Vision

Xiang Li, Wenyue Hua, Kaijie Zhu, Lingyao Li, Haoyang Ling, Jinkui Chi, Qi Dou, Jindong Wang, Yongfeng Zhang, Xin Ma, Lizhou Fan

机构 * Shandong University, School of Control Science ； Rutgers University, Department of Computer Science 110 Frelinghuysen Road Piscataway New Brunswick NJ USA 08854 ； University of California, Santa Barbara 1210 Cheadle Hall Santa Barbara California USA 93106 ； University of Michigan, School of Information 500 S State St Ann Arbor Michigan USA 48109 ； The Chinese University of Hong Kong, Department of Computer Science ； The College of William \& Mary, School of Computing, Data Sciences \& Physics 200 Stadium Dr Williamsburg Virginia USA 23185 ； The Chinese University of Hong Kong, Department of Psychiatry 9 Chuen On Rd Tai Po New Territories Hong Kong SAR, China 999077 ； Rutgers University, Department of Computer Science ； University of California, Santa Barbara ； University of Michigan, School of Information ； The College of William \& Mary, School of Computing, Data Sciences \& Physics ； The Chinese University of Hong Kong, Department of Psychiatry 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 25 pages, 9 figures, 2 tables

Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning

Songtao Jiang, Yuxi Chen, Sibo Song, Yan Zhang, Yeying Jin, Yang Feng, Jian Wu, Zuozhu Liu

机构 * Zhejiang University, Zhejiang, China（浙江大学） ； Alibaba Group, Zhejiang, China（阿里巴巴集团） ； Angelalign Technology Inc., Shanghai, China（Angelalign技术有限公司） ； ChohoTech Inc., Hangzhou, China（楚合科技有限公司） ； Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence, Zhejiang, China（浙江省医学影像人工智能重点实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

CLARIFY: A Specialist-Generalist Framework for Accurate and Lightweight Dermatological Visual Question Answering

Aranya Saha, Tanvir Ahmed Khan, Ismam Nur Swapnil, Mohammad Ariful Haque

机构 * Aranya Saha ∗ , Tanvir Ahmed Khan ∗ , Ismam Nur Swapnil ∗ , and Mohammad Ariful Haque ∗（无明确机构） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 10 pages, 8 figures, Prepared for submission to IEEE Transactions on Human-Machine Systems

VAGUE: Visual Contexts Clarify Ambiguous Expressions

Heejeong Nam, Jinwoo Ahn, Keummin Ka, Jiwan Chung, Youngjae Yu

机构 * Brown University（布朗大学） ； UC Berkeley（加州大学伯克利分校） ； Yonsei University（延世大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ICCV 2025, 32 pages

Integrating Large Language Model for Improved Causal Discovery

Taiyu Ban, Lyuzhou Chen, Derui Lyu, Xiangyu Wang, Qinrui Zhu, Qiang Tu, Huanhuan Chen

机构 * School of Computer Science and Technology, University of Science and Technology of China（计算机科学与技术学院，中国科学技术大学） ； University of Science and Technology of China（中国科学技术大学） ； Laboratory for Big Data and Decision, University of Science and Technology of China（大数据与决策实验室，中国科学技术大学） ； First Affiliated Hospital of University of Science and Technology of China（中国科学技术大学第一附属医院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 13 pages, 5 figures

Journal ref IEEE Transactions on Artificial Intelligence, 2025

SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Models

Zhenwei Tang, Difan Jiao, Blair Yang, Ashton Anderson

机构 * Department of Computer Science, University of Toronto（多伦多大学计算机科学系） ； Coolwei AI Lab（Coolwei人工智能实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments COLM 2025

PerPilot: Personalizing VLM-based Mobile Agents via Memory and Exploration

Xin Wang, Zhiyao Cui, Hao Li, Ya Zeng, Chenxu Wang, Ruiqi Song, Yihang Chen, Kun Shao, Qiaosheng Zhang, Jinzhuo Liu, Siyue Ren, Shuyue Hu, Zhen Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

EMPOWER: Evolutionary Medical Prompt Optimization With Reinforcement Learning

Yinda Chen, Yangfan He, Jing Yang, Dapeng Zhang, Zhenlong Yuan, Muhammad Attique Khan, Jamel Baili, Por Lip Yee

机构 * MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China（脑启发智能感知与认知国家重点实验室，中国科学技术大学） ； Department of Computer Science, University of Minnesota-Twin Cities（计算机科学系，明尼苏达大学双城分校） ； Center of Research for Cyber Security and Network (CSNET), Faculty of Computer Science and Information Technology, Universiti Malaya（网络安全与网络研究中心（CSNET），马来亚大学计算机科学与信息技术学院） ； DSLAB, School of Information Science & Engineering, Lanzhou University（信息科学与工程学院，兰州大学） ； Institute of Computing Technology, Chinese Academy of Sciences（计算技术研究所，中国科学院） ； Department of AI, Prince Mohammad bin Fahd University（人工智能系，普林姆·法赫德大学） ； Department of Computer Engineering, College of Computer Science, King Khalid University（计算机工程系，国王·卡利德大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

OmniMRI: A Unified Vision--Language Foundation Model for Generalist MRI Interpretation

Xingxin He, Aurora Rofena, Ruimin Feng, Haozhe Liao, Zhaoye Zhou, Albert Jang, Fang Liu

机构 * Athinoula A. Martinos Center for Biomedical Imaging（阿提诺拉A.马丁诺斯生物医学成像中心） ； Harvard Medical School（哈佛医学院） ； Massachusetts General Hospital（麻省总医院） ； University Campus Bio-Medico of Rome（罗马生物医学大学校园） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI