大模型推理能力

WiS Platform: Enhancing Evaluation of LLM-Based Multi-Agent Systems Through Game-Based Analysis

Chengwei Hu, Jianhui Zheng, Yancheng He, Hangyu Guo, Junguang Jiang, Han Zhu, Kai Sun, Yuning Jiang, Wenbo Su, Bo Zheng

机构 * Taobao & Tmall Group of Alibaba（阿里巴巴淘宝及天猫集团） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Enterprise Large Language Model Evaluation Benchmark

Liya Wang, David Yi, Damien Jose, John Passarelli, James Gao, Jordan Leventis, Kang Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Submitted to MLNLP 2025 at https://csity2025.org/mlnlp/index

GlyphPattern: An Abstract Pattern Recognition Benchmark for Vision-Language Models

Zixuan Wu, Yoolim Kim, Carolyn Jane Anderson

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Journal ref Findings of the ACL 2025

MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration

Yucheng Zhou, Lingran Song, Jianbing Shen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ACL 2025 Findings

KnowMap: Efficient Knowledge-Driven Task Adaptation for LLMs

Kelin Fu, Kaigui Bian

机构 * School of Computer Science（计算机科学学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Spiritual-LLM : Gita Inspired Mental Health Therapy In the Era of LLMs

Janak Kapuriya, Aman Singh, Jainendra Shukla, Rajiv Ratn Shah

机构 * Data Science Institute University of Galway（都柏林大学数据科学研究所） ； MIDAS Lab IIIT Delhi（德里IIIT MIDAS实验室） ； HMI Lab IIIT Delhi（德里IIIT HMI实验室） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.AI

Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challenges

Zimo Ji, Daoyuan Wu, Wenyuan Jiang, Pingchuan Ma, Zongjie Li, Shuai Wang

机构 * Hong Kong University of Science and Technology（香港科技大学） ； D-INFK, ETH Zürich（ETH Zürich的D-INFK） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Research

Shuo Yan, Ruochen Li, Ziming Luo, Zimu Wang, Daoyang Li, Liqiang Jing, Kaiyu He, Peilin Wu, George Michalopoulos, Yue Zhang, Ziyang Zhang, Mian Zhang, Zhiyu Chen, Xinya Du

机构 * University of Texas at Dallas（德克萨斯大学达拉斯分校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

CORONA: A Coarse-to-Fine Framework for Graph-based Recommendation with Large Language Models

Junze Chen, Xinjie Yang, Cheng Yang, Junfei Bao, Zeyuan Guo, Yawen Li, Chuan Shi

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

CLEAR-3K: Assessing Causal Explanatory Capabilities in Language Models

Naiming Liu, Richard Baraniuk, Shashank Sonkar

机构 * Rice University（里士大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

SemAgent: A Semantics Aware Program Repair Agent

Anvith Pabba, Alex Mathai, Anindya Chakraborty, Baishakhi Ray

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

TARDIS STRIDE: A Spatio-Temporal Road Image Dataset and World Model for Autonomy

Héctor Carrión, Yutong Bai, Víctor A. Hernández Castro, Kishan Panaganti, Ayush Zenith, Matthew Trang, Tony Zhang, Pietro Perona, Jitendra Malik

机构 * Tera AI ； UC Santa Cruz（加州大学圣克ruz分校） ； UC Berkeley（加州大学伯克利分校） ； California Institute of Technology（加州理工学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Computer Vision, Pattern Recognition, Early-Fusion, Dataset, Data Augmentation

CDS: Knowledge Component-Driven Data Synthesis Guided by Cognitive Diagnosis Theory

Haokun Zhao, Jinyi Han, Jiaqing Liang, Yanghua Xiao, Xiaojun Meng, Jiansheng Wei

机构 * College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院） ； Shanghai Institute of Artificial Intelligence for Education, East China Normal University（华东师范大学教育人工智能研究所） ； School of Data Science, Fudan University（复旦大学数据科学学院） ； Huawei Noah’s Ark Lab（华为诺亚实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

sPhinX: Sample Efficient Multilingual Instruction Fine-Tuning Through N-shot Guided Prompting

Sanchit Ahuja, Kumar Tanmay, Hardik Hansrajbhai Chauhan, Barun Patra, Kriti Aggarwal, Luciano Del Corro, Arindam Mitra, Tejas Indulal Dhamecha, Ahmed Awadallah, Monojit Choudhary, Vishrav Chaudhary, Sunayana Sitaram

机构 * Microsoft Corporation（微软公司） ； Harvard University（哈佛大学） ； Adobe Research（Adobe研究） ； MBZUAI University（MBZUAI大学） ； Hippocratic AI ； Meta AI 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 20 pages, 12 tables, 5 figures

SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification

Chengye Wang, Yifei Shen, Zexi Kuang, Arman Cohan, Yilun Zhao

机构 * Yale NLP Lab（耶鲁大学自然语言处理实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Research on Graph-Retrieval Augmented Generation Based on Historical Text Knowledge Graphs

Yang Fan, Zhang Qi, Xing Wenqian, Liu Chang, Liu Liu

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

MinosEval: Distinguishing Factoid and Non-Factoid for Tailored Open-Ended QA Evaluation with LLMs

Yongqi Fan, Yating Wang, Guandong Wang, Jie Zhai, Jingping Liu, Qi Ye, Tong Ruan

机构 * School of Information Science and Engineering, East China University of Science and Technology（信息科学与工程学院，东华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

PEDANTIC: A Dataset for the Automatic Examination of Definiteness in Patent Claims

Valentin Knappich, Annemarie Friedrich, Anna Hätty, Simon Razniewski

机构 * Bosch Center for AI（博世人工智能中心） ； University of Augsburg（奥斯堡大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments PatentSemTech@SIGIR2025

Root Defence Strategies: Ensuring Safety of LLM at the Decoding Level

Xinyi Zeng, Yuying Shang, Jiawei Chen, Jingyuan Zhang, Yu Tian

机构 * Aerospace Information Research Institute, Chinese Academy of Sciences（航天信息研究所，中国科学院） ； Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua University（计算机科学与技术系，人工智能研究院，清华大学） ； Shanghai Key Laboratory of Multi. Info. Processing, East China Normal University（上海多信息处理重点实验室，华东师范大学） ； Kuaishou-Inc（快手公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 19 pages, 9 figures

Explainable Detection of Implicit Influential Patterns in Conversations via Data Augmentation

Sina Abdidizaji, Md Kowsher, Niloofar Yousefi, Ivan Garibay

机构 * Industrial Engineering and Management Systems, University of Central Florida（工业工程与管理系统，中央佛罗里达大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted at the HCI International conference 2025

Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text

Amr Mohamed, Yang Zhang, Michalis Vazirgiannis, Guokan Shang

机构 * MBZUAI ； Ecole Polytechnique（巴黎高等理工学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

OrgAccess: A Benchmark for Role Based Access Control in Organization Scale LLMs

Debdeep Sanyal, Umakanta Maharana, Yash Sinha, Hong Ming Tan, Shirish Karande, Mohan Kankanhalli, Murari Mandal

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 56 Pages

Uncovering Overfitting in Large Language Model Editing

Mengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren, Shu Wu, Zhumin Chen

机构 * Shandong University（山东大学） ； School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院） ； New Laboratory of Pattern Recognition (NLPR) State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ICLR 2025

PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization

Meiling Tao, Chenghao Zhu, Dongyi Ding, Tiannan Wang, Yuchen Eleanor Jiang, Wangchunshu Zhou

机构 * University of Electronic Science and Technology of China（电子科技大学） ； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； South China Agricultural University（华南农业大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Work in progress

Team Anotheroption at SemEval-2025 Task 8: Bridging the Gap Between Open-Source and Proprietary LLMs in Table QA

Nikolas Evkarpidi, Elena Tutubalina

机构 * HSE University（俄罗斯伏尔加格勒国立大学） ； AIRI Sber AI Kazan Federal University（卡兹别克联邦大学） 纠错

专题命中 推理评测 ：self-correction(abstract)；分类 cs.CL

Comments Accepted for publication at the 19th International Workshop on Semantic Evaluation (SemEval-2025), to be held in conjunction with ACL 2025. 15 pages, 5 figures; full paper title was added

AnalogXpert: Automating Analog Topology Synthesis by Incorporating Circuit Design Expertise into Large Language Models

Haoyi Zhang, Shizhao Sun, Yibo Lin, Runsheng Wang, Jiang Bian

机构 * School of Integrated Circuits, Peking University（集成电路学院，北京大学） ； Institute of EDA, Peking University（EDA研究院，北京大学） ； Beijing Advanced Innovation Center for Integrated Circuits（北京集成电路先进创新中心） ； Microsoft Research Asia（微软亚洲研究院） 纠错

专题命中 推理评测 ：CoT(abstract)；分类 cs.AI

AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security

Zikui Cai, Shayan Shabihi, Bang An, Zora Che, Brian R. Bartoldson, Bhavya Kailkhura, Tom Goldstein, Furong Huang

机构 * University of Maryland College Park（马里兰大学学院公园分校） ； Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室） ； Capital One 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments ICLR 2025 Workshop BuildingTrust

Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation

Xintong Wang, Jingheng Pan, Yixiao Liu, Xiaohu Zhao, Chenyang Lyu, Minghao Wu, Chris Biemann, Longyue Wang, Linlong Xu, Weihua Luo, Kaifu Zhang

机构 * Alibaba International Digital Commerce Group（阿里巴巴国际数字商务集团） ； Universität Hamburg, Department of Informatics（汉堡大学信息学院） ； Nanyang Technological University（南洋理工大学） ； Monash University（莫纳什大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

MANBench: Is Your Multimodal Model Smarter than Human?

Han Zhou, Qitong Xu, Yiheng Dong, Xin Yang

机构 * School of Electronic Information and Communications, Huazhong University of Science and Technology（电子信息与通讯学院，华中科技大学） ； School of Basic Medicine, Huazhong University of Science and Technology（基础医学院，华中科技大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Multimodal Benchmark, Project Url: https://github.com/micdz/MANBench, ACL2025 Findings

NSW-EPNews: A News-Augmented Benchmark for Electricity Price Forecasting with LLMs

Zhaoge Bi, Linghan Huang, Haolin Jin, Qingwen Zeng, Huaming Chen

机构 * The University of Sydney（悉尼大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments 9 pages' main texts. Submitted to NeurIPS 2025 Datasets and Benchmarks Track