大模型推理能力

VerifyBench: A Systematic Benchmark for Evaluating Reasoning Verifiers Across Domains

Xuzhao Li, Xuchen Li, Shiyu Hu, Yongzhen Guo, Wentao Zhang

专题命中 推理评测 ：reasoning(title,abstract)；verifier(abstract)；分类 cs.AI

Comments Preprint, Under review

Agentar-Fin-R1: Enhancing Financial Intelligence through Domain Expertise, Training Efficiency, and Advanced Reasoning

Yanjun Zheng, Xiyang Du, Longfei Liao, Xiaoke Zhao, Zhaowen Zhou, Jingze Song, Bo Zhang, Jiawei Liu, Xiang Qi, Zhe Li, Zhiqiang Zhang, Wei Wang, Peng Zhang

机构 * Ant Group（蚂蚁集团） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discovery

Kun Li, Zhennan Wu, Shoupeng Wang, Jia Wu, Shirui Pan, Wenbin Hu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 29 pages, 8 figures, 2 tables

Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning

Shengyuan Wang, Jie Feng, Tianhui Liu, Dan Pei, Yong Li

机构 * College of AI, Tsinghua University（清华大学人工智能学院） ； Department of Electronic Engineering, BNRist, Tsinghua University（清华大学电子工程系） ； School of Electronic and Information Engineering, Beijing Jiaotong University（北京交通大学电子信息学院） ； Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 19 pages, 9 figures

Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition

Andy Zou, Maxwell Lin, Eliot Jones, Micha Nowak, Mateusz Dziemian, Nick Winter, Alexander Grattan, Valent Nathanael, Ayla Croft, Xander Davies, Jai Patel, Robert Kirk, Nate Burnikell, Yarin Gal, Dan Hendrycks, J. Zico Kolter, Matt Fredrikson

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

MMCircuitEval: A Comprehensive Multimodal Circuit-Focused Benchmark for Evaluating LLMs

Chenchen Zhao, Zhengyuan Shi, Xiangyu Wen, Chengjie Liu, Yi Liu, Yunhao Zhou, Yuxiang Zhao, Hefei Feng, Yinan Zhu, Gwok-Waa Wan, Xin Cheng, Weiyu Chen, Yongqi Fu, Chujie Chen, Chenhao Xue, Guangyu Sun, Ying Wang, Yibo Lin, Jun Yang, Ning Xu, Xi Wang, Qiang Xu

机构 * Department of Computer Science and Engineering, The Chinese University of Hong Kong（中国香港中文大学计算机科学与工程系） ； School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院） ； School of Integrated Circuits, Peking University（北京大学集成电路学院） ； School of Intergrated Circuits, Southeast University（东南大学集成电路学院） ； School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院） ； Department of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术系） ； National Center of Technology Innovation for EDA（EDA技术创新国家中心） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments 10 pages, 1 figure, 5 tables. To appear in ICCAD 2025

MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models

Hafsteinn Einarsson

机构 * University of Iceland（爱沙尼亚大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Trust the Model: Compact VLMs as In-Context Judges for Image-Text Data Quality

Daulet Toibazar, Kesen Wang, Sherif Mohamed, Abdulaziz Al-Badawi, Abdulrahman Alfulayt, Pedro J. Moreno

机构 * Humain Riyadh, KSA（利雅得人类，沙特阿拉伯） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text

Mizanur Rahman, Md Tahmid Rahman Laskar, Shafiq Joty, Enamul Hoque

机构 * York University（约克大学） ； Dialpad ； Salesforce AI Research（Salesforce人工智能研究） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Zero-shot Performance of Generative AI in Brazilian Portuguese Medical Exam

Cesar Augusto Madid Truyts, Amanda Gomes Rabelo, Gabriel Mesquita de Souza, Daniel Scaldaferri Lages, Adriano Jose Pereira, Uri Adrian Prync Flato, Eduardo Pontes dos Reis, Joaquim Edson Vieira, Paulo Sergio Panse Silveira, Edson Amaro Junior

机构 * Einstein Global Advanced Technologies for Equity（埃因斯坦全球先进科技以公平为宗旨） ； Hospital Israelita Albert Einstein（埃因斯坦医院） ； Departamento de Pacientes Graves（重症患者部门） ； Stanford Center for Artificial Intelligence in Medicine and Imaging（斯坦福大学医学与成像人工智能中心） ； Departmento de Cirurgia（外科部门） ； Faculdade de Medicina, Universidade de São Paulo（圣保罗大学医学院） ； Faculdade Israelita de Ciências da Saúde Albert Einstein（埃因斯坦以色列健康科学学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

TN-AutoRCA: Benchmark Construction and Agentic Framework for Self-Improving Alarm-Based Root Cause Analysis in Telecommunication Networks

Keyu Wu, Qianjin Yu, Manlin Mei, Ruiting Liu, Jun Wang, Kailai Zhang, Yelun Bao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 10 pages

Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models

Yifan Jiang, Yibo Xue, Yukun Kang, Pin Zheng, Jian Peng, Feiran Wu, Changliang Xu

机构 * Hangzhou Institute for Advanced Study（杭州先进研究所） ； University of Chinese Academy of Sciences（中国科学院大学） ； Alibaba Group（阿里巴巴集团） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Appendix at: https://github.com/PAMPAS-Lab/ANA-PPT-Anamation/blob/main/Appendix.pdf

Agent WARPP: Workflow Adherence via Runtime Parallel Personalization

Maria Emilia Mazzolenis, Ruirui Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Accepted at the ICML 2025 Workshop on Multi-Agent Systems in the Era of Foundation Models: Opportunities, Challenges, and Futures. Code repo: https://github.com/emiliamazzo/WARPP/

Object-centric Video Question Answering with Visual Grounding and Referring

Haochen Wang, Qirui Chen, Cilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, Weidi Xie, Stratis Gavves

机构 * University of Amsterdam（阿姆斯特丹大学） ； SAI, Shanghai Jiao Tong University（上海交通大学SAI研究所） ； Xiaohongshu Inc（小红书公司） 纠错

专题命中 推理评测 ：reasoning(abstract)