大模型推理能力

Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs

Chenqian Le, Ziheng Gong, Chihang Wang, Haowei Ni, Panfeng Li, Xupeng Chen

机构 * New York University（纽约大学） ； Columbia University（哥伦比亚大学） ； University of Michigan（密歇根大学） 纠错

专题命中 推理评测 ：CoT(title,abstract)；reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Accepted by 2025 International Conference on Artificial Intelligence, Human-Computer Interaction and Natural Language Processing

Journal ref Proceedings of the 2025 International Conference on Artificial Intelligence, Human-Computer Interaction and Natural Language Processing (ICAHN), 2025, pp. 43-46

DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry

Changti Wu, Shijie Lian, Zihao Liu, Lei Zhang, Laurence Tianruo Yang, Kai Chen

机构 * East China Normal University（华东师范大学） ； Zhongguancun Academy（中关村学院） ； Huazhong University of Science and Technology（华中科技大学） ； Peking University（北京大学） ； Zhengzhou University（郑州大学） ； Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments The code and dataset are available at \href{https://zgca-ai4edu.github.io/DynaSolidGeo/}{DynaSolidGeo}

Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages

Max Zuo, Francisco Piedrahita Velez, Xiaochen Li, Michael L. Littman, Stephen H. Bach

机构 * Brown University（布朗大学） 纠错

专题命中 推理评测 ：planning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NAACL Main Conference 2025

Journal ref Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)

STAR-1: Safer Alignment of Reasoning LLMs with 1K Data

Zijun Wang, Haoqin Tu, Yuhan Wang, Juncheng Wu, Yanqing Liu, Jieru Mei, Brian R. Bartoldson, Bhavya Kailkhura, Cihang Xie

机构 * UC Santa Cruz（加州大学圣克ruz分校） ； Google（谷歌） ； Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective

Songsong Yu, Yuxin Chen, Hao Ju, Lianjie Jia, Fuxi Zhang, Shaofei Huang, Yuhan Wu, Rundi Cui, Binghao Ran, Zaibin Zhang, Zhedong Zheng, Zhipeng Zhang, Yifan Wang, Lin Song, Lijun Wang, Yanwei Li, Ying Shan, Huchuan Lu

机构 * ARC Lab, Tencent PCG（腾讯PCG部门ARC实验室） ； Shanghai Jiao Tong University（上海交通大学） ； University of Macau（澳门大学） ； Dalian University of Technology（大连理工大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 推理评测 ：reasoning(abstract,comments)；planning(abstract)；分类 cs.AI

Comments a comprehensive visual spatial reasoning evaluation tool, 25 pages, 16 figures

SPEAR-MM: Selective Parameter Evaluation and Restoration via Model Merging for Efficient Financial LLM Adaptation

Berkcan Kapusuzoglu, Supriyo Chakraborty, Renkun Ni, Stephen Rawls, Sambit Sahu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents

Manasi Sharma, Chen Bo Calvin Zhang, Chaithanya Bandi, Clinton Wang, Ankit Aich, Huy Nghiem, Tahseen Rabbani, Ye Htet, Brian Jang, Sumana Basu, Aishwarya Balwani, Denis Peskoff, Marcos Ayestaran, Sean M. Hendryx, Brad Kenstler, Bing Liu

机构 * Scale AI ； University of Maryland（马里兰大学） ； University of Chicago（芝加哥大学） ； Washington University, St. Louis（圣路易斯华盛顿大学） ； McGill University（麦吉尔大学） ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages, 21 figures, pre-print

On the generalization of language models from in-context learning and finetuning: a controlled study

Andrew K. Lampinen, Arslan Chaudhry, Stephanie C. Y. Chan, Cody Wild, Diane Wan, Alex Ku, Jörg Bornschein, Razvan Pascanu, Murray Shanahan, James L. McClelland

机构 * Google DeepMind（谷歌DeepMind） ； Google DeepMind & Stanford University（谷歌DeepMind与斯坦福大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments FoRLM workshop, NeurIPS 2025

Towards Outcome-Oriented, Task-Agnostic Evaluation of AI Agents

Waseem AlShikh, Muayad Sayed Ali, Brian Kennedy, Dmytro Mozolevskyi

机构 * Writer, Inc.（Writer公司） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI

NOTAM-Evolve: A Knowledge-Guided Self-Evolving Optimization Framework with LLMs for NOTAM Interpretation

Maoqi Liu, Quan Fang, Yuhao Wu, Can Zhao, Yang Yang, Kaiquan Cai

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to AAAI 2026

A Multimodal Recaptioning Framework to Account for Perceptual Diversity Across Languages in Vision-Language Modeling

Kyle Buettner, Jacob T. Emmerson, Adriana Kovashka

机构 * Intelligent Systems Program（智能系统项目） ； Department of Computer Science（计算机科学系） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at IJCNLP-AACL 2025 (Main)

Beyond the Pixels: VLM-based Evaluation of Identity Preservation in Reference-Guided Synthesis

Aditi Singhania, Krutik Malani, Riddhi Dhawan, Arushi Jain, Garv Tandon, Nippun Sharma, Souymodip Chakraborty, Vineet Batra, Ankit Phogat

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Towards a Standard, Enterprise-Relevant Agentic AI Benchmark: Lessons from 5.5 billion tokens' worth of agentic AI evaluations

JV Roig

机构 * Kamiwaza AI 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 34 pages

Generalizable Insights for Graph Transformers in Theory and Practice

Timo Stoll, Luis Müller, Christopher Morris

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments Accepted at NeurIPS 2025 as spotlight

CellARC: Measuring Intelligence with Cellular Automata

Miroslav Lžičař

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments 22 pages, 11 figures. Working draft. Dataset and leaderboard available at https://cellarc.mireklzicar.com

Design, Results and Industry Implications of the World's First Insurance Large Language Model Evaluation Benchmark

Hua Zhou, Bing Ma, Yufei Zhang, Yi Zhao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 16 pages, 11 models,1 set of evaluation framework,5 core dimensions, 54 sub-indicators, 14,430 high-quality questions

3MDBench: Medical Multimodal Multi-agent Dialogue Benchmark

Ivan Sviridov, Amina Miftakhova, Artemiy Tereshchenko, Galina Zubkova, Pavel Blinov, Andrey Savchenko

机构 * Sber AI Lab（Sber AI实验室） ； HSE University（俄罗斯高等经济大学） ； ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院信息与系统问题研究所可信人工智能研究中心） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP 25 (main)

Journal ref https://aclanthology.org/2025.emnlp-main.1353/

VideoChain: A Transformer-Based Framework for Multi-hop Video Question Generation

Arpan Phukan, Anupam Pandey, Deepjyoti Bodo, Asif Ekbal

机构 * Department of Computer Science and Engineering, IIT Patna（计算机科学与工程系，印度理工学院帕纳） 纠错

专题命中 推理评测 ：reasoning(abstract)