大模型推理能力

Agentar-DeepFinance-100K: A Large-Scale Financial Dataset via Systematic Chain-of-Thought Synthesis Optimization

Xiaoke Zhao, Zhaowen Zhou, Lin Chen, Lihong Wang, Zhiyi Huang, Kaiyuan Zheng, Yanjun Zheng, Xiyang Du, Longfei Liao, Jiawei Liu, Xiang Qi, Bo Zhang, Peng Zhang, Wei Wang, Zhe Li

专题命中 推理评测 ：chain-of-thought(title,abstract)；reasoning(abstract)；CoT(abstract)

ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness

Yijun Liang, Ming Li, Chenrui Fan, Ziyue Li, Dang Nguyen, Kwesi Cobbina, Shweta Bhardwaj, Jiuhai Chen, Fuxiao Liu, Tianyi Zhou

机构 * University of Maryland, College Park（马里兰大学学院公园分校） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by NeurIPS2025. 36 pages, including references and appendix. Code is available at https://github.com/tianyi-lab/ColorBench

FedCoT: Federated Chain-of-Thought Distillation for Large Language Models

Tao Fan, Weijing Chen, Yan Kang, Guoqiang Ma, Hanlin Gu, Yuanfeng Song, Lixin Fan, Qiang Yang

机构 * Hong Kong University of Science and Technology（香港科学与技术大学） ； WeBank ； Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 推理评测 ：chain-of-thought(title,abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond

Yinghao Hu, Yaoyao Yu, Leilei Gan, Bin Wei, Kun Kuang, Fei Wu

机构 * College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院） ； Guanghua Law School, Zhejiang University（浙江大学光华法学院） ； School of Software Technology, Zhejiang University（浙江大学软件学院） ； Law & AI Lab, Zhejiang University（浙江大学法律与人工智能实验室） ； Shanghai AI Laboratory（上海人工智能实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 23 pages, Published in Findings of the Association for Computational Linguistics: EMNLP 2025

How Well Do LLMs Understand Drug Mechanisms? A Knowledge + Reasoning Evaluation Dataset

Sunil Mohan, Theofanis Karaletsos

机构 * CZI 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments An earlier version of this paper appears in IEEE FLLM 2025. GitHub: https://github.com/czi-ai/DrugMechCounterfactuals

An Empirical Study of Reasoning Steps in Thinking Code LLMs

Haoran Xue, Gias Uddin, Song Wang

机构 * York University（约克大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

NOVO: Bridging LLaVA and SAM with Visual-only Prompts for Reasoning Segmentation

Kyung-Yoon Yoon, Yeong-Jun Cho

机构 * Department of Artificial Intelligence Convergence（人工智能融合系） 纠错

专题命中 推理评测 ：reasoning(title,abstract)

How Reasoning Influences Intersectional Biases in Vision Language Models

Adit Desai, Sudipta Roy, Mohna Chakraborty

机构 * Student（学生） 纠错

专题命中 推理评测 ：reasoning(title,abstract)

Towards Competent AI for Fundamental Analysis in Finance: A Benchmark Dataset and Evaluation

Zonghan Wu, Congyuan Zou, Junlin Wang, Chenhan Wang, Hangjing Yang, Yilei Shao

机构 * Shanghai AI Finance School, East China Normal University（上海人工智能金融学院，华东师范大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.AI

DrKGC: Dynamic Subgraph Retrieval-Augmented LLMs for Knowledge Graph Completion across General and Biomedical Domains

Yongkang Xiao, Sinian Zhang, Yi Dai, Huixue Zhou, Jue Hou, Jie Ding, Rui Zhang

机构 * University of Minnesota（明尼苏达大学） ； University of Michigan（密歇根大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at EMNLP 2025 Findings

LLM-Driven Completeness and Consistency Evaluation for Cultural Heritage Data Augmentation in Cross-Modal Retrieval

Jian Zhang, Junyi Guo, Junyi Yuan, Huanda Lu, Yanlin Zhou, Fangyu Wu, Qiufeng Wang, Dongming Lu

机构 * Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学） ； NingboTech University（宁波科技学院） ； Dunhuang Academy（敦煌研究院） ； Zhejiang University（浙江大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)

When Language Shapes Thought: Cross-Lingual Transfer of Factual Knowledge in Question Answering

Eojin Kang, Juae Kim

机构 * Hankuk University of Foreign Studies（韩国外国语大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at CIKM2025 (Expanded version)

Evaluation of retrieval-based QA on QUEST-LOFT

Nathan Scales, Nathanael Schärli, Olivier Bousquet

机构 * Google DeepMind（谷歌DeepMind） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Latent Traits and Cross-Task Transfer: Deconstructing Dataset Interactions in LLM Fine-tuning

Shambhavi Krishna, Atharva Naik, Chaitali Agarwal, Sudharshan Govindan, Taesung Lee, Haw-Shiuan Chang

机构 * University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） ； Anthropic 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Proceedings of the 14th Joint Conference on Lexical and Computational Semantics (*SEM 2025)

Benchmarking Large Language Models with Integer Sequence Generation Tasks

Daniel O'Malley, Manish Bhattarai, Nishath Rajiv Ranasinghe, Erick Draayer, Javier Santos

机构 * Earth and Environmental Sciences Division Los Alamos National Laboratory（地球与环境科学 division 洛斯阿拉莫斯国家实验室） ； Theoretical Division Los Alamos National Laboratory（理论 division 洛斯阿拉莫斯国家实验室） ； Computational Sciences Division Los Alamos National Laboratory（计算科学 division 洛斯阿拉莫斯国家实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Evaluating the Ability of Large Language Models to Reason about Cardinal Directions, Revisited

Anthony G Cohn, Robert E Blackwell

机构 * School of Computer Science, University of Leeds, UK（利兹大学计算机科学学院） ； Alan Turing Institute, London, UK（艾伦·图灵研究所） 纠错

专题命中 推理评测 ：reasoning(abstract,comments)；分类 cs.CL

Comments 8 pages, 5 figures. Accepted at QR 2025 : 38th International Workshop on Qualitative Reasoning at IJCAI. arXiv admin note: substantial text overlap with arXiv:2406.16528

EMODIS: A Benchmark for Context-Dependent Emoji Disambiguation in Large Language Models

Jiacheng Huang, Ning Yu, Xiaoyin Yi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by AAAI2026

EduGuardBench: A Holistic Benchmark for Evaluating the Pedagogical Fidelity and Adversarial Safety of LLMs as Simulated Teachers

Yilin Jiang, Mingzi Zhang, Xuanyu Yin, Sheng Jin, Suyu Lu, Zuocan Ying, Zengyi Yu, Xiangjie Kong

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 22 pages, 9 figures, accepted by AAAI2026 as oral paper

Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights

Hyunjae Kim, Jiwoong Sohn, Aidan Gilson, Nicholas Cochran-Caggiano, Serina Applebaum, Heeju Jin, Seihee Park, Yujin Park, Jiyeong Park, Seoyoung Choi, Brittany Alexandra Herrera Contreras, Thomas Huang, Jaehoon Yun, Ethan F. Wei, Roy Jiang, Leah Colucci, Eric Lai, Amisha Dave, Tuo Guo, Maxwell B. Singer, Yonghoe Koo, Ron A. Adelman, James Zou, Andrew Taylor, Arman Cohan, Hua Xu, Qingyu Chen

机构 * Yale School of Medicine（耶鲁医学院） ； Yale University（耶鲁大学） ； ETH Zurich（苏黎世联邦理工学院） ； Harvard Medical School（哈佛医学院） ； Geisel School of Medicine at Dartmouth（达特茅斯大学盖塞医学院） ； Seoul National University College of Medicine（首尔国立大学医学院） ； Hanyang University College of Medicine（翰林大学医学院） ； PA Leadership Charter School（宾夕法尼亚州西切斯特市领导学校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 34 pages, 6 figures

SDS KoPub VDR: A Benchmark Dataset for Visual Document Retrieval in Korean Public Documents

Jaehoon Lee, Sohyun Kim, Wanggeun Park, Geon Lee, Seungkyung Kim, Minyoung Lee

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 27 pages, 15 figures, 6 tables

GUI-360$^\circ$: A Comprehensive Dataset and Benchmark for Computer-Using Agents

Jian Mu, Chaoyun Zhang, Chiming Ni, Lu Wang, Bo Qiao, Kartik Mathur, Qianhui Wu, Yuhang Xie, Xiaojun Ma, Mengyu Zhou, Si Qin, Liqun Li, Yu Kang, Minghua Ma, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

机构 * Nanjing University（南京大学） ； Microsoft（微软） ； ZJU-UIUC（浙大-UIUC） ； Peking University（北京大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

LegalEval-Q: A New Benchmark for The Quality Evaluation of LLM-Generated Legal Text

Li yunhan, Wu gengshen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 10 pages, 11 figures

Mufu: Multilingual Fused Learning for Low-Resource Translation with LLM

Zheng Wei Lim, Nitish Gupta, Honglin Yu, Trevor Cohn

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 29 pages

10 Open Challenges Steering the Future of Vision-Language-Action Models

Soujanya Poria, Navonil Majumder, Chia-Yu Hung, Amir Ali Bagherzadeh, Chuan Li, Kenneth Kwok, Ziwei Wang, Cheston Tan, Jiajun Wu, David Hsu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments AAAI 2026 (Senior Track)

Catching Contamination Before Generation: Spectral Kill Switches for Agents

Valentin Noël

机构 * Devoteam 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments Preprint under review (2025). 9 pages, 2 figures. Code and scripts: to be released

In-Context Adaptation of VLMs for Few-Shot Cell Detection in Optical Microscopy

Shreyan Ganguly, Angona Biswas, Jaydeep Rade, Md Hasibul Hasan Hasib, Nabila Masud, Nitish Singla, Abhipsa Dash, Ushashi Bhattacharjee, Aditya Balu, Anwesha Sarkar, Adarsh Krishnamurthy, Soumik Sarkar

机构 * Iowa State University（爱荷华州立大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

DOCUEVAL: An LLM-based AI Engineering Tool for Building Customisable Document Evaluation Workflows

Hao Zhang, Qinghua Lu, Liming Zhu

机构 * CSIRO’s Data61（CSIRO数据61） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Evaluating LLM-based Workflows for Switched-Mode Power Supply Design

Simon Nau, Jan Krummenauer, André Zimmermann

机构 * Robert Bosch GmbH, Cross-Domain Computing Solutions（罗伯特·博世有限公司，跨领域计算解决方案） ； University of Stuttgart, Institute for Micro Integration (IFM)（斯图加特大学，微系统集成研究所） ； Hahn-Schickard, Stuttgart, Germany（哈恩-施克尔德研究所，斯图加特，德国） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answering

Jihao Zhao, Chunlai Zhou, Daixuan Li, Shuaishuai Zu, Biao Qin

机构 * School of Information, Renmin University of China（信息学院，中国人民大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

LIVS: A Pluralistic Alignment Dataset for Inclusive Public Spaces

Rashid Mushkani, Shravan Nayak, Hugo Berard, Allison Cohen, Shin Koseki, Hadrien Bertrand

机构 * Mila–Quebec AI Institute（魁北克AI研究所） 纠错

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Comments ICML 2025