大模型推理能力

SkinCaRe: A Multimodal Dermatology Dataset Annotated with Medical Caption and Chain-of-Thought Reasoning

Yuhao Shen, Liyuan Sun, Yan Xu, Wenbin Liu, Shuping Zhang, Shawn Afvari, Zhongyi Han, Jiaoyan Song, Yongzhi Ji, Tao Lu, Xiaonan He, Xin Gao, Juexiao Zhou

机构 * School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK–Shenzhen)（数据科学学院，香港中文大学（深圳）） ； Computer Science Program, CEMSE Division, King Abdullah University of Science and Technology (KAUST)（计算机科学项目，科学与工程学院，国王 Abdullah 科学技术大学） ； Center of Excellence on Smart Health, KAUST（智能健康卓越中心，国王 Abdullah 科学技术大学） ； Center of Excellence for Generative AI, KAUST（生成式人工智能卓越中心，国王 Abdullah 科学技术大学） ； Department of Dermatology, Beijing AnZhen Hospital, Capital Medical University（皮肤科，北京安贞医院，首都医科大学） ； Department of Dermatology, Tianjin Institute of Integrative Dermatology, Tianjin Academy of Traditional Chinese Medicine Affiliated Hospital（皮肤科，天津整合皮肤科研究院，天津中医药大学附属医院） ； Department of Dermatology, Beijing Aerospace General Hospital（皮肤科，北京航天总医院） ； Department of Dermatology, The First Affiliated Hospital, Shantou University Medical College（皮肤科，汕头大学医学院第一附属医院） ； DermAssure, LLC（DermAssure 公司） ； School of Medicine, New York Medical College（医学院，纽约医学院） ； Capital Medical University（首都医科大学） ； Department of Dermatology, Second Hospital of Jilin University（皮肤科，吉林大学第二医院） ； Emergency Critical Care Center, Beijing AnZhen Hospital, Capital Medical University（急诊重症中心，北京安贞医院，首都医科大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(title,abstract)；CoT(abstract)

Dissecting Long-Chain-of-Thought Reasoning Models: An Empirical Study

Yongyu Mu, Jiali Zeng, Bei Li, Xinyan Guan, Fandong Meng, Jie Zhou, Tong Xiao, Jingbo Zhu

机构 * Pattern Recognition Center, WeChat AI, Tencent Inc.（腾讯人工智能研究院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(title,abstract)；分类 cs.CL、cs.LG

Comments Working in process

Agentar-DeepFinance-100K: A Large-Scale Financial Dataset via Systematic Chain-of-Thought Synthesis Optimization

Xiaoke Zhao, Zhaowen Zhou, Lin Chen, Lihong Wang, Zhiyi Huang, Kaiyuan Zheng, Yanjun Zheng, Xiyang Du, Longfei Liao, Jiawei Liu, Xiang Qi, Bo Zhang, Peng Zhang, Wei Wang, Zhe Li

专题命中 推理评测 ：chain-of-thought(title,abstract)；reasoning(abstract)；CoT(abstract)

ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness

Yijun Liang, Ming Li, Chenrui Fan, Ziyue Li, Dang Nguyen, Kwesi Cobbina, Shweta Bhardwaj, Jiuhai Chen, Fuxiao Liu, Tianyi Zhou

机构 * University of Maryland, College Park（马里兰大学学院公园分校） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by NeurIPS2025. 36 pages, including references and appendix. Code is available at https://github.com/tianyi-lab/ColorBench

FedCoT: Federated Chain-of-Thought Distillation for Large Language Models

Tao Fan, Weijing Chen, Yan Kang, Guoqiang Ma, Hanlin Gu, Yuanfeng Song, Lixin Fan, Qiang Yang

机构 * Hong Kong University of Science and Technology（香港科学与技术大学） ； WeBank ； Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 推理评测 ：chain-of-thought(title,abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond

Yinghao Hu, Yaoyao Yu, Leilei Gan, Bin Wei, Kun Kuang, Fei Wu

机构 * College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院） ； Guanghua Law School, Zhejiang University（浙江大学光华法学院） ； School of Software Technology, Zhejiang University（浙江大学软件学院） ； Law & AI Lab, Zhejiang University（浙江大学法律与人工智能实验室） ； Shanghai AI Laboratory（上海人工智能实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 23 pages, Published in Findings of the Association for Computational Linguistics: EMNLP 2025

How Well Do LLMs Understand Drug Mechanisms? A Knowledge + Reasoning Evaluation Dataset

Sunil Mohan, Theofanis Karaletsos

机构 * CZI 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments An earlier version of this paper appears in IEEE FLLM 2025. GitHub: https://github.com/czi-ai/DrugMechCounterfactuals

An Empirical Study of Reasoning Steps in Thinking Code LLMs

Haoran Xue, Gias Uddin, Song Wang

机构 * York University（约克大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

NOVO: Bridging LLaVA and SAM with Visual-only Prompts for Reasoning Segmentation

Kyung-Yoon Yoon, Yeong-Jun Cho

机构 * Department of Artificial Intelligence Convergence（人工智能融合系） 纠错

专题命中 推理评测 ：reasoning(title,abstract)

How Reasoning Influences Intersectional Biases in Vision Language Models

Adit Desai, Sudipta Roy, Mohna Chakraborty

机构 * Student（学生） 纠错

专题命中 推理评测 ：reasoning(title,abstract)

Towards Competent AI for Fundamental Analysis in Finance: A Benchmark Dataset and Evaluation

Zonghan Wu, Congyuan Zou, Junlin Wang, Chenhan Wang, Hangjing Yang, Yilei Shao

机构 * Shanghai AI Finance School, East China Normal University（上海人工智能金融学院，华东师范大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.AI

DrKGC: Dynamic Subgraph Retrieval-Augmented LLMs for Knowledge Graph Completion across General and Biomedical Domains

Yongkang Xiao, Sinian Zhang, Yi Dai, Huixue Zhou, Jue Hou, Jie Ding, Rui Zhang

机构 * University of Minnesota（明尼苏达大学） ； University of Michigan（密歇根大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at EMNLP 2025 Findings

LLM-Driven Completeness and Consistency Evaluation for Cultural Heritage Data Augmentation in Cross-Modal Retrieval

Jian Zhang, Junyi Guo, Junyi Yuan, Huanda Lu, Yanlin Zhou, Fangyu Wu, Qiufeng Wang, Dongming Lu

机构 * Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学） ； NingboTech University（宁波科技学院） ； Dunhuang Academy（敦煌研究院） ； Zhejiang University（浙江大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)

When Language Shapes Thought: Cross-Lingual Transfer of Factual Knowledge in Question Answering

Eojin Kang, Juae Kim

机构 * Hankuk University of Foreign Studies（韩国外国语大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at CIKM2025 (Expanded version)

Evaluation of retrieval-based QA on QUEST-LOFT

Nathan Scales, Nathanael Schärli, Olivier Bousquet

机构 * Google DeepMind（谷歌DeepMind） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Latent Traits and Cross-Task Transfer: Deconstructing Dataset Interactions in LLM Fine-tuning

Shambhavi Krishna, Atharva Naik, Chaitali Agarwal, Sudharshan Govindan, Taesung Lee, Haw-Shiuan Chang

机构 * University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） ； Anthropic 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Proceedings of the 14th Joint Conference on Lexical and Computational Semantics (*SEM 2025)

Benchmarking Large Language Models with Integer Sequence Generation Tasks

Daniel O'Malley, Manish Bhattarai, Nishath Rajiv Ranasinghe, Erick Draayer, Javier Santos

机构 * Earth and Environmental Sciences Division Los Alamos National Laboratory（地球与环境科学 division 洛斯阿拉莫斯国家实验室） ； Theoretical Division Los Alamos National Laboratory（理论 division 洛斯阿拉莫斯国家实验室） ； Computational Sciences Division Los Alamos National Laboratory（计算科学 division 洛斯阿拉莫斯国家实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Evaluating the Ability of Large Language Models to Reason about Cardinal Directions, Revisited

Anthony G Cohn, Robert E Blackwell

机构 * School of Computer Science, University of Leeds, UK（利兹大学计算机科学学院） ； Alan Turing Institute, London, UK（艾伦·图灵研究所） 纠错

专题命中 推理评测 ：reasoning(abstract,comments)；分类 cs.CL

Comments 8 pages, 5 figures. Accepted at QR 2025 : 38th International Workshop on Qualitative Reasoning at IJCAI. arXiv admin note: substantial text overlap with arXiv:2406.16528

EMODIS: A Benchmark for Context-Dependent Emoji Disambiguation in Large Language Models

Jiacheng Huang, Ning Yu, Xiaoyin Yi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by AAAI2026

EduGuardBench: A Holistic Benchmark for Evaluating the Pedagogical Fidelity and Adversarial Safety of LLMs as Simulated Teachers

Yilin Jiang, Mingzi Zhang, Xuanyu Yin, Sheng Jin, Suyu Lu, Zuocan Ying, Zengyi Yu, Xiangjie Kong

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 22 pages, 9 figures, accepted by AAAI2026 as oral paper

Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights

Hyunjae Kim, Jiwoong Sohn, Aidan Gilson, Nicholas Cochran-Caggiano, Serina Applebaum, Heeju Jin, Seihee Park, Yujin Park, Jiyeong Park, Seoyoung Choi, Brittany Alexandra Herrera Contreras, Thomas Huang, Jaehoon Yun, Ethan F. Wei, Roy Jiang, Leah Colucci, Eric Lai, Amisha Dave, Tuo Guo, Maxwell B. Singer, Yonghoe Koo, Ron A. Adelman, James Zou, Andrew Taylor, Arman Cohan, Hua Xu, Qingyu Chen

机构 * Yale School of Medicine（耶鲁医学院） ； Yale University（耶鲁大学） ； ETH Zurich（苏黎世联邦理工学院） ； Harvard Medical School（哈佛医学院） ； Geisel School of Medicine at Dartmouth（达特茅斯大学盖塞医学院） ； Seoul National University College of Medicine（首尔国立大学医学院） ； Hanyang University College of Medicine（翰林大学医学院） ； PA Leadership Charter School（宾夕法尼亚州西切斯特市领导学校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 34 pages, 6 figures

SDS KoPub VDR: A Benchmark Dataset for Visual Document Retrieval in Korean Public Documents

Jaehoon Lee, Sohyun Kim, Wanggeun Park, Geon Lee, Seungkyung Kim, Minyoung Lee

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 27 pages, 15 figures, 6 tables

GUI-360$^\circ$: A Comprehensive Dataset and Benchmark for Computer-Using Agents

Jian Mu, Chaoyun Zhang, Chiming Ni, Lu Wang, Bo Qiao, Kartik Mathur, Qianhui Wu, Yuhang Xie, Xiaojun Ma, Mengyu Zhou, Si Qin, Liqun Li, Yu Kang, Minghua Ma, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

机构 * Nanjing University（南京大学） ； Microsoft（微软） ； ZJU-UIUC（浙大-UIUC） ； Peking University（北京大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

LegalEval-Q: A New Benchmark for The Quality Evaluation of LLM-Generated Legal Text

Li yunhan, Wu gengshen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 10 pages, 11 figures

Mufu: Multilingual Fused Learning for Low-Resource Translation with LLM

Zheng Wei Lim, Nitish Gupta, Honglin Yu, Trevor Cohn

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 29 pages

10 Open Challenges Steering the Future of Vision-Language-Action Models

Soujanya Poria, Navonil Majumder, Chia-Yu Hung, Amir Ali Bagherzadeh, Chuan Li, Kenneth Kwok, Ziwei Wang, Cheston Tan, Jiajun Wu, David Hsu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments AAAI 2026 (Senior Track)

Catching Contamination Before Generation: Spectral Kill Switches for Agents

Valentin Noël

机构 * Devoteam 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments Preprint under review (2025). 9 pages, 2 figures. Code and scripts: to be released

In-Context Adaptation of VLMs for Few-Shot Cell Detection in Optical Microscopy

Shreyan Ganguly, Angona Biswas, Jaydeep Rade, Md Hasibul Hasan Hasib, Nabila Masud, Nitish Singla, Abhipsa Dash, Ushashi Bhattacharjee, Aditya Balu, Anwesha Sarkar, Adarsh Krishnamurthy, Soumik Sarkar

机构 * Iowa State University（爱荷华州立大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

DOCUEVAL: An LLM-based AI Engineering Tool for Building Customisable Document Evaluation Workflows

Hao Zhang, Qinghua Lu, Liming Zhu

机构 * CSIRO’s Data61（CSIRO数据61） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Evaluating LLM-based Workflows for Switched-Mode Power Supply Design

Simon Nau, Jan Krummenauer, André Zimmermann

机构 * Robert Bosch GmbH, Cross-Domain Computing Solutions（罗伯特·博世有限公司，跨领域计算解决方案） ； University of Stuttgart, Institute for Micro Integration (IFM)（斯图加特大学，微系统集成研究所） ； Hahn-Schickard, Stuttgart, Germany（哈恩-施克尔德研究所，斯图加特，德国） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI