大模型推理能力

A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages

Raoyuan Zhao, Yihong Liu, Hinrich Schütze, Michael A. Hedderich

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(title,abstract)；CoT(abstract)；分类 cs.CL

Comments preprint

Benchmarking Chinese Commonsense Reasoning with a Multi-hop Reasoning Perspective

Wangjie You, Xusheng Wang, Xing Wang, Wenxiang Jiao, Chao Feng, Juntao Li, Min Zhang

机构 * Douyin Content Group, ByteDance（字节跳动内容部） ； School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Unleashing Perception-Time Scaling to Multimodal Reasoning Models

Yifan Li, Zhenghao Chen, Ziheng Wu, Kun Zhou, Ruipu Luo, Can Zhang, Zhentao He, Yufei Zhan, Wayne Xin Zhao, Minghui Qiu

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学北京校区人工智能学院） ； Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大模型与智能治理重点实验室） ； ByteDance（字节跳动） ； University of California, San Diego（加州大学圣地亚哥分校） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；math reasoning(abstract)；分类 cs.CL

BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception

Junyan Ye, Dongzhi Jiang, Jun He, Baichuan Zhou, Zilong Huang, Zhiyuan Yan, Hongsheng Li, Conghui He, Weijia Li

机构 * Sun Yat-sen University（中山大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； CUHK MMLab（香港中文大学多模态实验室） ； Peking University（北京大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)

Comments Accepted to 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Track on Datasets and Benchmarks

CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models

Aneesh Komanduri, Karuna Bhaila, Xintao Wu

机构 * Department of Electrical Engineering and Computer Science University of Arkansas（电气工程与计算机科学系 奎萨克大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025 Main)

ARS: Adaptive Reasoning Suppression for Efficient Large Reasoning Language Models

Dongqi Zheng

机构 * Independent Researcher（独立研究者） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by 39th NeurIPS - Foundations of Reasoning in Language Models

CapGeo: A Caption-Assisted Approach to Geometric Reasoning

Yuying Li, Siyi Qian, Hao Liang, Leqi Zheng, Ruichuan An, Yongzhen Guo, Wentao Zhang

机构 * THU（清华大学） ； PKU（北京大学） ； Ant Group（蚂蚁集团） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments preprint, under review

RegexPSPACE: A Benchmark for Evaluating LLM Reasoning on PSPACE-complete Regex Problems

Hyundong Jin, Joonghyuk Hahn, Yo-Sub Han

机构 * Department of Computer Science Yonsei University（计算机科学系延世大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Do LLMs Know They Are Being Tested? Evaluation Awareness and Incentive-Sensitive Failures in GPT-OSS-20B

Nisar Ahmed, Muhammad Imran Zaman, Gulshan Saleem, Ali Hassan

机构 * Sparkverse AI Ltd（Sparkverse AI有限公司） ； Department of Informatics and Systems（信息学与系统系） ； University of Management and Technology（管理与技术大学） ； Faculty of Information Technology and Computer Science（信息科技与计算机科学学院） ； University of Central Punjab（旁遮普大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Alif: Advancing Urdu Large Language Models via Multilingual Synthetic Data Distillation

Muhammad Ali Shafique, Kanwal Mehreen, Muhammad Arham, Maaz Amjad, Sabur Butt, Hamza Farooq

机构 * University of British Columbia（不列颠哥伦比亚大学） ； Texas Tech University（德克萨斯技术大学） ； Institute for the Future of Education, Tecnológico de Monterrey（教育未来研究所，墨西哥蒙特雷技术学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to the EMNLP 2025 Workshop on Multilingual Representation Learning (MRL)

Mnemosyne: An Unsupervised, Human-Inspired Long-Term Memory Architecture for Edge-Based LLMs

Aneesh Jonelagadda, Christina Hahn, Haoze Zheng, Salvatore Penachio

机构 * Kaliber AI, Research Division（Kaliber AI 研究部门） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 12 pages, 4 figures

Online Rubrics Elicitation from Pairwise Comparisons

MohammadHossein Rezaei, Robert Vacareanu, Zihao Wang, Clinton Wang, Bing Liu, Yunzhong He, Afra Feyza Akyürek

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices

Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Ridwan Mahbub, Mizanur Rahman, Amran Bhuiyan, Israt Jahan, Mir Tafseer Nayeem, Shafiq Joty, Enamul Hoque, Jimmy Huang

机构 * York University（约克大学） ； University of Alberta（阿尔伯塔大学） ； Salesforce AI Research（Salesforce AI研究） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Accepted to the EMNLP 2025 Industry Track

RADAR: Mechanistic Pathways for Detecting Data Contamination in LLM Evaluation

Ashish Kattamuri, Harshwardhan Fartale, Arpita Vats, Rahul Raja, Ishita Prasad

机构 * Proofpoint ； Indian Institute of Science（印度科学研究院） ； Linkedin ； Meta FAIR 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2025 Workshop on Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling

Inverse-Free Wilson Loops for Transformers: A Practical Diagnostic for Invariance and Order Sensitivity

Edward Y. Chang, Ethan Y. Chang

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.AI、cs.LG

Comments 24 pages, 10 figures, 2 tables

From What to Why: Thought-Space Recommendation with Small Language Models

Prosenjit Biswas, Pervez Shaik, Abhinav Thorat, Ravi Kolla, Niranjan Pedanekar

机构 * Sony Research India（索尼印度研究） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 3 figures

MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation

Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen

机构 * Singapore University of Technology and Design（新加坡科技设计大学） ； Agency for Science, Technology and Research, Singapore（新加坡科技研究局） ； Indian Institute of Technology Delhi（印度理工学院德里分校） ； Alibaba DAMO Academy（阿里巴巴达摩院） ； Microsoft Research Asia（微软亚洲研究院） ； Shanghai University of Finance and Economics（上海财经大学） ； Inner Mongolia University（内蒙古大学） ； Kyoto University（京都大学） ； Jiangxi Normal University（江西师范大学） ； Korea University（韩国大学） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation

Mufei Li, Dongqi Fu, Limei Wang, Si Zhang, Hanqing Zeng, Kaan Sancak, Ruizhong Qiu, Haoyu Wang, Xiaoxin He, Xavier Bresson, Yinglong Xia, Chonglin Sun, Pan Li

机构 * Georgia Institute of Technology（佐治亚理工学院） ； Meta AI ； University of Illinois Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Code available at https://github.com/Graph-COM/HaystackCraft

LLP: LLM-based Product Pricing in E-commerce

Hairu Wang, Sheng You, Qiheng Zhang, Xike Xie, Shuguang Han, Yuchen Wu, Fei Huang, Jufeng Chen

机构 * University of Science and Technology of China（中国科学技术大学） ； Xianyu of Alibaba（阿里巴巴闲鱼） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition

Yushuo Zheng, Zicheng Zhang, Xiongkuo Min, Huiyu Duan, Guangtao Zhai

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

GBA-UBF : A Large-Scale and Fine-Grained Building Function Classification Dataset in the Greater Bay Area

Chunsong Chen, Yichen Hou, Huan Chen, Junlin Li, Rong Fu, Qiushen Lai, Yiping Chen, Ting Han

专题命中 推理评测 ：planning(abstract)

Comments 12 pages, 10 figures, The 4th ACM SIGSPATIAL International Workshop on Spatial Big Data and AI for Industrial Applications (GeoIndustry '25), November 3--6, 2025, Minneapolis, MN, USA

Automating eHMI Action Design with LLMs for Automated Vehicle Communication

Ding Xia, Xinyue Gui, Fan Gao, Dongyuan Li, Mark Colley, Takeo Igarashi

机构 * The University of Tokyo（东京大学） ； University College London（伦敦大学学院） 纠错

专题命中 推理评测 ：reasoning(abstract)

Comments Accepted as findings for EMNLP 2025