大模型推理能力

KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs

Elan Markowitz, Krupa Galiya, Greg Ver Steeg, Aram Galstyan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments To be presented at NAACL-HLT, KnowledgeNLP Workshop (2025)

NTSEBENCH: Cognitive Reasoning Benchmark for Vision Language Models

Pranshu Pandya, Vatsal Gupta, Agney S Talwarr, Tushar Kataria, Dan Roth, Vivek Gupta

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 28 pages, 3 figures, 12 tables

Memorizing is Not Enough: Deep Knowledge Injection Through Reasoning

Ruoxi Xu, Yunjie Ji, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Ben He, Yingfei Sun, Xiangang Li, Le Sun

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL

Mohammadreza Pourreza, Shayan Talaei, Ruoxi Sun, Xingchen Wan, Hailong Li, Azalia Mirhoseini, Amin Saberi, Sercan "O. Arik

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Mohammadreza Pourreza and Shayan Talaei contributed equally to this work

The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction

Yihuai Hong, Dian Zhou, Meng Cao, Lei Yu, Zhijing Jin

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Browsing Lost Unformed Recollections: A Benchmark for Tip-of-the-Tongue Search and Reasoning

Sky CH-Wang, Darshan Deshpande, Smaranda Muresan, Anand Kannappan, Rebecca Qian

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Evaluating System 1 vs. 2 Reasoning Approaches for Zero-Shot Time Series Forecasting: A Benchmark and Insights

Haoxin Liu, Zhiyuan Zhao, Shiduo Li, B. Aditya Prakash

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

A Survey on Enhancing Causal Reasoning Ability of Large Language Models

Xin Li, Zhuo Cai, Shoujin Wang, Kun Yu, Fang Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Geometric Analysis of Reasoning Trajectories: A Phase Space Approach to Understanding Valid and Invalid Multi-Hop Reasoning in LLMs

Javier Marin

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Knowledge Updating? No More Model Editing! Just Selective Contextual Reasoning

Guoxiu He, Xin Song, Aixin Sun

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Exploring and Evaluating Multimodal Knowledge Reasoning Consistency of Multimodal Large Language Models

Boyu Jia, Junzhe Zhang, Huixuan Zhang, Xiaojun Wan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities

Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, Bryan Catanzaro

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI

Zhen Huang, Zengzhi Wang, Shijie Xia, Xuefeng Li, Haoyang Zou, Ruijie Xu, Run-Ze Fan, Lyumanshan Ye, Ethan Chern, Yixin Ye, Yikai Zhang, Yuqing Yang, Ting Wu, Binjie Wang, Shichao Sun, Yang Xiao, Yiyuan Li, Fan Zhou, Steffi Chern, Yiwei Qin, Yan Ma, Jiadi Su, Yixiu Liu, Yuxiang Zheng, Shaoting Zhang, Dahua Lin, Yu Qiao, Pengfei Liu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by NeurIPS 2024

Visual Description Grounding Reduces Hallucinations and Boosts Reasoning in LVLMs

Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Utkarsh Tyagi, Oriol Nieto, Zeyu Jin, Dinesh Manocha

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ICLR 2025. Project: https://sreyan88.github.io/VDGD/

HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoning

Zhuohang Jiang, Pangjing Wu, Ziran Liang, Peter Q. Chen, Xu Yuan, Ye Jia, Jiancheng Tu, Chen Li, Peter H. F. Ng, Qing Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Models

Jiachun Li, Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, Jun Zhao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted as ICLR 2025 conference paper (26 pages, 16 tables, 9 figures)

TripCraft: A Benchmark for Spatio-Temporally Fine Grained Travel Planning

Soumyabrata Chaudhuri, Pranav Purkar, Ritwik Raghav, Shubhojit Mallick, Manish Gupta, Abhik Jana, Shreya Ghosh

专题命中 推理评测 ：planning(title,abstract)；分类 cs.CL、cs.AI

Comments 27 pages, 18 Tables and 6 Figures

UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models

Xin Xu, Jiaxin Zhang, Tianhao Chen, Zitong Chao, Jishan Hu, Can Yang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ICLR 2025

Journal ref International Conference on Learning Representations (ICLR 2025)

EPERM: An Evidence Path Enhanced Reasoning Model for Knowledge Graph Question and Answering

Xiao Long, Liansheng Zhuang, Aodi Li, Minghong Yao, Shafei Wang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Zero-Shot Commonsense Validation and Reasoning with Large Language Models: An Evaluation on SemEval-2020 Task 4 Dataset

Rawand Alfugaha, Mohammad AL-Smadi

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reasoning Bias of Next Token Prediction Training

Pengxiao Lin, Zhongwang Zhang, Zhi-Qin John Xu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments 19 pages, 11 figures

LinguaLIFT: An Effective Two-stage Instruction Tuning Framework for Low-Resource Language Reasoning

Hongbin Zhang, Kehai Chen, Xuefeng Bai, Yang Xiang, Min Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

GameArena: Evaluating LLM Reasoning through Live Computer Games

Lanxiang Hu, Qiyu Li, Anze Xie, Nan Jiang, Ion Stoica, Haojian Jin, Hao Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

EnigmaEval: A Benchmark of Long Multimodal Reasoning Challenges

Clinton J. Wang, Dean Lee, Cristina Menghini, Johannes Mols, Jack Doughty, Adam Khoja, Jayson Lynch, Sean Hendryx, Summer Yue, Dan Hendrycks

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

A Cognitive Evaluation Benchmark of Image Reasoning and Description for Large Vision-Language Models

Xiujie Song, Mengyue Wu, Kenny Q. Zhu, Chunhao Zhang, Yanyi Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

CIRCUIT: A Benchmark for Circuit Interpretation and Reasoning Capabilities of LLMs

Lejla Skelic, Yan Xu, Matthew Cox, Wenjie Lu, Tao Yu, Ruonan Han

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Multimodal Cognitive Reframing Therapy via Multi-hop Psychotherapeutic Reasoning

Subin Kim, Hoonrae Kim, Heejin Do, Gary Geunbae Lee

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments NAACL 2025 Main

DebateBench: A Challenging Long Context Reasoning Benchmark For Large Language Models

Utkarsh Tiwari, Aryan Seth, Adi Mukherjee, Kaavya Mer, Kavish, Dhruv Kumar

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Evaluating LLM Reasoning in the Operations Research Domain with ORQA

Mahdi Mostajabdaveh, Timothy T. Yu, Samarendra Chandan Bindu Dash, Rindranirina Ramamonjison, Jabo Serge Byusa, Giuseppe Carenini, Zirui Zhou, Yong Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 10 figures. Accepted and to be published in AAAI25

Large Language Models Meet Graph Neural Networks for Text-Numeric Graph Reasoning

Haoran Song, Jiarui Feng, Guangfu Li, Michael Province, Philip Payne, Yixin Chen, Fuhai Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments 29 pages, 6 figures