大模型推理能力

Connecting the Dots: Evaluating Abstract Reasoning Capabilities of LLMs Using the New York Times Connections Word Game

Prisha Samadarshi, Mariam Mustafa, Anushka Kulkarni, Raven Rothkopf, Tuhin Chakrabarty, Smaranda Muresan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning

Jiachun Li, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Jun Zhao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2024 Findings

GSR-BENCH: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMs

Navid Rajabi, Jana Kosecka

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to NeurIPS 2024 Workshop on Compositional Learning

ReFeR: Improving Evaluation and Reasoning through Hierarchy of Models

Yaswanth Narsupalli, Abhranil Chandra, Sreevatsa Muppirala, Manish Gupta, Pawan Goyal

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Paper Under Review

Temporal Fact Reasoning over Hyper-Relational Knowledge Graphs

Zifeng Ding, Jingcheng Wu, Jingpei Wu, Yan Xia, Volker Tresp

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Findings of EMNLP 2024

A Case Study of Web App Coding with OpenAI Reasoning Models

Yi Cui

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reasoning Factual Knowledge in Structured Data with Large Language Models

Sirui Huang, Yanggan Gu, Xuming Hu, Zhonghao Li, Qing Li, Guandong Xu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

New Curriculum, New Chance -- Retrieval Augmented Generation for Lesson Planning in Ugandan Secondary Schools. Prototype Quality Evaluation

Simon Kloker, Herbertson Bukoli, Twaha Kateete

专题命中 推理评测 ：planning(title,abstract)；分类 cs.AI、cs.LG

Comments Presented at Ndejje University Second Annual Research Dissemination Symposium 2024

PLUGH: A Benchmark for Spatial Understanding and Reasoning in Large Language Models

Alexey Tikhonov

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Wordplay Workshop @ ACL 2024

Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thought

Zaijing Li, Gongwei Chen, Rui Shao, Yuquan Xie, Dongmei Jiang, Liqiang Nie

专题命中 推理评测 ：chain-of-thought(title,abstract)；分类 cs.CL、cs.AI

A Comprehensive Evaluation on Event Reasoning of Large Language Models

Zhengwei Tao, Zhi Jin, Yifan Zhang, Xiancai Chen, Haiyan Zhao, Jia Li, Bing Liang, Chongyang Tao, Qun Liu, Kam-Fai Wong

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learning

Hosung Lee, Sejin Kim, Seungpil Lee, Sanha Hwang, Jihwan Lee, Byung-Jun Lee, Sundong Kim

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by CoLLAs 2024, Project page: https://github.com/confeitoHS/arcle

Steamroller Problems: An Evaluation of LLM Reasoning Capability with Automated Theorem Prover Strategies

Lachlan McGinness, Peter Baumgartner

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Look Globally and Reason: Two-stage Path Reasoning over Sparse Knowledge Graphs

Saiping Guan, Jiyao Wei, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted to CIKM 2024

HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning

Zhecan Wang, Garrett Bingham, Adams Yu, Quoc Le, Thang Luong, Golnaz Ghiasi

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted as a main conference paper at ECCV 2024 (https://github.com/google/haloquest)

RDBE: Reasoning Distillation-Based Evaluation Enhances Automatic Essay Scoring

Ali Ghiasvand Mohammadkhani

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Exploring State Space and Reasoning by Elimination in Tsetlin Machines

Ahmed K. Kadhim, Ole-Christoffer Granmo, Lei Jiao, Rishad Shafik

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 8 figures

A Critical Review of Causal Reasoning Benchmarks for Large Language Models

Linying Yang, Vik Shirvaikar, Oscar Clivio, Fabian Falck

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments AAAI 2024 Workshop on ''Are Large Language Models Simply Causal Parrots?''

SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning

Bin Wang, Zhengyuan Liu, Xin Huang, Fangkai Jiao, Yang Ding, AiTi Aw, Nancy F. Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Published at NAACL 2024. Code: https://seaeval.github.io/

Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models

Mosh Levy, Alon Jacoby, Yoav Goldberg

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2024

RVISA: Reasoning and Verification for Implicit Sentiment Analysis

Wenna Lai, Haoran Xie, Guandong Xu, Qing Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 6 figures, and 4 tables

Fake News Detection and Manipulation Reasoning via Large Vision-Language Models

Ruihan Jin, Ruibo Fu, Zhengqi Wen, Shuai Zhang, Yukun Liu, Jianhua Tao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language Models by Learning from Knowledge Graphs

Yifei Zhang, Xintao Wang, Jiaqing Liang, Sirui Xia, Lida Chen, Yanghua Xiao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models

Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Haotian Wang, Ming Liu, Bing Qin

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2024

Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Models

Wenxuan Ding, Shangbin Feng, Yuhan Liu, Zhaoxuan Tan, Vidhisha Balachandran, Tianxing He, Yulia Tsvetkov

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reasoning Like a Doctor: Improving Medical Dialogue Systems via Diagnostic Reasoning Process Alignment

Kaishuai Xu, Yi Cheng, Wenjun Hou, Qiaoyu Tan, Wenjie Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024 Findings

Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning

Bingchen Zhao, Yongshuo Zong, Letian Zhang, Timothy Hospedales

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments First three authors contributed equally. Dataset: https://huggingface.co/datasets/VLLMs/MIRB

MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, Wenhu Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments CVPR 2024 Oral

Eliciting Better Multilingual Structured Reasoning from LLMs through Code

Bryan Li, Tamer Alkhouli, Daniele Bonadiman, Nikolaos Pappas, Saab Mansour

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Multimodal Reasoning with Multimodal Knowledge Graph

Junlin Lee, Yequan Wang, Jing Li, Min Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024 (Main Conference)