大模型推理能力

Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs) More Self-Confident Even When They Are Wrong

Tairan Fu, Javier Conde, Gonzalo Martínez, María Grandury, Pedro Reviriego

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Self-playing Adversarial Language Game Enhances LLM Reasoning

Pengyu Cheng, Tianhao Hu, Han Xu, Zhisong Zhang, Zheng Yuan, Yong Dai, Lei Han, Nan Du, Xiaolong Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted by NeurIPS 2024

On the Reasoning Capacity of AI Models and How to Quantify It

Santosh Kumar Radha, Oktay Goktas

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

UTMath: Math Evaluation with Unit Test via Reasoning-to-Coding Thoughts

Bo Yang, Qingping Yang, Yingwei Ma, Runtao Liu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

ZNO-Eval: Benchmarking reasoning capabilities of large language models in Ukrainian

Mykyta Syromiatnikov, Victoria Ruvinskaya, Anastasiya Troynina

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 7 pages, 5 figures. X International conference "Informatics. Culture. Technology." (2024)

Journal ref X International conference "Informatics. Culture. Technology." (2024) 185-191

MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs

Jihyung Kil, Zheda Mai, Justin Lee, Zihe Wang, Kerrie Cheng, Lemeng Wang, Ye Liu, Arpita Chowdhury, Wei-Lun Chao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments This paper has been accepted to NeurIPS 2024. The first two authors contributed equally to this work

Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generation

Alireza Salemi, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Weize Kong, Tao Chen, Zhuowan Li, Michael Bendersky, Hamed Zamani

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reasoning over Uncertain Text by Generative Large Language Models

Aliakbar Nafar, Kristen Brent Venable, Parisa Kordjamshidi

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks

Shiduo Zhang, Zhe Xu, Peiju Liu, Xiaopeng Yu, Yuan Li, Qinghui Gao, Zhaoye Fei, Zhangyue Yin, Zuxuan Wu, Yu-Gang Jiang, Xipeng Qiu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Adaptive-Solver Framework for Dynamic Strategy Selection in Large Language Model Reasoning

Jianpeng Zhou, Wanjun Zhong, Yanlin Wang, Jiahai Wang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by Information Processing & Management

Improving Retrieval Augmented Language Model with Self-Reasoning

Yuan Xia, Jingbo Zhou, Zhenhui Shi, Jun Chen, Haifeng Huang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments AAAI 2025 (main conference)

SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluation

Hangfeng He, Hongming Zhang, Dan Roth

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor

Jiali Chen, Xusen Hei, Yuqi Xue, Yuancheng Wei, Jiayuan Xie, Yi Cai, Qing Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACM MM 2024

VisScience: An Extensive Benchmark for Evaluating K12 Educational Multi-modal Scientific Reasoning

Zhihuan Jiang, Zhen Yang, Jinhao Chen, Zhengxiao Du, Weihan Wang, Bin Xu, Jie Tang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 89 pages, 70 figures

Inducing Human-like Biases in Moral Reasoning Language Models

Artem Karpov, Seong Hah Cho, Austin Meek, Raymond Koopmanschap, Lucy Farnik, Bogdan-Ionut Cirstea

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted to the 2nd Workshop on Unifying Representations in Neural Models (UniReps) at NeurIPS 2024

Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpus

Seungpil Lee, Woochang Sim, Donghyeon Shin, Wongyu Seo, Jiwon Park, Seokki Lee, Sanha Hwang, Sejin Kim, Sundong Kim

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Evaluating and Enhancing Large Language Models for Conversational Reasoning on Knowledge Graphs

Yuxuan Huang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance

Kai Xiong, Xiao Ding, Ting Liu, Bing Qin, Dongliang Xu, Qing Yang, Hongtao Liu, Yixin Cao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack

Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Ivan Rodkin, Dmitry Sorokin, Artyom Sorokin, Mikhail Burtsev

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024 Datasets and Benchmarks Track

JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking

Tong Niu, Shafiq Joty, Ye Liu, Caiming Xiong, Yingbo Zhou, Semih Yavuz

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers?

Neeladri Bhuiya, Viktor Schlegel, Stefan Winkler

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 3 figures, EMNLP 2024 Main Conference

AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoning

Shirley Wu, Shiyu Zhao, Qian Huang, Kexin Huang, Michihiro Yasunaga, Kaidi Cao, Vassilis N. Ioannidis, Karthik Subbian, Jure Leskovec, James Zou

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2024 main conference

MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark

S Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, Dinesh Manocha

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Project Website: https://sakshi113.github.io/mmau_homepage/

StrucText-Eval: Evaluating Large Language Model's Reasoning Ability in Structure-Rich Text

Zhouhong Gu, Haoning Ye, Xingzhou Chen, Zeyang Zhou, Hongwei Feng, Yanghua Xiao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

LAR-ECHR: A New Legal Argument Reasoning Task and Dataset for Cases of the European Court of Human Rights

Odysseas S. Chlapanis, Dimitrios Galanis, Ion Androutsopoulos

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Published in Natural Legal Language Processing (NLLP) 2024 workshop

A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning

Yuanning Cui, Zequn Sun, Wei Hu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted in the 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities

Lichang Chen, Hexiang Hu, Mingda Zhang, Yiwen Chen, Zifeng Wang, Yandong Li, Pranav Shyam, Tianyi Zhou, Heng Huang, Ming-Hsuan Yang, Boqing Gong

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 6 figures, 12 tables

Concept-Reversed Winograd Schema Challenge: Evaluating and Improving Robust Reasoning in Large Language Models via Abstraction

Kaiqiao Han, Tianqing Fang, Zhaowei Wang, Yangqiu Song, Mark Steedman

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

MMLU-Pro+: Evaluating Higher-Order Reasoning and Shortcut Learning in LLMs

Saeid Asgari Taghanaki, Aliasgahr Khani, Amir Khasahmadi

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to NeurIPS 2024, Safe Generative AI

Zero-shot Commonsense Reasoning over Machine Imagination

Hyuntae Park, Yeachan Kim, Jun-Hyung Park, SangKeun Lee

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 21 pages, 9 figures, EMNLP 2024 (Findings)