大模型推理能力

CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models

Yizhi LI, Ge Zhang, Xingwei Qu, Jiali Li, Zhaoqun Li, Zekun Wang, Hao Li, Ruibin Yuan, Yinghao Ma, Kai Zhang, Wangchunshu Zhou, Yiming Liang, Lei Zhang, Lei Ma, Jiajun Zhang, Zuowen Li, Stephen W. Huang, Chenghua Lin, Jie Fu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Camera-ready version for ACL 2024. Project page at https://yizhilll.github.io/CIF-Bench/

REBUS: A Robust Evaluation Benchmark of Understanding Symbols

Andrew Gritsevskiy, Arjun Panickssery, Aaron Kirtland, Derik Kauffman, Hans Gundlach, Irina Gritsevskaya, Joe Cavanagh, Jonathan Chiang, Lydia La Roux, Michelle Hung

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 20 pages, 5 figures. For code, see http://github.com/cvndsh/rebus

AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators

Jingwei Ni, Minjing Shi, Dominik Stammbach, Mrinmaya Sachan, Elliott Ash, Markus Leippold

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments ACL2024 Main Conference

The Critique of Critique

Shichao Sun, Junlong Li, Weizhe Yuan, Ruifeng Yuan, Wenjie Li, Pengfei Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to Findings of ACL 2024

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

Albert Gu, Tri Dao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLM

Eliya Nachmani, Alon Levkovitch, Roy Hirsch, Julian Salazar, Chulayuth Asawaroengchai, Soroosh Mariooryad, Ehud Rivlin, RJ Skerry-Ryan, Michelle Tadmor Ramanovich

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2024 camera-ready

X-VILA: Cross-Modality Alignment for Large Language Model

Hanrong Ye, De-An Huang, Yao Lu, Zhiding Yu, Wei Ping, Andrew Tao, Jan Kautz, Song Han, Dan Xu, Pavlo Molchanov, Hongxu Yin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Technical Report

SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models

Lee Hyun, Kim Sung-Bin, Seungju Han, Youngjae Yu, Tae-Hyun Oh

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 14 figures

Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction

Tingchen Fu, Deng Cai, Lemao Liu, Shuming Shi, Rui Yan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to the findings of ACL2024

IM-RAG: Multi-Round Retrieval-Augmented Generation Through Learning Inner Monologues

Diji Yang, Jinmeng Rao, Kezhen Chen, Xiaoyuan Guo, Yawen Zhang, Jie Yang, Yi Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Proceedings of the 47th International ACM SIGIR 2024

KG-RAG: Bridging the Gap Between Knowledge and Creativity

Diego Sanmartin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Acquiring and Modelling Abstract Commonsense Knowledge via Conceptualization

Mutian He, Tianqing Fang, Weiqi Wang, Yangqiu Song

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 43 pages, 11 figures; Accepted by Artificial Intelligence

GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving

Jiaxin Zhang, Zhongzhi Li, Mingliang Zhang, Fei Yin, Chenglin Liu, Yashar Moshfeghi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted in ACL 2024 Findings

Benchmarking Large Language Models on CFLUE -- A Chinese Financial Language Understanding Evaluation Dataset

Jie Zhu, Junhui Li, Yalong Wen, Lifan Guo

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024

Journal ref The 62nd Annual Meeting of the Association for Computational Linguistics(ACL),2024

IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Control

Rohan Chitnis, Yingchen Xu, Bobak Hashemi, Lucas Lehnert, Urun Dogan, Zheqing Zhu, Olivier Delalleau

专题命中 推理评测 ：planning(abstract)；分类 cs.AI、cs.LG

Journal ref Short version published at ICRA 2024 (https://tinyurl.com/icra24-iqltdmpc)

MedConceptsQA: Open Source Medical Concepts QA Benchmark

Ofir Ben Shoham, Nadav Rappoport

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

SUTRA: Scalable Multilingual Language Model Architecture

Abhijit Bendale, Michael Sapienza, Steven Ripplinger, Simon Gibbs, Jaewon Lee, Pranav Mistry

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Open-SQL Framework: Enhancing Text-to-SQL on Open-source Large Language Models

Xiaojun Chen, Tianle Wang, Tianhao Qiu, Jianbin Qin, Min Yang

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI

Aloe: A Family of Fine-tuned Open Healthcare LLMs

Ashwin Kumar Gururajan, Enrique Lopez-Cuena, Jordi Bayarri-Planas, Adrian Tormos, Daniel Hinjos, Pablo Bernabeu-Perez, Anna Arias-Duart, Pablo Agustin Martin-Torres, Lucia Urcelay-Ganzabal, Marta Gonzalez-Mallo, Sergio Alvarez-Napagao, Eduard Ayguadé-Parra, Ulises Cortés Dario Garcia-Gasulla

专题命中 推理评测 ：CoT(abstract)；分类 cs.CL、cs.AI

Comments Five appendix

Towards Generalizable Agents in Text-Based Educational Environments: A Study of Integrating RL with LLMs

Bahar Radmehr, Adish Singla, Tanja Käser

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments Accepted as a full paper at EDM 2024: The 17th International Conference on Educational Data Mining, 14-17 of July 2024, Atlanta

FoundaBench: Evaluating Chinese Fundamental Knowledge Capabilities of Large Language Models

Wei Li, Ren Ma, Jiang Wu, Chenya Gu, Jiahui Peng, Jinyang Len, Songyang Zhang, Hang Yan, Dahua Lin, Conghui He

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Make Your LLM Fully Utilize the Context

Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 7 figures, 3 tables, 9 examples

Learning Performance-Improving Code Edits

Alexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob Gardner, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, Amir Yazdanbakhsh

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at ICLR 2024 (Spotlight). Project website: https://pie4perf.com/

Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model

Jacob Eisenstein, Daniel Andor, Bernd Bohnet, Michael Collins, David Mimno

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments added details about a human evaluation

Multi Class Depression Detection Through Tweets using Artificial Intelligence

Muhammad Osama Nusrat, Waseem Shahzad, Saad Ahmed Jamal

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 33 pages

FlowMind: Automatic Workflow Generation with LLMs

Zhen Zeng, William Watson, Nicole Cho, Saba Rahimi, Shayleen Reynolds, Tucker Balch, Manuela Veloso

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Published in ACM ICAIF 2023

HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses

Xinke Jiang, Ruizhe Zhang, Yongxin Xu, Rihong Qiu, Yue Fang, Zhiyuan Wang, Jinyi Tang, Hongxin Ding, Xu Chu, Junfeng Zhao, Yasha Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments version 2

ViLLM-Eval: A Comprehensive Evaluation Suite for Vietnamese Large Language Models

Trong-Hieu Nguyen, Anh-Cuong Le, Viet-Cuong Nguyen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments arXiv admin note: text overlap with arXiv:2305.08322 by other authors

MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning

Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Yacoob, Dong Yu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to NAACL 2024

Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds

Victoria Basmov, Yoav Goldberg, Reut Tsarfaty

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI