大模型推理能力

Language Modeling with Editable External Knowledge

Belinda Z. Li, Emmy Liu, Alexis Ross, Abbas Zeitoun, Graham Neubig, Jacob Andreas

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Demonstration Notebook: Finding the Most Suited In-Context Learning Example from Interactions

Yiming Tang, Bin Dong

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

InSaAF: Incorporating Safety through Accuracy and Fairness | Are LLMs ready for the Indian Legal Domain?

Yogesh Tripathi, Raghav Donakanti, Sahil Girhepuje, Ishan Kavathekar, Bhaskara Hanuma Vedula, Gokul S Krishnan, Shreya Goyal, Anmol Goel, Balaraman Ravindran, Ponnurangam Kumaraguru

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

ConceptPsy:A Benchmark Suite with Conceptual Comprehensiveness in Psychology

Junlei Zhang, Hongliang He, Nirui Song, Zhanchao Zhou, Shuyuan He, Shuai Zhang, Huachuan Qiu, Anqi Li, Yong Dai, Lizhi Ma, Zhenzhong Lan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Under Review

FinDABench: Benchmarking Financial Data Analysis Ability of Large Language Models

Shu Liu, Shangqing Zhao, Chenghao Jia, Xinlin Zhuang, Zhaoguang Long, Jie Zhou, Aimin Zhou, Man Lan, Qingquan Wu, Chong Yang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Long-context LLMs Struggle with Long In-context Learning

Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, Wenhu Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Large Language Models for Automated Open-domain Scientific Hypotheses Discovery

Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, Erik Cambria

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024 (findings)

Towards a Personal Health Large Language Model

Justin Cosentino, Anastasiya Belyaeva, Xin Liu, Nicholas A. Furlotte, Zhun Yang, Chace Lee, Erik Schenck, Yojan Patel, Jian Cui, Logan Douglas Schneider, Robby Bryant, Ryan G. Gomes, Allen Jiang, Roy Lee, Yun Liu, Javier Perez, Jameson K. Rogers, Cathy Speed, Shyam Tailor, Megan Walker, Jeffrey Yu, Tim Althoff, Conor Heneghan, John Hernandez, Mark Malhotra, Leor Stern, Yossi Matias, Greg S. Corrado, Shwetak Patel, Shravya Shetty, Jiening Zhan, Shruthi Prabhakara, Daniel McDuff, Cory Y. McLean

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 72 pages

BloomVQA: Assessing Hierarchical Multi-modal Comprehension

Yunye Gong, Robik Shrestha, Jared Claypoole, Michael Cogswell, Arijit Ray, Christopher Kanan, Ajay Divakaran

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Accepted by ACL Findings (2024). Dataset available at https://huggingface.co/datasets/ygong/BloomVQA

Benchmarking Large Language Models on Communicative Medical Coaching: a Novel System and Dataset

Hengguan Huang, Songtao Wang, Hongfu Liu, Hao Wang, Ye Wang

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments Accepted to Findings of the Association for Computational Linguistics: ACL 2024

The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models

Noah Y. Siegel, Oana-Maria Camburu, Nicolas Heess, Maria Perez-Ortiz

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments To be published in ACL 2024. 19 pages, 2 figures

CLOMO: Counterfactual Logical Modification with Large Language Models

Yinya Huang, Ruixin Hong, Hongming Zhang, Wei Shao, Zhicheng Yang, Dong Yu, Changshui Zhang, Xiaodan Liang, Linqi Song

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Journal ref Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL 2024)

Fusion-Eval: Integrating Assistant Evaluators with LLMs

Lei Shu, Nevan Wichers, Liangchen Luo, Yun Zhu, Yinxiao Liu, Jindong Chen, Lei Meng

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Models

Andrew Zhu, Alyssa Hwang, Liam Dugan, Chris Callison-Burch

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 2 figures. ACL 2024

CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models

Yizhi LI, Ge Zhang, Xingwei Qu, Jiali Li, Zhaoqun Li, Zekun Wang, Hao Li, Ruibin Yuan, Yinghao Ma, Kai Zhang, Wangchunshu Zhou, Yiming Liang, Lei Zhang, Lei Ma, Jiajun Zhang, Zuowen Li, Stephen W. Huang, Chenghua Lin, Jie Fu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Camera-ready version for ACL 2024. Project page at https://yizhilll.github.io/CIF-Bench/

REBUS: A Robust Evaluation Benchmark of Understanding Symbols

Andrew Gritsevskiy, Arjun Panickssery, Aaron Kirtland, Derik Kauffman, Hans Gundlach, Irina Gritsevskaya, Joe Cavanagh, Jonathan Chiang, Lydia La Roux, Michelle Hung

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 20 pages, 5 figures. For code, see http://github.com/cvndsh/rebus

AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators

Jingwei Ni, Minjing Shi, Dominik Stammbach, Mrinmaya Sachan, Elliott Ash, Markus Leippold

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments ACL2024 Main Conference

The Critique of Critique

Shichao Sun, Junlong Li, Weizhe Yuan, Ruifeng Yuan, Wenjie Li, Pengfei Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to Findings of ACL 2024

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

Albert Gu, Tri Dao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLM

Eliya Nachmani, Alon Levkovitch, Roy Hirsch, Julian Salazar, Chulayuth Asawaroengchai, Soroosh Mariooryad, Ehud Rivlin, RJ Skerry-Ryan, Michelle Tadmor Ramanovich

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2024 camera-ready

X-VILA: Cross-Modality Alignment for Large Language Model

Hanrong Ye, De-An Huang, Yao Lu, Zhiding Yu, Wei Ping, Andrew Tao, Jan Kautz, Song Han, Dan Xu, Pavlo Molchanov, Hongxu Yin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Technical Report

SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models

Lee Hyun, Kim Sung-Bin, Seungju Han, Youngjae Yu, Tae-Hyun Oh

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 14 figures

Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction

Tingchen Fu, Deng Cai, Lemao Liu, Shuming Shi, Rui Yan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to the findings of ACL2024

IM-RAG: Multi-Round Retrieval-Augmented Generation Through Learning Inner Monologues

Diji Yang, Jinmeng Rao, Kezhen Chen, Xiaoyuan Guo, Yawen Zhang, Jie Yang, Yi Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Proceedings of the 47th International ACM SIGIR 2024

KG-RAG: Bridging the Gap Between Knowledge and Creativity

Diego Sanmartin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Acquiring and Modelling Abstract Commonsense Knowledge via Conceptualization

Mutian He, Tianqing Fang, Weiqi Wang, Yangqiu Song

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 43 pages, 11 figures; Accepted by Artificial Intelligence

GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving

Jiaxin Zhang, Zhongzhi Li, Mingliang Zhang, Fei Yin, Chenglin Liu, Yashar Moshfeghi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted in ACL 2024 Findings

Benchmarking Large Language Models on CFLUE -- A Chinese Financial Language Understanding Evaluation Dataset

Jie Zhu, Junhui Li, Yalong Wen, Lifan Guo

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024

Journal ref The 62nd Annual Meeting of the Association for Computational Linguistics(ACL),2024

IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Control

Rohan Chitnis, Yingchen Xu, Bobak Hashemi, Lucas Lehnert, Urun Dogan, Zheqing Zhu, Olivier Delalleau

专题命中 推理评测 ：planning(abstract)；分类 cs.AI、cs.LG

Journal ref Short version published at ICRA 2024 (https://tinyurl.com/icra24-iqltdmpc)

MedConceptsQA: Open Source Medical Concepts QA Benchmark

Ofir Ben Shoham, Nadav Rappoport

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG