语言大模型 / LLM

TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models

Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Haotian Wang, Ming Liu, Bing Qin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2024

Muffin or Chihuahua? Challenging Multimodal Large Language Models with Multipanel VQA

Yue Fan, Jing Gu, Kaiwen Zhou, Qianqi Yan, Shan Jiang, Ching-Chen Kuo, Xinze Guan, Xin Eric Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2024

Improving Entity Recognition Using Ensembles of Deep Learning and Fine-tuned Large Language Models: A Case Study on Adverse Event Extraction from Multiple Sources

Yiming Li, Deepthi Viswaroopan, William He, Jianfu Li, Xu Zuo, Hua Xu, Cui Tao

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Self-assessment, Exhibition, and Recognition: a Review of Personality in Large Language Models

Zhiyuan Wen, Yu Yang, Jiannong Cao, Haoming Sun, Ruosong Yang, Shuaiqi Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming

Simone Tedeschi, Felix Friedrich, Patrick Schramowski, Kristian Kersting, Roberto Navigli, Huu Nguyen, Bo Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments 17 pages, preprint

GAugLLM: Improving Graph Contrastive Learning for Text-Attributed Graphs with Large Language Models

Yi Fang, Dongzhe Fan, Daochen Zha, Qiaoyu Tan

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Full-ECE: A Metric For Token-level Calibration on Large Language Models

Han Liu, Yupeng Zhang, Bingning Wang, Weipeng Chen, Xiaolin Hu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

KGPA: Robustness Evaluation for Large Language Models via Cross-Domain Knowledge Graphs

Aihua Pei, Zehua Yang, Shunan Zhu, Ruoxi Cheng, Ju Jia, Lina Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

Tianle Gu, Zeyang Zhou, Kexin Huang, Dandan Liang, Yixu Wang, Haiquan Zhao, Yuanqi Yao, Xingge Qiao, Keqing Wang, Yujiu Yang, Yan Teng, Yu Qiao, Yingchun Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models

Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, Qing Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments This is the long version of the corresponding survey paper accepted by KDD2024

Multi-Modal Retrieval For Large Language Model Based Speech Recognition

Jari Kolehmainen, Aditya Gourav, Prashanth Gurunath Shivakumar, Yile Gu, Ankur Gandhe, Ariya Rastrow, Grant Strimel, Ivan Bulyko

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Optimizing Large Language Models for OpenAPI Code Completion

Bohdan Petryshyn, Mantas Lukoševičius

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments Update: a better quality and readability of figures, better explanation of code infilling and document splitting in training, some text polishing, making it more compact

Can I understand what I create? Self-Knowledge Evaluation of Large Language Models

Zhiquan Tan, Lai Wei, Jindong Wang, Xing Xie, Weiran Huang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Benchmarking Large Language Models on Communicative Medical Coaching: a Novel System and Dataset

Hengguan Huang, Songtao Wang, Hongfu Liu, Hao Wang, Ye Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to Findings of the Association for Computational Linguistics: ACL 2024

Supervised Learning and Large Language Model Benchmarks on Mental Health Datasets: Cognitive Distortions and Suicidal Risks in Chinese Social Media

Hongzhi Qi, Qing Zhao, Jianqiang Li, Changwei Song, Wei Zhai, Dan Luo, Shuo Liu, Yi Jing Yu, Fan Wang, Huijing Zou, Bing Xiang Yang, Guanghui Fu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments 10 pages

The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models

Noah Y. Siegel, Oana-Maria Camburu, Nicolas Heess, Maria Perez-Ortiz

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments To be published in ACL 2024. 19 pages, 2 figures

CLOMO: Counterfactual Logical Modification with Large Language Models

Yinya Huang, Ruixin Hong, Hongming Zhang, Wei Shao, Zhicheng Yang, Dong Yu, Changshui Zhang, Xiaodan Liang, Linqi Song

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Journal ref Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL 2024)

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, Ion Stoica

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments Website - https://livecodebench.github.io/

FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Models

Andrew Zhu, Alyssa Hwang, Liam Dugan, Chris Callison-Burch

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 2 figures. ACL 2024

The Revolution of Multimodal Large Language Models: A Survey

Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2024 (Findings)

PatentGPT: A Large Language Model for Intellectual Property

Zilong Bai, Ruiji Zhang, Linqing Chen, Qijun Cai, Yuan Zhong, Cong Wang, Yan Fang, Jie Fang, Jing Sun, Weikuan Wang, Lizhi Zhou, Haoran Hua, Tian Qiu, Chaochao Wang, Cheng Sun, Jianping Lu, Yixin Wang, Yubin Xia, Meng Hu, Haowen Liu, Peng Xu, Licong Xu, Fu Bian, Xiaolong Gu, Lisha Zhang, Weilei Wang, Changyang Tu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 9 figures

PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models

Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, Linqi Song

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2024 main conference

Exploring Vulnerabilities and Protections in Large Language Models: A Survey

Frank Weizhen Liu, Chenhui Hu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters

Haibo Jin, Andy Zhou, Joe D. Menke, Haohan Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments 20 pages

Understanding Intrinsic Socioeconomic Biases in Large Language Models

Mina Arzaghi, Florian Carichon, Golnoosh Farnadi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Human-like Category Learning by Injecting Ecological Priors from Large Language Models into Neural Networks

Akshay K. Jagadish, Julian Coda-Forno, Mirko Thalmann, Eric Schulz, Marcel Binz

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Comments 27 pages (9 pages of main text, 4 pages of references, and 14 pages of appendix), 13 figures, and 7 Tables

Journal ref Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024

SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generation

Ziqin Luo, Haixia Han, Haokun Zhao, Guochao Jiang, Chengyu Du, Tingyun Li, Jiaqing Liang, Deqing Yang, Yanghua Xiao

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments The relevant code will be released in subsequent versions

Machine Unlearning in Large Language Models

Saaketh Koundinya Gundavarapu, Shreya Agarwal, Arushi Arora, Chandana Thimmalapura Jagadeeshaiah

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages

Prompt-Time Ontology-Driven Symbolic Knowledge Capture with Large Language Models

Tolga Çöplü, Arto Bendiken, Andrii Skomorokhov, Eduard Bateiko, Stephen Cobb

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 7 pages, 5 figures

Evaluating Large Language Models with Human Feedback: Establishing a Swedish Benchmark

Birger Moell

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI