语言大模型 / LLM

Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models

Pat Verga, Sebastian Hofstatter, Sophia Althammer, Yixuan Su, Aleksandra Piktus, Arkady Arkhangorodsky, Minjie Xu, Naomi White, Patrick Lewis

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions

Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F. Wong, Lidia S. Chao

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

A Survey on Prompting Techniques in LLMs

Prabin Bhandari

专题命中 评测与基准 ：prompting(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 4 Figures

Towards Measuring the Representation of Subjective Global Opinions in Language Models

Esin Durmus, Karina Nguyen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, Deep Ganguli

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?

Omid Ghahroodi, Marzia Nouri, Mohammad Vali Sanian, Alireza Sahebi, Doratossadat Dastgheib, Ehsaneddin Asgari, Mahdieh Soleymani Baghshah, Mohammad Hossein Rohban

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agents

Luca Gioacchini, Giuseppe Siracusano, Davide Sanvito, Kiril Gashteovski, David Friede, Roberto Bifulco, Carolin Lawrence

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2024)

Towards Objectively Benchmarking Social Intelligence for Language Agents at Action Level

Chenxu Wang, Bin Dai, Huaping Liu, Baoyuan Wang

专题命中 评测与基准 ：language agent(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response

Zihao Deng, Yinghao Ma, Yudong Liu, Rongchen Guo, Ge Zhang, Wenhu Chen, Wenhao Huang, Emmanouil Benetos

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Journal ref 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics

LLM-RadJudge: Achieving Radiologist-Level Evaluation for X-Ray Report Generation

Zilong Wang, Xufang Luo, Xinyang Jiang, Dongsheng Li, Lili Qiu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 6 figures

An Examination of the Compositionality of Large Generative Vision-Language Models

Teli Ma, Rong Li, Junwei Liang

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI

RouterBench: A Benchmark for Multi-LLM Routing System

Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, Shriyash Kaustubh Upadhyay

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection

Zhixin Lai, Xuesheng Zhang, Suiyao Chen

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification

Reza Esfandiarpoor, Stephen H. Bach

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2024

Elephants Never Forget: Testing Language Models for Memorization of Tabular Data

Sebastian Bordt, Harsha Nori, Rich Caruana

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Table Representation Learning Workshop at NeurIPS 2023

Prometheus: Inducing Fine-grained Evaluation Capability in Language Models

Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, Minjoon Seo

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2024

LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica, Hao Zhang

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Exploring LLM-based Agents for Root Cause Analysis

Devjeet Roy, Xuchao Zhang, Rashi Bhave, Chetan Bansal, Pedro Las-Casas, Rodrigo Fonseca, Saravan Rajmohan

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Automatic Histograms: Leveraging Language Models for Text Dataset Exploration

Emily Reif, Crystal Qian, James Wexler, Minsuk Kahng

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

SPML: A DSL for Defending Language Models Against Prompt Attacks

Reshabh K Sharma, Vinayak Gupta, Dan Grossman

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Dissecting Human and LLM Preferences

Junlong Li, Fan Zhou, Shichao Sun, Yikai Zhang, Hai Zhao, Pengfei Liu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Using Zero-shot Prompting in the Automatic Creation and Expansion of Topic Taxonomies for Tagging Retail Banking Transactions

Daniel de S. Moraes, Pedro T. C. Santos, Polyana B. da Costa, Matheus A. S. Pinto, Ivan de J. P. Pinto, Álvaro M. G. da Veiga, Sergio Colcher, Antonio J. G. Busson, Rafael H. Rocha, Rennan Gaio, Rafael Miceli, Gabriela Tourinho, Marcos Rabaioli, Leandro Santos, Fellipe Marques, David Favaro

专题命中 评测与基准 ：prompting(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergün, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Minh Chien, Sebastian Ruder, Surya Guthikonda, Emad A. Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, Sara Hooker

专题命中 评测与基准 ：instruction tuning(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Enhancing the Stability of LLM-based Speech Generation Systems through Self-Supervised Representations

Álvaro Martín-Cortinas, Daniel Sáez-Trigueros, Iván Vallés-Pérez, Biel Tura-Vecino, Piotr Biliński, Mateusz Lajszczak, Grzegorz Beringer, Roberto Barra-Chicote, Jaime Lorenzo-Trueba

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 10 pages, 1 figure, 3 tables

Vi(E)va LLM! A Conceptual Stack for Evaluating and Interpreting Generative AI-based Visualizations

Luca Podo, Muhammad Ishmal, Marco Angelini

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

HR-MultiWOZ: A Task Oriented Dialogue (TOD) Dataset for HR LLM Agent

Weijie Xu, Zicheng Huang, Wenxiang Hu, Xi Fang, Rajesh Kumar Cherukuri, Naumaan Nayyar, Lorenzo Malandri, Srinivasan H. Sengamedu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 13 pages, 9 figures

Journal ref EACL 2024

Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosis

Haochun Wang, Sendong Zhao, Zewen Qiang, Nuwa Xi, Bing Qin, Ting Liu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents

Shaoguang Mao, Yuzhe Cai, Yan Xia, Wenshan Wu, Xun Wang, Fengyi Wang, Tao Ge, Furu Wei

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

An Exploratory Assessment of LLM's Potential Toward Flight Trajectory Reconstruction Analysis

Qilei Zhang, John H. Mott

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 6 pages

Self-Assessment Tests are Unreliable Measures of LLM Personality

Akshat Gupta, Xiaoyang Song, Gopala Anumanchipalli

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, Ion Stoica

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2023 Datasets and Benchmarks Track