语言大模型 / LLM - arXivDaily 专题

2309.06085 2023-09-20 cs.CL 88%

BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models

Wei Qi Leong, Jian Gang Ngui, Yosephine Susanto, Hamsawardhini Rengarajan, Kengatharaiyer Sarveswaran, William Chandra Tjhi

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 86 pages, 7 figures, added link to repository in abstract, minor formatting changes and typo corrections

2309.09727 2023-09-19 cs.DL cs.CL 88%

When Large Language Models Meet Citation: A Survey

Yang Zhang, Yufei Wang, Kai Wang, Quan Z. Sheng, Lina Yao, Adnan Mahmood, Wei Emma Zhang, Rongying Zhao

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2304.00723 2023-09-19 cs.CL 88%

Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study

Yi Chen, Rui Wang, Haiyun Jiang, Shuming Shi, Ruifeng Xu

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2309.02077 2023-09-06 cs.CL 88%

An Automatic Evaluation Framework for Multi-turn Medical Consultations Capabilities of Large Language Models

Yusheng Liao, Yutong Meng, Hongcheng Liu, Yanfeng Wang, Yu Wang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 10 pages, 9figures

2309.01125 2023-09-06 cs.LG 88%

AutoML-GPT: Large Language Model for AutoML

Yun-Da Tsai, Yu-Che Tsai, Bo-Wei Huang, Chun-Pai Yang, Shou-De Lin

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

2308.14353 2023-08-29 cs.CL 88%

ZhuJiu: A Multi-dimensional, Multi-faceted Chinese Benchmark for Large Language Models

Baoli Zhang, Haining Xie, Pengfan Du, Junhao Chen, Pengfei Cao, Yubo Chen, Shengping Liu, Kang Liu, Jun Zhao

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2307.08487 2023-08-29 cs.CL 88%

Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models

Huachuan Qiu, Shuai Zhang, Anqi Li, Hongliang He, Zhenzhong Lan

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Code and data are available at https://github.com/qiuhuachuan/latent-jailbreak

2308.12415 2023-08-25 cs.SE cs.AI 88%

Benchmarking Causal Study to Interpret Large Language Models for Source Code

Daniel Rodriguez-Cardenas, David N. Palacio, Dipin Khati, Henry Burke, Denys Poshyvanyk

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 6 pages, 4 tables, 3 figures, accepted to ICSME 2023

2308.10053 2023-08-22 cs.IR cs.AI 88%

Large Language Models as Zero-Shot Conversational Recommenders

Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, Julian McAuley

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments Accepted as CIKM 2023 long paper. Longer version is coming soon (e.g., more details about dataset)

2308.09597 2023-08-21 cs.CL cs.HC 88%

ChatHaruhi: Reviving Anime Character in Reality via Large Language Model

Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi MI, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, Haozhen Sun

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments v1 - First version of techique report

2308.09115 2023-08-21 cs.CL cond-mat.mtrl-sci 88%

MaScQA: A Question Answering Dataset for Investigating Materials Science Knowledge of Large Language Models

Mohd Zaki, Jayadeva, Mausam, N. M. Anoop Krishnan

专题命中评测与基准：language model(title,abstract)；large language model(title)；prompting(abstract)；分类 cs.CL

2308.06261 2023-08-14 cs.NI cs.AI 88%

Enhancing Network Management Using Code Generated by Large Language Models

Sathiya Kumaran Mani, Yajie Zhou, Kevin Hsieh, Santiago Segarra, Ranveer Chandra, Srikanth Kandula

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

2308.05596 2023-08-11 cs.CL cs.SI 88%

You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content

Xinlei He, Savvas Zannettou, Yun Shen, Yang Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments To Appear in the 45th IEEE Symposium on Security and Privacy, May 20-23, 2024

2304.11679 2023-08-11 cs.CL 88%

Domain Mastery Benchmark: An Ever-Updating Benchmark for Evaluating Holistic Domain Knowledge of Large Language Model--A Preliminary Release

Zhouhong Gu, Xiaoxuan Zhu, Haoning Ye, Lin Zhang, Zhuozhi Xiong, Zihan Li, Qianyu He, Sihang Jiang, Hongwei Feng, Yanghua Xiao

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments The paper is updated, but we make a mistake that submit a new arxiv paper but not replace this one, the new version is in arXiv:2306.05783

2307.09751 2023-07-28 cs.IR cs.AI 88%

Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community

Qingyao Ai, Ting Bai, Zhao Cao, Yi Chang, Jiawei Chen, Zhumin Chen, Zhiyong Cheng, Shoubin Dong, Zhicheng Dou, Fuli Feng, Shen Gao, Jiafeng Guo, Xiangnan He, Yanyan Lan, Chenliang Li, Yiqun Liu, Ziyu Lyu, Weizhi Ma, Jun Ma, Zhaochun Ren, Pengjie Ren, Zhiqiang Wang, Mingwen Wang, Ji-Rong Wen, Le Wu, Xin Xin, Jun Xu, Dawei Yin, Peng Zhang, Fan Zhang, Weinan Zhang, Min Zhang, Xiaofei Zhu

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 17 pages

2301.09790 2023-07-25 cs.CL 88%

The Next Chapter: A Study of Large Language Models in Storytelling

Zhuohan Xie, Trevor Cohn, Jey Han Lau

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted to INLG2023

2307.10778 2023-07-21 cs.CL 88%

Extreme Multi-Label Skill Extraction Training using Large Language Models

Jens-Joris Decorte, Severine Verlinden, Jeroen Van Hautte, Johannes Deleu, Chris Develder, Thomas Demeester

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted to the International workshop on AI for Human Resources and Public Employment Services (AI4HR&PES) as part of ECML-PKDD 2023

2307.02288 2023-07-21 cs.CL cs.HC 88%

Performance Comparison of Large Language Models on VNHSGE English Dataset: OpenAI ChatGPT, Microsoft Bing Chat, and Google Bard

Xuan-Quy Dao

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 11 pages, 8 figures

2307.09705 2023-07-20 cs.CL 88%

CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility

Guohai Xu, Jiayi Liu, Ming Yan, Haotian Xu, Jinghui Si, Zhuoran Zhou, Peng Yi, Xing Gao, Jitao Sang, Rong Zhang, Ji Zhang, Chao Peng, Fei Huang, Jingren Zhou

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Working in Process

2304.08763 2023-07-17 cs.CL 88%

A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Models

Qianqian Xie, Zheheng Luo, Benyou Wang, Sophia Ananiadou

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 30 pages, 5 figures

2305.11991 2023-07-06 cs.CL 88%

Evaluation of medium-large Language Models at zero-shot closed book generative question answering

René Peinl, Johannes Wirth

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Journal ref Under review in ARIA 2023

2305.12199 2023-07-05 cs.CL 88%

VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models

Xuan-Quy Dao, Ngoc-Bich Le, The-Duy Vo, Xuan-Dung Phan, Bac-Bien Ngo, Van-Tien Nguyen, Thi-My-Thanh Nguyen, Hong-Phuoc Nguyen

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 74 pages, 44 figures

2304.10691 2023-06-09 eess.IV cs.CV cs.LG 88%

SkinGPT-4: An Interactive Dermatology Diagnostic System with Visual Large Language Model

Juexiao Zhou, Xiaonan He, Liyuan Sun, Jiannan Xu, Xiuying Chen, Yuetan Chu, Longxi Zhou, Xingyu Liao, Bin Zhang, Xin Gao

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

2306.04556 2023-06-08 cs.LG cs.HC cs.SE 88%

StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code

Hannah McLean Babe, Sydney Nguyen, Yangtian Zi, Arjun Guha, Molly Q Feldman, Carolyn Jane Anderson

专题命中评测与基准：large language model(title)；language model(title)；LLM(abstract)；prompting(abstract)

2306.03203 2023-06-07 cs.CL cs.SE 88%

A Static Evaluation of Code Completion by Large Language Models

Hantian Ding, Varun Kumar, Yuchen Tian, Zijian Wang, Rob Kwiatkowski, Xiaopeng Li, Murali Krishna Ramanathan, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta, Dan Roth, Bing Xiang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL 2023 industry track

2303.02927 2023-06-07 cs.AI cs.HC cs.PL 88%

LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models

Victor Dibia

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments Accepted at ACL 2023 (Demonstration track). Fix formatting issues, update information on evaluation metrics, prompts and project website (https://microsoft.github.io/lida/)

2305.19821 2023-06-01 cs.CL cs.CV 88%

LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting

Rita Ramos, Bruno Martins, Desmond Elliott

专题命中评测与基准：language model(title,abstract)；prompting(title,abstract)；分类 cs.CL

Comments To appear in the Findings of ACL 2023

2305.10843 2023-05-29 cs.CV cs.AI 88%

X-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models

Yixiong Chen, Li Liu, Chris Ding

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 18 pages, 6 tables, 11 figures, NeurIPS 2023 submission

2305.10263 2023-05-23 cs.CL 88%

M3KE: A Massive Multi-Level Multi-Subject Knowledge Evaluation Benchmark for Chinese Large Language Models

Chuang Liu, Renren Jin, Yuqi Ren, Linhao Yu, Tianyu Dong, Xiaohan Peng, Shuting Zhang, Jianxiang Peng, Peiyi Zhang, Qingqing Lyu, Xiaowen Su, Qun Liu, Deyi Xiong

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2305.10645 2023-05-22 cs.CL 88%

Are Large Language Models Fit For Guided Reading?

Peter Ochieng

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

1. 评测与基准 32208 篇

BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models

When Large Language Models Meet Citation: A Survey

Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study

An Automatic Evaluation Framework for Multi-turn Medical Consultations Capabilities of Large Language Models

AutoML-GPT: Large Language Model for AutoML

ZhuJiu: A Multi-dimensional, Multi-faceted Chinese Benchmark for Large Language Models

Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models

Benchmarking Causal Study to Interpret Large Language Models for Source Code

Large Language Models as Zero-Shot Conversational Recommenders

ChatHaruhi: Reviving Anime Character in Reality via Large Language Model

MaScQA: A Question Answering Dataset for Investigating Materials Science Knowledge of Large Language Models

Enhancing Network Management Using Code Generated by Large Language Models

You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content

Domain Mastery Benchmark: An Ever-Updating Benchmark for Evaluating Holistic Domain Knowledge of Large Language Model--A Preliminary Release

Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community

The Next Chapter: A Study of Large Language Models in Storytelling

Extreme Multi-Label Skill Extraction Training using Large Language Models

Performance Comparison of Large Language Models on VNHSGE English Dataset: OpenAI ChatGPT, Microsoft Bing Chat, and Google Bard

CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility

A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Models

Evaluation of medium-large Language Models at zero-shot closed book generative question answering

VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models

SkinGPT-4: An Interactive Dermatology Diagnostic System with Visual Large Language Model

StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code

A Static Evaluation of Code Completion by Large Language Models

LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models

LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting

X-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models

M3KE: A Massive Multi-Level Multi-Subject Knowledge Evaluation Benchmark for Chinese Large Language Models

Are Large Language Models Fit For Guided Reading?