语言大模型 / LLM - arXivDaily 专题

2308.15645 2023-12-29 cs.PL cs.AI cs.SE 89%

AskIt: Unified Programming Interface for Programming with Large Language Models

Katsumi Okuda, Saman Amarasinghe

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments To be published in 2024 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)

2312.15746 2023-12-27 cs.IR cs.AI 89%

Large Language Models are Not Stable Recommender Systems

Tianhui Ma, Yuan Cheng, Hengshu Zhu, Hui Xiong

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2312.07488 2023-12-22 cs.CV cs.AI cs.RO 89%

LMDrive: Closed-Loop End-to-End Driving with Large Language Models

Hao Shao, Yuxuan Hu, Letian Wang, Steven L. Waslander, Yu Liu, Hongsheng Li

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments project page: https://hao-shao.com/projects/lmdrive.html

2310.07521 2023-12-19 cs.CL 89%

Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity

Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, Yidong Wang, Linyi Yang, Jindong Wang, Xing Xie, Zheng Zhang, Yue Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 62 pages; 300+ references

2311.03731 2023-12-15 cs.CL 89%

A Survey of Large Language Models Attribution

Dongfang Li, Zetian Sun, Xinshuo Hu, Zhenyu Liu, Ziyang Chen, Baotian Hu, Aiguo Wu, Min Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2312.06002 2023-12-12 cs.CL 89%

Large Language Models on Lexical Semantic Change Detection: An Evaluation

Ruiyu Wang, Matthew Choi

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

2312.04019 2023-12-08 q-bio.BM cs.AI 89%

Efficiently Predicting Protein Stability Changes Upon Single-point Mutation with Large Language Models

Yijie Zhang, Zhangyang Gao, Cheng Tan, Stan Z. Li

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2211.08412 2023-12-05 cs.CL 89%

Evaluating the Factual Consistency of Large Language Models Through News Summarization

Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, Colin Raffel

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2312.00164 2023-12-04 cs.CY cs.AI 89%

Towards Accurate Differential Diagnosis with Large Language Models

Daniel McDuff, Mike Schaekermann, Tao Tu, Anil Palepu, Amy Wang, Jake Garrison, Karan Singhal, Yash Sharma, Shekoofeh Azizi, Kavita Kulkarni, Le Hou, Yong Cheng, Yun Liu, S Sara Mahdavi, Sushant Prakash, Anupam Pathak, Christopher Semturs, Shwetak Patel, Dale R Webster, Ewa Dominowska, Juraj Gottweis, Joelle Barral, Katherine Chou, Greg S Corrado, Yossi Matias, Jake Sunshine, Alan Karthikesalingam, Vivek Natarajan

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2311.18658 2023-12-01 cs.CL 89%

ArcMMLU: A Library and Information Science Benchmark for Large Language Models

Shitou Zhang, Zuchao Li, Xingshen Liu, Liming Yang, Ping Wang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2311.16103 2023-11-29 cs.CV cs.AI 89%

Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models

Munan Ning, Bin Zhu, Yujia Xie, Bin Lin, Jiaxi Cui, Lu Yuan, Dongdong Chen, Li Yuan

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments Benchmark is available at https://github.com/PKU-YuanGroup/Video-Bench

2310.00741 2023-11-29 cs.CL 89%

FELM: Benchmarking Factuality Evaluation of Large Language Models

Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, Junxian He

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by NeurIPS 2023 Track on Datasets and Benchmarks

2311.13053 2023-11-23 cs.CL 89%

Beyond Text: Unveiling Multimodal Proficiency of Large Language Models with MultiAPI Benchmark

Xiao Liu, Jianfeng Lin, Jiawei Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Work in Progress

2311.12338 2023-11-22 cs.IR cs.AI 89%

A Survey on Large Language Models for Personalized and Explainable Recommendations

Junyi Chen

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2311.09517 2023-11-17 cs.CL 89%

GEE! Grammar Error Explanation with Large Language Models

Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, Kevin Gimpel, Mohit Iyyer

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments Preprint, 24 pages, code and data available in https://github.com/Yixiao-Song/GEE-with-LLMs

2311.06838 2023-11-14 cs.CL 89%

GIELLM: Japanese General Information Extraction Large Language Model Utilizing Mutual Reinforcement Effect

Chengguang Gan, Qinghao Zhang, Tatsunori Mori

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 10 pages, 6 figures

2311.06697 2023-11-14 cs.CL 89%

Trusted Source Alignment in Large Language Models

Vasilisa Bashlovkina, Zhaobin Kuang, Riley Matthews, Edward Clifford, Yennie Jun, William W. Cohen, Simon Baumgartner

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2308.05345 2023-11-14 cs.LG cs.AR 89%

RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model

Yao Lu, Shang Liu, Qijun Zhang, Zhiyao Xie

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

Journal ref Asia and South Pacific Design Automation Conference (ASP-DAC) 2024

2307.03025 2023-11-14 cs.CL 89%

Style Over Substance: Evaluation Biases for Large Language Models

Minghao Wu, Alham Fikri Aji

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Work in progress, 15 pages, 5 tables, 12 figures

2309.00608 2023-11-10 cs.SE cs.LG cs.PL 89%

Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair

Yuxiang Wei, Chunqiu Steven Xia, Lingming Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

Comments Paper accepted at ESEC/FSE 2023

2311.03837 2023-11-08 cs.IR cs.CL 89%

OLaLa: Ontology Matching with Large Language Models

Sven Hertling, Heiko Paulheim

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments Accepted at K-CAP 2023 conference

2309.10160 2023-11-07 physics.med-ph cs.AI 89%

RadOnc-GPT: A Large Language Model for Radiation Oncology

Zhengliang Liu, Peilong Wang, Yiwei Li, Jason Holmes, Peng Shu, Lian Zhang, Chenbin Liu, Ninghao Liu, Dajiang Zhu, Xiang Li, Quanzheng Li, Samir H. Patel, Terence T. Sio, Tianming Liu, Wei Liu

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；分类 cs.AI

2309.03412 2023-11-07 cs.CL 89%

From Base to Conversational: Japanese Instruction Dataset and Tuning Large Language Models

Masahiro Suzuki, Masanori Hirano, Hiroki Sakaji

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；分类 cs.CL

Comments 10 pages, 1 figure, 2 tables. The paper is a camera-ready version of IEEE BigData 2023

2311.01152 2023-11-03 cs.CL 89%

Predicting Question-Answering Performance of Large Language Models through Semantic Consistency

Ella Rabinovich, Samuel Ackerman, Orna Raz, Eitan Farchi, Ateret Anaby-Tavor

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments EMNLP2023 GEM workshop, 17 pages

2311.00681 2023-11-02 cs.CL 89%

Are Large Language Models Reliable Judges? A Study on the Factuality Evaluation Capabilities of LLMs

Xue-Yong Fu, Md Tahmid Rahman Laskar, Cheng Chen, Shashi Bhushan TN

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments accepted by Generation, Evaluation & Metrics (GEM) Workshop at EMNLP 2023

2310.19737 2023-10-31 cs.AI 89%

Adversarial Attacks and Defenses in Large Language Models: Old and New Threats

Leo Schwinn, David Dobre, Stephan Günnemann, Gauthier Gidel

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2310.11638 2023-10-31 cs.CL 89%

Systematic Assessment of Factual Knowledge in Large Language Models

Linhao Luo, Thuy-Trang Vu, Dinh Phung, Gholamreza Haffari

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2023 Findings

2305.14802 2023-10-27 cs.CL 89%

Estimating Large Language Model Capabilities without Labeled Test Data

Harvey Yiyun Fu, Qinyuan Ye, Albert Xu, Xiang Ren, Robin Jia

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2023 Findings. Camera-ready version. Code: https://github.com/harvey-fin/icl-estimate

2310.15317 2023-10-25 cs.CL cs.CY 89%

Exploring the Potential of Large Language Models in Generating Code-Tracing Questions for Introductory Programming Courses

Aysa Xuemo Fan, Ranran Haoran Zhang, Luc Paquette, Rui Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by Findings of EMNLP, 2023

2306.03030 2023-10-24 cs.CL 89%

Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset

Junling Liu, Peilin Zhou, Yining Hua, Dading Chong, Zhongyu Tian, Andrew Liu, Helin Wang, Chenyu You, Zhenhua Guo, Lei Zhu, Michael Lingzhi Li

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by NeurIPS 2023 Datasets and Benchmarks Track

1. 评测与基准 32096 篇

AskIt: Unified Programming Interface for Programming with Large Language Models

Large Language Models are Not Stable Recommender Systems

LMDrive: Closed-Loop End-to-End Driving with Large Language Models

Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity

A Survey of Large Language Models Attribution

Large Language Models on Lexical Semantic Change Detection: An Evaluation

Efficiently Predicting Protein Stability Changes Upon Single-point Mutation with Large Language Models

Evaluating the Factual Consistency of Large Language Models Through News Summarization

Towards Accurate Differential Diagnosis with Large Language Models

ArcMMLU: A Library and Information Science Benchmark for Large Language Models

Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models

FELM: Benchmarking Factuality Evaluation of Large Language Models

Beyond Text: Unveiling Multimodal Proficiency of Large Language Models with MultiAPI Benchmark

A Survey on Large Language Models for Personalized and Explainable Recommendations

GEE! Grammar Error Explanation with Large Language Models

GIELLM: Japanese General Information Extraction Large Language Model Utilizing Mutual Reinforcement Effect

Trusted Source Alignment in Large Language Models

RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model

Style Over Substance: Evaluation Biases for Large Language Models

Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair

OLaLa: Ontology Matching with Large Language Models

RadOnc-GPT: A Large Language Model for Radiation Oncology

From Base to Conversational: Japanese Instruction Dataset and Tuning Large Language Models

Predicting Question-Answering Performance of Large Language Models through Semantic Consistency

Are Large Language Models Reliable Judges? A Study on the Factuality Evaluation Capabilities of LLMs

Adversarial Attacks and Defenses in Large Language Models: Old and New Threats

Systematic Assessment of Factual Knowledge in Large Language Models

Estimating Large Language Model Capabilities without Labeled Test Data

Exploring the Potential of Large Language Models in Generating Code-Tracing Questions for Introductory Programming Courses

Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset