语言大模型 / LLM

Building a Large Japanese Web Corpus for Large Language Models

Naoaki Okazaki, Kakeru Hattori, Hirai Shota, Hiroki Iida, Masanari Ohi, Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Rio Yokota, Sakae Mizuki

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 17 pages

An Integrated Data Processing Framework for Pretraining Foundation Models

Yiding Sun, Feng Wang, Yutao Zhu, Wayne Xin Zhao, Jiaxin Mao

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments 6 pages, 2 figures; accepted by SIGIR'24 demo track

Investigating Data Contamination in Modern Benchmarks for Large Language Models

Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, Arman Cohan

专题命中 预训练与数据 ：large language model(title)；language model(title)；pretraining(abstract)；prompting(abstract)

Comments NAACL 2024 Version

Unlocking Emergent Modularity in Large Language Models

Zihan Qiu, Zeyu Huang, Jie Fu

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Comments NAACL2024 Main Conference

Journal ref 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics

MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansion

Pengyue Jia, Yiding Liu, Xiangyu Zhao, Xiaopeng Li, Changying Hao, Shuaiqiang Wang, Dawei Yin

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to NAACL 2024

Othering and low status framing of immigrant cuisines in US restaurant reviews and large language models

Yiwei Luo, Kristina Gligorić, Dan Jurafsky

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments ICWSM '24

Unsupervised Pretraining for Fact Verification by Language Model Distillation

Adrián Bazaga, Pietro Liò, Gos Micklem

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments ICLR 2024 Camera Ready

Large Language Models to Identify Social Determinants of Health in Electronic Health Records

Marco Guevara, Shan Chen, Spencer Thomas, Tafadzwa L. Chaunzwa, Idalid Franco, Benjamin Kann, Shalini Moningi, Jack Qian, Madeleine Goldstein, Susan Harper, Hugo JWL Aerts, Guergana K. Savova, Raymond H. Mak, Danielle S. Bitterman

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Peer-reviewed version published at NPJ Digital Medicine: https://www.nature.com/articles/s41746-023-00970-0

Journal ref NPJ Digit Med. 2024 Jan 11;7(1):6

STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models

Mingyu Derek Ma, Xiaoxuan Wang, Po-Nien Kung, P. Jeffrey Brantingham, Nanyun Peng, Wei Wang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments To appear at AAAI'24. More info is at https://derek.ma/STAR

Test-Time Backdoor Attacks on Multimodal Large Language Models

Dong Lu, Tianyu Pang, Chao Du, Qian Liu, Xianjun Yang, Min Lin

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model

Yongyu Yan, Kui Xue, Xiaoming Shi, Qi Ye, Jingping Liu, Tong Ruan

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.LG

LLaMandement: Large Language Models for Summarization of French Legislative Proposals

Joseph Gesnouin, Yannis Tannier, Christophe Gomes Da Silva, Hatim Tapory, Camille Brier, Hugo Simon, Raphael Rozenberg, Hermann Woehrel, Mehdi El Yakaabi, Thomas Binder, Guillaume Marie, Emilie Caron, Mathile Nogueira, Thomas Fontas, Laure Puydebois, Marie Theophile, Stephane Morandi, Mael Petit, David Creissac, Pauline Ennouchy, Elise Valetoux, Celine Visade, Severine Balloux, Emmanuel Cortes, Pierre-Etienne Devineau, Ulrich Tan, Esther Mac Namara, Su Yang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 21 pages, 9 figures

DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence

Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, Wenfeng Liang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Orion-14B: Open-source Multilingual Large Language Models

Du Chen, Yi Huang, Xiaopu Li, Yongqiang Li, Yongqiang Liu, Haihui Pan, Leichao Xu, Dacheng Zhang, Zhipeng Zhang, Kun Han

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments Authors are alphabetically listed by last names, except the corresponding author who is listed last

Synthetic Data Generation in Low-Resource Settings via Fine-Tuning of Large Language Models

Jean Kaddour, Qi Liu

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challenges

Xiaoqian Liu, Jianbin Jiao, Junge Zhang

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)；分类 cs.AI、cs.LG

YAYI 2: Multilingual Open-Source Large Language Models

Yin Luo, Qingchao Kong, Nan Xu, Jia Cao, Bao Hao, Baoyu Qu, Bo Chen, Chao Zhu, Chenyang Zhao, Donglei Zhang, Fan Feng, Feifei Zhao, Hailong Sun, Hanxuan Yang, Haojun Pan, Hongyu Liu, Jianbin Guo, Jiangtao Du, Jingyi Wang, Junfeng Li, Lei Sun, Liduo Liu, Lifeng Dong, Lili Liu, Lin Wang, Liwen Zhang, Minzheng Wang, Pin Wang, Ping Yu, Qingxiao Li, Rui Yan, Rui Zou, Ruiqun Li, Taiwen Huang, Xiaodong Wang, Xiaofei Wu, Xin Peng, Xina Zhang, Xing Fang, Xinglin Xiao, Yanni Hao, Yao Dong, Yigang Wang, Ying Liu, Yongyu Jiang, Yungan Wang, Yuqi Wang, Zhangsheng Wang, Zhaoxin Yu, Zhen Luo, Wenji Mao, Lei Wang, Dajun Zeng

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Exploring Distributional Shifts in Large Language Models for Code Analysis

Shushan Arakelyan, Rocktim Jyoti Das, Yi Mao, Xiang Ren

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Automatic detection of problem-gambling signs from online texts using large language models

Elke Smith, Nils Reiter, Jan Peters

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining

Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc V. Le, Tengyu Ma, Adams Wei Yu

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2023

Sabiá: Portuguese Large Language Models

Ramon Pires, Hugo Abonizio, Thales Sales Almeida, Rodrigo Nogueira

专题命中 预训练与数据 ：language model(title,abstract)；large language model(title)；pretraining(abstract)；分类 cs.CL、cs.AI

Robust Safety Classifier for Large Language Models: Adversarial Prompt Shield

Jinhwa Kim, Ali Derakhshan, Ian G. Harris

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 2 figures

Conceptor-Aided Debiasing of Large Language Models

Li S. Yifei, Lyle Ungar, João Sedoc

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 25 pages

T5 meets Tybalt: Author Attribution in Early Modern English Drama Using Large Language Models

Rebecca M. M. Hicke, David Mimno

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments Published in CHR 2023

Pretraining Language Models with Text-Attributed Heterogeneous Graphs

Tao Zou, Le Yu, Yifei Huang, Leilei Sun, Bowen Du

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2023 Findings

Challenges and Contributing Factors in the Utilization of Large Language Models (LLMs)

Xiaoliang Chen, Liangbin Li, Le Chang, Yunhe Huang, Yuxuan Zhao, Yuxiao Zhang, Dinuo Li

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Generative error correction for code-switching speech recognition using large language models

Chen Chen, Yuchen Hu, Chao-Han Huck Yang, Hexin Liu, Sabato Marco Siniscalchi, Eng Siong Chng

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Submitted to ICASSP2024

CarExpert: Leveraging Large Language Models for In-Car Conversational Question Answering

Md Rashad Al Hasan Rony, Christian Suess, Sinchana Ramakanth Bhat, Viju Sudhi, Julia Schneider, Maximilian Vogel, Roman Teucher, Ken E. Friedl, Soumya Sahoo

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted into EMNLP 2023 (industry track), corresponding Author: Md Rashad Al Hasan Rony

Procedural Text Mining with Large Language Models

Anisa Rula, Jennifer D'Souza

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 4 figures, Accepted to The Twelfth International Conference on Knowledge Capture (K-Cap 2023)

Gender bias and stereotypes in Large Language Models

Hadas Kotek, Rikker Dockum, David Q. Sun

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments ACM Collective Intelligence

Journal ref In Collective Intelligence Conference (CI '23), November 06-09, 2023, Delft, Netherlands. ACM, New York, NY, USA (2023)