语言大模型 / LLM

Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling

Zile Qiao, Wei Ye, Yong Jiang, Tong Mo, Pengjun Xie, Weiping Li, Fei Huang, Shikun Zhang

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Eliciting In-Context Learning in Vision-Language Models for Videos Through Curated Data Distributional Properties

Keunwoo Peter Yu, Zheyuan Zhang, Fengyuan Hu, Shane Storks, Joyce Chai

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, LaTeX; Accepted to EMNLP 2024 Main

Tracing Privacy Leakage of Language Models to Training Data via Adjusted Influence Functions

Jinxin Liu, Zao Yang

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Harnessing the Intrinsic Knowledge of Pretrained Language Models for Challenging Text Classification Settings

Lingyu Gao

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments PhD thesis

Bilingual Adaptation of Monolingual Foundation Models

Gurpreet Gosal, Yishi Xu, Gokul Ramakrishnan, Rituraj Joshi, Avraham Sheinin, Zhiming, Chen, Biswajit Mishra, Natalia Vassilieva, Joel Hestness, Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Onkar Pandit, Satheesh Katipomu, Samta Kamboj, Samujjwal Ghosh, Rahul Pal, Parvez Mullah, Soundar Doraiswamy, Mohamed El Karim Chami, Preslav Nakov

专题命中 预训练与数据 ：foundation model(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Building pre-train LLM Dataset for the INDIC Languages: a case study on Hindi

Shantipriya Parida, Shakshi Panwar, Kusum Lata, Sanskruti Mishra, Sambit Sekhar

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted as a book chapter in the book Title "APPLIED SPEECH AND TEXT PROCESSING FOR LOW RESOURCE LANGUAGES"

Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes

Nabeel Seedat, Nicolas Huynh, Boris van Breugel, Mihaela van der Schaar

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Presented at the 41st International Conference on Machine Learning (ICML) 2024. *Seedat & Huynh contributed equally

AST-T5: Structure-Aware Pretraining for Code Generation and Understanding

Linyuan Gong, Mostafa Elhoushi, Alvin Cheung

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 15 pages; ICML 2024: https://icml.cc/virtual/2024/poster/33601

Explore the Limits of Omni-modal Pretraining at Scale

Yiyuan Zhang, Handong Li, Jing Liu, Xiangyu Yue

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Project Website: https://invictus717.github.io/MiCo/

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models

Tianwen Wei, Bo Zhu, Liang Zhao, Cheng Cheng, Biye Li, Weiwei Lü, Peng Cheng, Jianhao Zhang, Xiaoyu Zhang, Liang Zeng, Xiaokun Wang, Yutuan Ma, Rui Hu, Shuicheng Yan, Han Fang, Yahui Zhou

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters

Yixin Song, Haotong Xie, Zhengyan Zhang, Bo Wen, Li Ma, Zeyu Mi, Haibo Chen

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Probing Language Models for Pre-training Data Detection

Zhenhua Liu, Tong Zhu, Chuanyuan Tan, Haonan Lu, Bing Liu, Wenliang Chen

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL-2024 main conference

DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows

Ajay Patel, Colin Raffel, Chris Callison-Burch

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Published in ACL 2024

GECKO: Generative Language Model for English, Code and Korean

Sungwoo Oh, Donggyu Kim

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning

Sanchit Sinha, Yuguang Yue, Victor Soto, Mayank Kulkarni, Jianhua Lu, Aidong Zhang

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments KDD 2024, 11 pages(9 main, 2 ref, 1 App) Openreview https://openreview.net/forum?id=JwecLNhWDy&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DKDD.org%2F2024%2FResearch_Track%2FAuthors%23your-submissions)

T-CLAP: Temporal-Enhanced Contrastive Language-Audio Pretraining

Yi Yuan, Zhuo Chen, Xubo Liu, Haohe Liu, Xuenan Xu, Dongya Jia, Yuanzhe Chen, Mark D. Plumbley, Wenwu Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Preprint submitted to IEEE MLSP 2024

TextGram: Towards a better domain-adaptive pretraining

Sharayu Hiwarkhedkar, Saloni Mittal, Vidula Magdum, Omkar Dhekane, Raviraj Joshi, Geetanjali Kale, Arnav Ladkat

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted at SPELLL 2023

Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities

Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Iida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, Naoaki Okazaki

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

LLM In-Context Recall is Prompt Dependent

Daniel Machlab, Rick Battle

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Language Models for Text Classification: Is In-Context Learning Enough?

Aleksandra Edwards, Jose Camacho-Collados

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments Accepted at LREC-COLING 2024

Calibrated Language Models Must Hallucinate

Adam Tauman Kalai, Santosh S. Vempala

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；post-training(abstract)；分类 cs.CL、cs.AI

Comments In Proceedings of the 56th Annual ACM Symposium on Theory of Computing (STOC) 2024

Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models

Martin Riddell, Ansong Ni, Arman Cohan

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Transformer-based Causal Language Models Perform Clustering

Xinbo Wu, Lav R. Varshney

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Added new experimental results and fixed some errors

Is It a Free Lunch for Removing Outliers during Pretraining?

Baohao Liao, Christof Monz

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 5 pages, 3 figures, 1 table

LAPDoc: Layout-Aware Prompting for Documents

Marcel Lamott, Yves-Noel Weweler, Adrian Ulges, Faisal Shafait, Dirk Krechel, Darko Obradovic

专题命中 预训练与数据 ：prompting(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Under review at ICDAR2024

Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seeking

Anjali Khurana, Hari Subramonyam, Parmit K Chilana

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted for publication in the Proceedings of the 29th International Conference on Intelligent User Interfaces (IUI'24), March 18--21, 2024, in Greenville, SC, USA

TAROT: A Hierarchical Framework with Multitask Co-Pretraining on Semi-Structured Data towards Effective Person-Job Fit

Yihan Cao, Xu Chen, Lun Du, Hao Chen, Qiang Fu, Shi Han, Yushu Du, Yanbin Kang, Guangming Lu, Zi Li

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments ICASSP 2024 camera ready. 5 pages, 1 figure, 3 tables

The Falcon Series of Open Language Models

Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, Guilherme Penedo

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

LLM aided semi-supervision for Extractive Dialog Summarization

Nishant Mishra, Gaurav Sahu, Iacer Calixto, Ameen Abu-Hanna, Issam H. Laradji

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments to be published in EMNLP Findings

Data Similarity is Not Enough to Explain Language Model Performance

Gregory Yauney, Emily Reif, David Mimno

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Journal ref Published in EMNLP 2023