语言大模型 / LLM

From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning

Nan Xu, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NAACL 2025

Enhancing Knowledge Retrieval with In-Context Learning and Semantic Search through Generative AI

Mohammed-Khalil Ghali, Abdelrahman Farrag, Daehan Won, Yu Jin

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Automatic Labelling with Open-source LLMs using Dynamic Label Schema Integration

Thomas Walshe, Sae Young Moon, Chunyang Xiao, Yawwani Gunawardana, Fran Silavong

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 11 pages, 1 figure

Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code

Taishi Nakamura, Mayank Mishra, Simone Tedeschi, Yekun Chai, Jason T Stillerman, Felix Friedrich, Prateek Yadav, Tanmay Laud, Vu Minh Chien, Terry Yue Zhuo, Diganta Misra, Ben Bogin, Xuan-Son Vu, Marzena Karpinska, Arnav Varma Dantuluri, Wojciech Kusa, Tommaso Furlanello, Rio Yokota, Niklas Muennighoff, Suhas Pai, Tosin Adewumi, Veronika Laippala, Xiaozhe Yao, Adalberto Junior, Alpay Ariyak, Aleksandr Drozd, Jordan Clive, Kshitij Gupta, Liangyu Chen, Qi Sun, Ken Tsui, Noah Persaud, Nour Fahmy, Tianlong Chen, Mohit Bansal, Nicolo Monti, Tai Dang, Ziyang Luo, Tien-Tung Bui, Roberto Navigli, Virendra Mehta, Matthew Blumberg, Victor May, Huu Nguyen, Sampo Pyysalo

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Preprint

LLaVA-SLT: Visual Language Tuning for Sign Language Translation

Han Liang, Chengyu Huang, Yuecheng Xu, Cheng Tang, Weicai Ye, Juze Zhang, Xin Chen, Jingyi Yu, Lan Xu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data

Johannes Treutlein, Dami Choi, Jan Betley, Samuel Marks, Cem Anil, Roger Grosse, Owain Evans

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024. 10 pages, 8 figures

Is Programming by Example solved by LLMs?

Wen-Ding Li, Kevin Ellis

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

SecEncoder: Logs are All You Need in Security

Muhammed Fatih Bulut, Yingqi Liu, Naveed Ahmad, Maximilian Turner, Sami Ait Ouahmane, Cameron Andrews, Lloyd Greenwald

专题命中 预训练与数据 ：language model(abstract)；small language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

A Multilingual Sentiment Lexicon for Low-Resource Language Translation using Large Languages Models and Explainable AI

Melusi Malinga, Isaac Lupanda, Mike Wa Nkongolo, Phil van Deventer

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments This work is part of a PhD proposal in Information Technology at the University of Pretoria, supervised by Dr. Mike Wa Nkongolo and co-supervised by Dr. Phil van Deventer, under the Low-Resource Language Processing Lab in the Department of Informatics

Extracting Unlearned Information from LLMs with Activation Steering

Atakan Seyitoğlu, Aleksei Kuvshinov, Leo Schwinn, Stephan Günnemann

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024 Workshop Safe Generative AI

A Synthetic Dataset for Personal Attribute Inference

Hanna Yukhymenko, Robin Staab, Mark Vero, Martin Vechev

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Montessori-Instruct: Generate Influential Training Data Tailored for Student Learning

Xiaochuan Li, Zichun Yu, Chenyan Xiong

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Codes and data are open-sourced at https://github.com/cxcscmu/Montessori-Instruct

Can LLMs Reliably Simulate Human Learner Actions? A Simulation Authoring Framework for Open-Ended Learning Environments

Amogh Mannekote, Adam Davies, Jina Kang, Kristy Elizabeth Boyer

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text

Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, Tom Goldstein

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 20 pages, code available at https://github.com/ahans30/Binoculars

Aligners: Decoupling LLMs and Alignment

Lilian Ngweta, Mayank Agarwal, Subha Maity, Alex Gittens, Yuekai Sun, Mikhail Yurochkin

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Short version accepted as a Tiny Paper at the International Conference on Learning Representations (ICLR) 2024. Long version accepted to the Conference on Empirical Methods in Natural Language Processing (EMNLP) 2024 Findings

HexaCoder: Secure Code Generation via Oracle-Guided Synthetic Training Data

Hossein Hajipour, Lea Schönherr, Thorsten Holz, Mario Fritz

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 24 pages, 16 tables, 8 figures

OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving

Julong Wei, Shanshuai Yuan, Pengfei Li, Qingda Hu, Zhongxue Gan, Wenchao Ding

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Better Alignment with Instruction Back-and-Forth Translation

Thao Nguyen, Jeffrey Li, Sewoong Oh, Ludwig Schmidt, Jason Weston, Luke Zettlemoyer, Xian Li

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AgentInstruct: Toward Generative Teaching with Agentic Flows

Arindam Mitra, Luciano Del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Codas, Yadong Lu, Wei-ge Chen, Olga Vrousgos, Corby Rosset, Fillipe Silva, Hamed Khanpour, Yash Lara, Ahmed Awadallah

专题命中 预训练与数据 ：language model(abstract)；instruction tuning(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation

Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, Zehuan Yuan

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments Codes and models: \url{https://github.com/FoundationVision/LlamaGen}

MultiLegalPile: A 689GB Multilingual Legal Corpus

Joel Niklaus, Veton Matoshi, Matthias Stürmer, Ilias Chalkidis, Daniel E. Ho

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL 2024

Integrating Language-Derived Appearance Elements with Visual Cues in Pedestrian Detection

Sungjune Park, Hyunjun Kim, Yong Man Ro

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

EduAgent: Generative Student Agents in Learning

Songlin Xu, Xinyu Zhang, Lianhui Qin

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI、cs.LG

Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies

Anaelia Ovalle, Ninareh Mehrabi, Palash Goyal, Jwala Dhamala, Kai-Wei Chang, Richard Zemel, Aram Galstyan, Yuval Pinter, Rahul Gupta

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to NAACL 2024 findings

CapsFusion: Rethinking Image-Text Data at Scale

Qiying Yu, Quan Sun, Xiaosong Zhang, Yufeng Cui, Fan Zhang, Yue Cao, Xinlong Wang, Jingjing Liu

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments CVPR 2024. Code & Dataset: https://github.com/baaivision/CapsFusion

DoGE: Domain Reweighting with Generalization Estimation

Simin Fan, Matteo Pagliardini, Martin Jaggi

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking

Atharva Kulkarni, Bo-Hsiang Tseng, Joel Ruben Antony Moniz, Dhivya Piraviperumal, Hong Yu, Shruti Bhargava

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages. 4 figures, EACL 2024 main conference

Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning

Yanda Chen, Chandan Singh, Xiaodong Liu, Simiao Zuo, Bin Yu, He He, Jianfeng Gao

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments arXiv admin note: text overlap with arXiv:2307.08678

Improving In-Context Learning in Diffusion Models with Visual Context-Modulated Prompts

Tianqi Chen, Yongfei Liu, Zhendong Wang, Jianbo Yuan, Quanzeng You, Hongxia Yang, Mingyuan Zhou

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；pretraining(abstract)

Quantifying the redundancy between prosody and text

Lukas Wolf, Tiago Pimentel, Evelina Fedorenko, Ryan Cotterell, Alex Warstadt, Ethan Wilcox, Tamar Regev

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)