语言大模型 / LLM - arXivDaily 专题

2403.09675 2024-03-18 cs.CV cs.GR 85%

Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases

Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stewart Morris, Seung Jean Yoo, Aditya Ganeshan, R. Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, Daniel Ritchie

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments See ancillary files for link to supplemental material

2401.15459 2024-03-13 cs.SE 85%

Multi-LLM Collaboration + Data-Centric Innovation = 2x Better Vulnerability Repair

Xin Zhou, Kisub Kim, Bowen Xu, DongGyun Han, David Lo

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments Accepted in the ICSE 2024 Research Track with a different title "Out of Sight, Out of Mind: Better Automatic Vulnerability Repair by Broadening Input Ranges and Sources"

2403.00871 2024-03-05 cs.CR cs.AI cs.CL cs.LG 85%

Teach LLMs to Phish: Stealing Private Information from Language Models

Ashwinee Panda, Christopher A. Choquette-Choo, Zhengming Zhang, Yaoqing Yang, Prateek Mittal

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2024

2310.06714 2024-02-26 cs.AI cs.CL cs.LG 85%

Exploring Memorization in Fine-tuned Language Models

Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, Dawei Yin

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.11777 2024-02-20 cs.CL cs.AI cs.LG 85%

Uncovering Latent Human Wellbeing in Language Model Embeddings

Pedro Freire, ChengCheng Tan, Adam Gleave, Dan Hendrycks, Scott Emmons

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 10 pages, 5 figures, 1 table

2401.06059 2024-01-12 cs.CL cs.AI cs.LG 85%

Investigating Data Contamination for Pre-training Language Models

Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, Sanmi Koyejo

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 16 pages, 5 figures

2309.17352 2024-01-11 cs.SD eess.AS 85%

Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation

Shih-Lun Wu, Xuankai Chang, Gordon Wichern, Jee-weon Jung, François Germain, Jonathan Le Roux, Shinji Watanabe

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments ICASSP 2024 camera-ready paper. Winner of the DCASE 2023 Challenge Task 6A: Automated Audio Captioning (AAC)

2401.03851 2024-01-09 cs.CV q-bio.NC 85%

Aligned with LLM: a new multi-modal training paradigm for encoding fMRI activity in visual cortex

Shuxiao Ma, Linyuan Wang, Senbao Hou, Bin Yan

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

2309.03661 2023-12-15 cs.CV 85%

Prompt-based Context- and Domain-aware Pretraining for Vision and Language Navigation

Ting Liu, Yue Hu, Wansen Wu, Youkai Wang, Kai Xu, Quanjun Yin

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；prompting(abstract)

Comments 12 pages

2312.07843 2023-12-14 cs.RO 85%

Foundation Models in Robotics: Applications, Challenges, and the Future

Roya Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, Brian Ichter, Danny Driess, Jiajun Wu, Cewu Lu, Mac Schwager

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)

2311.01967 2023-11-06 cs.CL cs.AI cs.LG 85%

The language of prompting: What linguistic properties make a prompt successful?

Alina Leidinger, Robert van Rooij, Ekaterina Shutova

专题命中预训练与数据：prompting(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMNLP 2023 Findings

2310.13291 2023-10-23 cs.CL cs.AI cs.LG 85%

Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks

Ruixiang Tang, Gord Lueck, Rodolfo Quispe, Huseyin A Inan, Janardhan Kulkarni, Xia Hu

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2310.12303 2023-10-20 cs.CL cs.AI cs.LG 85%

Document-Level Language Models for Machine Translation

Frithjof Petrick, Christian Herold, Pavel Petrushkov, Shahram Khadivi, Hermann Ney

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted at WMT 2023

2306.09442 2023-10-12 cs.CL cs.AI cs.LG 85%

Explore, Establish, Exploit: Red Teaming Language Models from Scratch

Stephen Casper, Jason Lin, Joe Kwon, Gatlen Culp, Dylan Hadfield-Menell

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2306.09085 2023-06-16 cs.CV cs.AI cs.CL cs.LG cs.MM 85%

COSA: Concatenated Sample Pretrained Vision-Language Foundation Model

Sihan Chen, Xingjian He, Handong Li, Xiaojie Jin, Jiashi Feng, Jing Liu

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

2301.13823 2023-06-16 cs.CL cs.AI cs.CV cs.LG 85%

Grounding Language Models to Images for Multimodal Inputs and Outputs

Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in ICML 2023. Project page: https://jykoh.com/fromage

2306.02592 2023-06-06 cs.CL cs.AI cs.LG 85%

Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications

Han Xie, Da Zheng, Jun Ma, Houyu Zhang, Vassilis N. Ioannidis, Xiang Song, Qing Ping, Sheng Wang, Carl Yang, Yi Xu, Belinda Zeng, Trishul Chilimbi

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To be published in the KDD 2023 proceedings as a full paper

2207.06991 2023-04-27 cs.CL cs.AI cs.CV cs.LG 85%

Language Modelling with Pixels

Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, Desmond Elliott

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2023

2211.03263 2023-04-06 cs.CL cs.AI cs.LG 85%

AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages

Bonaventure F. P. Dossou, Atnafu Lambebo Tonja, Oreen Yousuf, Salomey Osei, Abigail Oppong, Iyanuoluwa Shode, Oluwabusayo Olufunke Awoyomi, Chris Chinenye Emezue

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Third Workshop on Simple and Efficient Natural Language Processing, EMNLP 2022

2109.00720 2023-01-26 cs.CL cs.AI cs.DB cs.IR cs.LG 85%

LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting

Xiang Chen, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, Huajun Chen, Ningyu Zhang

专题命中预训练与数据：prompting(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by COLING 2022

2301.08986 2023-01-24 cs.CL cs.AI cs.LG cs.NE 85%

Adapting a Language Model While Preserving its General Knowledge

Zixuan Ke, Yijia Shao, Haowei Lin, Hu Xu, Lei Shu, Bing Liu

专题命中预训练与数据：language model(title,abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2022

2209.07526 2022-10-21 cs.CV 85%

OmniVL:One Foundation Model for Image-Language and Video-Language Tasks

Junke Wang, Dongdong Chen, Zuxuan Wu, Chong Luo, Luowei Zhou, Yucheng Zhao, Yujia Xie, Ce Liu, Yu-Gang Jiang, Lu Yuan

专题命中预训练与数据：foundation model(title,abstract)；language model(abstract)；pretraining(abstract)

Comments To appear at NeurIPs 2022, Camera Ready with Typos fixed

2105.06020 2021-05-14 cs.CL cs.AI cs.LG 85%

Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level

Ruiqi Zhong, Dhruba Ghosh, Dan Klein, Jacob Steinhardt

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL 2021 Findings. Code and data: https://github.com/ruiqi-zhong/acl2021-instance-level

2010.03648 2021-04-15 cs.CL cs.AI cs.LG stat.ML 85%

A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks

Nikunj Saunshi, Sadhika Malladi, Sanjeev Arora

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments This version is the camera-ready version for ICLR 2021. Main changes include a detailed discussion about natural tasks, more detailed proof sketch and updated experimental evaluations

2010.11784 2021-04-08 cs.CL cs.AI cs.LG 85%

Self-Alignment Pretraining for Biomedical Entity Representations

Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, Nigel Collier

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NAACL 2021 camera-ready version

2004.14975 2020-11-11 cs.CL cs.AI cs.LG 85%

Investigating Transferability in Pretrained Language Models

Alex Tamkin, Trisha Singh, Davide Giovanardi, Noah Goodman

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Findings of EMNLP 2020

2608.14211 2026-08-17 cs.LG cs.AI 新提交 84%

Training Fair Tabular Foundation Models

训练公平表格基础模型

Patrik Kenfack, Jesse C. Cresswell, Anthony L. Caterini, Samira Ebrahimi Kahou, Ulrich Aïvodji

机构 * ÉTS Montréal（蒙特利尔高等技术学院）； Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）； Layer 6 AI（第六层人工智能公司）； University of Calgary（卡尔加里大学）； CIFAR（加拿大高级研究所）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI、cs.LG

AI总结针对表格基础模型（TFMs）公平性未被充分探索的问题，本研究提出FairTFM训练策略，将公平约束融入TFMs训练，在132个公平任务上实现公平性提升且保持竞争力准确性。

Comments Spotlight paper at the ICML 2026 Workshop on Foundation Models for Structured Data

2605.30393 2026-06-01 cs.LG cs.AI cs.CR 84%

NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Models

NumLeak: 基础模型中的公开数值基准作为潜在标签

Anany Kotawala

机构 * Princeton University（普林斯顿大学）

专题命中预训练与数据：foundation model(title,comments)；LLM(abstract_cn)；pretraining(abstract)；分类 cs.AI、cs.LG

AI总结提出NumLeak框架，通过API边界探测和开源因果模型的白盒验证，揭示基础模型在预训练中记忆公开数值基准，导致评估高估泛化能力。

Comments 23 pages, 12 figures, 17 tables. Accepted at the ICML 2026 Workshop on the Impact of Memorization on Trustworthy Foundation Models (MemFM)

2604.20549 2026-04-23 cs.CL cs.AI 84%

Toward Cross-Lingual Quality Classifiers for Multilingual Pretraining Data Selection

迈向多语言预训练数据选择的跨语言质量分类器

Yassine Turki, Vinko Sabolčec, Bettina Messmer, Martin Jaggi

机构 * Machine Learning Optimization Lab（机器学习优化实验室）； Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）

专题命中预训练与数据：pretraining(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本文探讨了通过跨语言迁移、第三四分位采样和保留率调整策略，提升多语言数据筛选效果，证明大规模多语言池化在稳定性与准确率上优于单语基线，尤其对低资源语言有显著提升。

Comments Accepted at the 3rd Workshop on Navigating and Addressing Data Problems for Foundation Models (DATA-FM @ ICLR 2026). 31 pages, 4 figures

2510.27522 2025-11-03 cs.LG cs.AI 84%

Leveraging Generic Time Series Foundation Models for EEG Classification

Théo Gnassounou, Yessin Moakher, Shifeng Xie, Vasilii Feofanov, Ievgen Redko

机构 * Université Paris-Saclay, Inria, CEA（巴黎萨克雷大学、法国国家信息与自动化研究所、法国原子能委员会）； Huawei Noah’s Ark Lab（华为诺亚实验室）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Journal ref NeurIPS 2025 Workshop "Recent Advances in Time Series Foundation Models Have We Reached the 'BERT Moment'?"

1. 预训练与数据 12429 篇

Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases

Multi-LLM Collaboration + Data-Centric Innovation = 2x Better Vulnerability Repair

Teach LLMs to Phish: Stealing Private Information from Language Models

Exploring Memorization in Fine-tuned Language Models

Uncovering Latent Human Wellbeing in Language Model Embeddings

Investigating Data Contamination for Pre-training Language Models

Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation

Aligned with LLM: a new multi-modal training paradigm for encoding fMRI activity in visual cortex

Prompt-based Context- and Domain-aware Pretraining for Vision and Language Navigation

Foundation Models in Robotics: Applications, Challenges, and the Future

The language of prompting: What linguistic properties make a prompt successful?

Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks

Document-Level Language Models for Machine Translation

Explore, Establish, Exploit: Red Teaming Language Models from Scratch

COSA: Concatenated Sample Pretrained Vision-Language Foundation Model

Grounding Language Models to Images for Multimodal Inputs and Outputs

Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications

Language Modelling with Pixels

AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages

LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting

Adapting a Language Model While Preserving its General Knowledge

OmniVL:One Foundation Model for Image-Language and Video-Language Tasks

Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level

A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks

Self-Alignment Pretraining for Biomedical Entity Representations

Investigating Transferability in Pretrained Language Models

Training Fair Tabular Foundation Models

NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Models

Toward Cross-Lingual Quality Classifiers for Multilingual Pretraining Data Selection

Leveraging Generic Time Series Foundation Models for EEG Classification