语言大模型 / LLM - arXivDaily 专题

2511.12768 2025-11-18 cs.CL cs.AI 84%

Evidence of Phase Transitions in Small Transformer-Based Language Models

Noah Hong, Tao Hong

机构 * Lynbrook High School（林brook高中）； Keysight Technologies（Keysight技术公司）

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2511.11572 2025-11-18 cs.GL cs.CL cs.LG 84%

LLM Architecture, Scaling Laws, and Economics: A Quick Summary

William H. Press

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 9 pages, 3 figures

2511.03251 2025-11-06 cs.LG cs.AI cs.SI 84%

GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models

Zhibin Wang, Zhixing Zhang, Shuqi Wang, Xuanting Xie, Zhao Kang

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2503.18065 2025-11-05 cs.CV cs.AI cs.CL cs.RO 84%

Unseen from Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigation

Ziming Wei, Bingqian Lin, Yunshuang Nie, Jiaqi Chen, Shikui Ma, Hang Xu, Xiaodan Liang

机构 * Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）； Shanghai Jiao Tong University（上海交通大学）； The University of Hong Kong（香港大学）； Hunan Artificial Intelligence and Robotics Institute Company Ltd.（湖南人工智能与机器人研究院有限公司）； Huawei Noah’s Ark Lab（华为诺亚实验室）； Peng Cheng Laboratory（鹏城实验室）

专题命中预训练与数据：foundation model(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by IEEE Transactions on Neural Networks and Learning Systems

2510.04340 2025-11-04 cs.CL cs.AI 84%

Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time

Daniel Tan, Anders Woodruff, Niels Warncke, Arun Jose, Maxime Riché, David Demitri Africa, Mia Taylor

机构 * University College London（伦敦大学学院）； Center on Long-Term Risk（长期风险中心）； McGill University（麦吉尔大学）； UK AI Security Institute（英国人工智能安全研究所）

专题命中预训练与数据：prompting(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 40 pages, 22 figures. Under review at ICLR 2026

2511.00346 2025-11-04 cs.CR cs.AI cs.LG 84%

Exploiting Latent Space Discontinuities for Building Universal LLM Jailbreaks and Data Extraction Attacks

Kayua Oleques Paim, Rodrigo Brandao Mansilha, Diego Kreutz, Muriel Figueredo Franco, Weverton Cordeiro

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 10 pages, 5 figures, 4 tables, Published at the Brazilian Symposium on Cybersecurity (SBSeg 2025)

2507.09846 2025-11-04 cs.LG cs.AI math.OC stat.ML 84%

Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training

Minhak Song, Beomhan Baek, Kwangjun Ahn, Chulhee Yun

机构 * KAIST（韩国科学技术院）； SNU & KAIST InnoCORE LLM（首尔国立大学及韩国科学技术院InnoCORE LLM）； Microsoft Research（微软研究院）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Published at NeurIPS 2025

2510.25771 2025-10-30 cs.CL cs.AI 84%

Gaperon: A Peppered English-French Generative Language Model Suite

Nathan Godey, Wissam Antoun, Rian Touchent, Rachel Bawden, Éric de la Clergerie, Benoît Sagot, Djamé Seddah

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

2510.24139 2025-10-29 cs.CL cs.AI 84%

Beyond Line-Level Filtering for the Pretraining Corpora of LLMs

Chanwoo Park, Suyoung Park, Yelim Ahn, Jongmin Kim, Jongyeon Park, Jaejin Lee

专题命中预训练与数据：pretraining(title)；language model(abstract)；small language model(abstract)；分类 cs.CL、cs.AI

Comments submitted to ACL ARR Rolling Review

2510.20860 2025-10-27 eess.AS cs.CL cs.LG 84%

Data-Centric Lessons To Improve Speech-Language Pretraining

Vishaal Udandarao, Zhiyun Lu, Xuankai Chang, Yongqiang Wang, Violet Z. Yao, Albin Madapally Jose, Fartash Faghri, Josh Gardner, Chung-Cheng Chiu

机构 * Apple（苹果公司）； University of Cambridge（剑桥大学）； University of Tübingen（图宾根大学）

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Tech Report

2510.18910 2025-10-23 cs.LG cs.AI 84%

Large Connectome Model: An fMRI Foundation Model of Brain Connectomes Empowered by Brain-Environment Interaction in Multitask Learning Landscape

Ziquan Wei, Tingting Dan, Guorong Wu

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 12 pages 6 figures

2510.08445 2025-10-21 cs.LG cs.AI 84%

Synthetic Series-Symbol Data Generation for Time Series Foundation Models

Wenxuan Wang, Kai Wu, Yujian Betterest Li, Dan Wang, Xiaoyu Zhang

机构 * School of Telecommunications Engineering Xidian University（电子科技大学电信工程学院）； School of Artificial Intelligence Xidian University（电子科技大学人工智能学院）； School of Cyber Engineering Xidian University（电子科技大学网络安全工程学院）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 64 pages, 25 figures, 35 tables, NeurIPS 2025 accepted

2504.09426 2025-10-14 cs.CV cs.AI cs.CL 84%

BabyVLM: Data-Efficient Pretraining of VLMs Inspired by Infant Learning

Shengao Wang, Arjun Chandra, Aoming Liu, Venkatesh Saligrama, Boqing Gong

机构 * Boston University（波士顿大学）

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to ICCV 2025

2510.10009 2025-10-14 cs.CL cs.AI cs.IR 84%

Beyond the limitation of a single query: Train your LLM for query expansion with Reinforcement Learning

Shu Zhao, Tan Yu, Anbang Xu

机构 * NVIDIA（英伟达）； Pennsylvania State University（宾夕法尼亚州立大学）

专题命中预训练与数据：LLM(title,abstract)；post-training(abstract)；分类 cs.CL、cs.AI

2510.09846 2025-10-14 cs.LG cs.AI 84%

CALM: A Causal Analysis Language Model for Tabular Data in Complex Systems with Local Scores, Conditional Independence Tests, and Relation Attributes

Zhenjiang Fan, Zengyi Qin, Yuanning Zheng, Bo Xiong, Summer Han

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

2510.04268 2025-10-07 cs.CL cs.AI 84%

LongTail-Swap: benchmarking language models' abilities on rare words

Robin Algayres, Charles-Éric Saint-James, Mahi Luthra, Jiayi Shen, Dongyan Lin, Youssef Benchekroun, Rashel Moritz, Juan Pino, Emmanuel Dupoux

机构 * Meta AI

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

2503.19551 2025-10-07 cs.CL cs.AI 84%

Scaling Laws of Synthetic Data for Language Models

Zeyu Qin, Qingxiu Dong, Xingxing Zhang, Li Dong, Xiaolong Huang, Ziyi Yang, Mahmoud Khademi, Dongdong Zhang, Hany Hassan Awadalla, Yi R. Fung, Weizhu Chen, Minhao Cheng, Furu Wei

机构 * Microsoft（微软公司）； HKUST（香港科技大学）； Peking University（北京大学）； Penn State University（宾夕法尼亚州立大学）

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments COLM 2025

2510.01571 2025-10-03 cs.LG cs.AI q-bio.BM 84%

From Supervision to Exploration: What Does Protein Language Model Learn During Reinforcement Learning?

Hanqun Cao, Hongrui Zhang, Junde Xu, Zhou Zhang, Lingdong Shen, Minghao Sun, Ge Liu, Jinbo Xu, Wu-Jun Li, Jinren Ni, Cesar de la Fuente-Nunez, Tianfan Fu, Yejin Choi, Pheng-Ann Heng, Fang Wu

机构 * The Chinese University of Hong Kong（香港中文大学）； Peking University（北京大学）； Stanford University（斯坦福大学）； Nanjing University（南京大学）； University of Pennsylvania（宾夕法尼亚大学）； National University of Singapore（新加坡国立大学）； University of Illinois Urbana-Champaign（伊利诺伊大学香槟分校）； Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 24 pages, 7 figures, 4 tables

2509.25498 2025-10-01 cs.CL cs.AI 84%

Not Wrong, But Untrue: LLM Overconfidence in Document-Based Queries

Nick Hagar, Wilma Agustianto, Nicholas Diakopoulos

机构 * Northwestern University（西北大学）； University of Minnesota（明尼苏达大学）

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to Computation + Journalism Symposium 2025

2507.12930 2025-09-30 cs.CL cs.AI 84%

Making Language Model a Hierarchical Classifier

Yihong Wang, Zhonglin Jiang, Ningyuan Xi, Yue Zhao, Qingqing Gu, Xiyuan Chen, Hao Wu, Sheng Xu, Hange Zhou, Yong Chen, Luo Ji

机构 * Geely AI Lab（地平线AI实验室）； Tsinghua University（清华大学）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

2509.18152 2025-09-24 cs.LG cs.AI 84%

WLFM: A Well-Logs Foundation Model for Multi-Task and Cross-Well Geological Interpretation

Zhenyu Qi, Qing Yu, Jichen Wang, Yun-Bo Zhao, Zerui Li, Wenjun Lv

机构 * Institute of Advanced Technology, University of Science and Technology of China（科学技术大学先进技术研究所）； Department of Automation, University of Science and Technology of China（科学技术大学自动化系）； Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合国家科学中心人工智能研究所）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2509.17317 2025-09-23 cs.CL cs.AI 84%

Scaling, Simplification, and Adaptation: Lessons from Pretraining on Machine-Translated Text

Dan John Velasco, Matthew Theodore Roque

机构 * Samsung R&D Institute Philippines（三星菲律宾研发院）

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Under review

2509.10369 2025-09-15 cs.LG cs.AI eess.SP q-bio.TO 84%

Data distribution impacts the performance and generalisability of contrastive learning-based foundation models of electrocardiograms

Gul Rukh Khattak, Konstantinos Patlatzoglou, Joseph Barker, Libor Pastika, Boroumand Zeidaabadi, Ahmed El-Medany, Hesham Aggour, Yixiu Liang, Antonio H. Ribeiro, Jeffrey Annis, Antonio Luiz Pinho Ribeiro, Junbo Ge, Daniel B. Kramer, Jonathan W. Waks, Evan Brittain, Nicholas Peters, Fu Siong Ng, Arunashis Sau

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Currently under review at npj Digital Medicine

2508.03682 2025-09-11 cs.LG cs.AI 84%

Self-Questioning Language Models

Lili Chen, Mihir Prabhudesai, Katerina Fragkiadaki, Hao Liu, Deepak Pathak

机构 * Carnegie Mellon University（卡内基梅隆大学）

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

2507.21509 2025-09-08 cs.CL cs.LG 84%

Persona Vectors: Monitoring and Controlling Character Traits in Language Models

Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, Jack Lindsey

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

2509.02046 2025-09-08 cs.LG cs.AI stat.ML 84%

Fantastic Pretraining Optimizers and Where to Find Them

Kaiyue Wen, David Hall, Tengyu Ma, Percy Liang

机构 * Stanford University（斯坦福大学）

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 108 pages, 8 figures, reproducible runs available at https://wandb.ai/marin-community/optimizer-scaling

2508.18598 2025-08-27 cs.CL cs.AI 84%

What do language models model? Transformers, automata, and the format of thought

Colin Klein

机构 * School of Philosophy（哲学学院）； The Australian National University（澳大利亚国立大学）

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2406.13069 2025-08-26 cs.CL cs.AI 84%

Evaluating $n$-Gram Novelty of Language Models Using Rusty-DAWG

William Merrill, Noah A. Smith, Yanai Elazar

机构 * New York University（纽约大学）； Allen Institute for AI（人工智能研究院）； University of Washington（华盛顿大学）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments To appear at EMNLP 2024

2508.15791 2025-08-25 cs.CL cs.AI 84%

InteChar: A Unified Oracle Bone Character List for Ancient Chinese Language Modeling

Xiaolei Diao, Zhihan Zhou, Lida Shi, Ting Wang, Ruihua Qi, Hao Xu, Daqian Shi

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

2508.13603 2025-08-20 cs.CL cs.AI 84%

Who Gets the Mic? Investigating Gender Bias in the Speaker Assignment of a Speech-LLM

Dariia Puhach, Amir H. Payberah, Éva Székely

机构 * Department of Linguistics（语言学系）； Department of Speech, Music, and Hearing（语音、音乐与听觉系）

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Journal ref Interspeech 2025 (2025), 2058-2062

1. 预训练与数据 12429 篇

Evidence of Phase Transitions in Small Transformer-Based Language Models

LLM Architecture, Scaling Laws, and Economics: A Quick Summary

GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models

Unseen from Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigation

Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time

Exploiting Latent Space Discontinuities for Building Universal LLM Jailbreaks and Data Extraction Attacks

Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training

Gaperon: A Peppered English-French Generative Language Model Suite

Beyond Line-Level Filtering for the Pretraining Corpora of LLMs

Data-Centric Lessons To Improve Speech-Language Pretraining

Large Connectome Model: An fMRI Foundation Model of Brain Connectomes Empowered by Brain-Environment Interaction in Multitask Learning Landscape

Synthetic Series-Symbol Data Generation for Time Series Foundation Models

BabyVLM: Data-Efficient Pretraining of VLMs Inspired by Infant Learning

Beyond the limitation of a single query: Train your LLM for query expansion with Reinforcement Learning

CALM: A Causal Analysis Language Model for Tabular Data in Complex Systems with Local Scores, Conditional Independence Tests, and Relation Attributes

LongTail-Swap: benchmarking language models' abilities on rare words

Scaling Laws of Synthetic Data for Language Models

From Supervision to Exploration: What Does Protein Language Model Learn During Reinforcement Learning?

Not Wrong, But Untrue: LLM Overconfidence in Document-Based Queries

Making Language Model a Hierarchical Classifier

WLFM: A Well-Logs Foundation Model for Multi-Task and Cross-Well Geological Interpretation

Scaling, Simplification, and Adaptation: Lessons from Pretraining on Machine-Translated Text

Data distribution impacts the performance and generalisability of contrastive learning-based foundation models of electrocardiograms

Self-Questioning Language Models

Persona Vectors: Monitoring and Controlling Character Traits in Language Models

Fantastic Pretraining Optimizers and Where to Find Them

What do language models model? Transformers, automata, and the format of thought

Evaluating $n$-Gram Novelty of Language Models Using Rusty-DAWG

InteChar: A Unified Oracle Bone Character List for Ancient Chinese Language Modeling

Who Gets the Mic? Investigating Gender Bias in the Speaker Assignment of a Speech-LLM