语言大模型 / LLM - arXivDaily 专题

2010.12858 2021-04-20 cs.CL 83%

When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models

Benjamin Muller, Antonis Anastasopoulos, Benoît Sagot, Djamé Seddah

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted at NAACL-HLT 2021

2103.10531 2021-04-15 cs.CL 83%

Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation

Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted at NAACL 2021

2010.12547 2021-04-13 cs.CL 83%

Multilingual BERT Post-Pretraining Alignment

Lin Pan, Chung-Wei Hang, Haode Qi, Abhishek Shah, Saloni Potdar, Mo Yu

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments Accepted at NAACL2021

2011.08539 2020-11-18 cs.CL 83%

MVP-BERT: Redesigning Vocabularies for Chinese BERT and Multi-Vocab Pretraining

Wei Zhu

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

2011.04946 2020-11-11 cs.CL 83%

When Do You Need Billions of Words of Pretraining Data?

Yian Zhang, Alex Warstadt, Haau-Sing Li, Samuel R. Bowman

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 10 pages, 6 figures

2011.03203 2020-11-09 cs.CL 83%

Unleashing the Power of Neural Discourse Parsers -- A Context and Structure Aware Approach Using Large Scale Pretraining

Grigorii Guz, Patrick Huber, Giuseppe Carenini

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 10 pages, 1 figure, COLING 2020

2010.02480 2020-10-30 cs.CL 83%

Pretrained Language Model Embryology: The Birth of ALBERT

Cheng-Han Chiang, Sung-Feng Huang, Hung-yi Lee

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2020, short paper

2010.09403 2020-10-20 cs.CL 83%

Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation

Dušan Variš, Ondřej Bojar

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments ACL-SRW 2019 (camera-ready)

2009.07610 2020-10-07 cs.CL 83%

Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT

Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments EMNLP 2020, main conference

2009.11462 2020-09-29 cs.CL 83%

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models

Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, Noah A. Smith

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Findings in EMNLP 2020

2007.11648 2020-07-24 cs.CL 83%

Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Models

Mittul Singh, Peter Smit, Sami Virpioja, Mikko Kurimo

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2005.13482 2020-05-28 cs.CL 83%

Syntactic Structure Distillation Pretraining For Bidirectional Encoders

Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, Phil Blunsom

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 17 pages, 6 tables, 2 figures. AK and LK contributed equally

2004.11026 2020-04-24 cs.CL 83%

QURIOUS: Question Generation Pretraining for Text Generation

Shashi Narayan, Gonçalo Simoes, Ji Ma, Hannah Craighead, Ryan Mcdonald

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 9 pages

1909.02339 2020-04-21 cs.CL 83%

Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity

Anne Lauscher, Ivan Vulić, Edoardo Maria Ponti, Anna Korhonen, Goran Glavaš

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

1912.10165 2019-12-24 cs.CL 83%

Zero-shot Text Classification With Generative Language Models

Raul Puri, Bryan Catanzaro

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

1912.09637 2019-12-23 cs.CL 83%

Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model

Wenhan Xiong, Jingfei Du, William Yang Wang, Veselin Stoyanov

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted to ICLR 2020

1909.01066 2019-09-05 cs.CL 83%

Language Models as Knowledge Bases?

Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, Sebastian Riedel

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments accepted at EMNLP 2019

1906.02940 2019-07-30 cs.LG cs.CV eess.IV stat.ML 83%

Selfie: Self-supervised Pretraining for Image Embedding

Trieu H. Trinh, Minh-Thang Luong, Quoc V. Le

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.LG

1907.11692 2019-07-29 cs.CL 83%

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

1904.00585 2019-05-20 cs.CL 83%

Using Similarity Measures to Select Pretraining Data for NER

Xiang Dai, Sarvnaz Karimi, Ben Hachey, Cecile Paris

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments NAACL 2019

1902.07938 2019-02-22 cs.CL 83%

Pretrained language model transfer on neural named entity recognition in Indonesian conversational texts

Rezka Leonandya, Fariz Ikhwantri

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted in CICLing 2019

1901.09128 2019-01-29 cs.CL 83%

Language Model Pre-training for Hierarchical Document Representations

Ming-Wei Chang, Kristina Toutanova, Kenton Lee, Jacob Devlin

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

1809.10040 2019-01-08 cs.CL 83%

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

Kelly W. Zhang, Samuel R. Bowman

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Journal ref Blackbox NLP Workshop, EMNLP 2018

1210.8440 2012-11-01 cs.CL 83%

Large Scale Language Modeling in Automatic Speech Recognition

Ciprian Chelba, Dan Bikel, Maria Shugrina, Patrick Nguyen, Shankar Kumar

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

cs/0108023 2009-11-30 cs.CL cs.IR 83%

Information Extraction Using the Structured Language Model

Ciprian Chelba, Milind Mahajan

专题命中预训练与数据：language model(title,abstract)；SLM(abstract)；分类 cs.CL

Comments EMNLP'01, Pittsburgh; 8 pages

Journal ref EMNLP/NAACL 2001 Conference Proceedings

cs/0001023 2009-11-30 cs.CL 83%

Structured Language Modeling for Speech Recognition

Ciprian Chelba, Frederick Jelinek

专题命中预训练与数据：language model(title,abstract)；SLM(abstract)；分类 cs.CL

Comments 4 pages + 2 pages of ERRATA

Journal ref Proceedings of NLDB'99, Klagenfurt, Austria

2606.11387 2026-06-11 cs.CL cs.AI cs.LG 新提交 83%

Small Experiments, Cheaper Decisions: A Case Study in Staged Promotion for Micro-Pretraining

小实验，更经济的决策：微预训练中分阶段提升的案例研究

Felipe Chavarro Polania

机构 * Hewlett Packard Enterprise（慧与科技公司）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结研究微预训练中分阶段提升协议，通过固定预算筛选配置，在Windows A100和Linux L40S上验证，发现早期排名不稳定，但最终协议以144 GPU小时找到最优配置，成本低于全量筛选。

Comments 14 pages, 5 figures; 12-hour dual-host micro-pretraining promotion study; source package includes curated ancillary artifacts

2602.17881 2026-02-23 cs.CL cs.AI cs.LG 83%

Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

理解语言模型中转向向量的不可靠性：几何预测器与线性近似的局限性

Joschka Braun

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG；foundation model(comments)

AI总结该研究探讨了语言模型中转向向量的可靠性问题，发现训练数据和行为表示对转向效果有显著影响，提出几何预测器和非线性近似方法以提升稳定性。

Comments Master's Thesis, University of Tübingen. 89 pages, 34 figures. Portions of this work were published at the ICLR 2025 Workshop on Foundation Models in the Wild (see arXiv:2505.22637)

2505.24167 2025-06-02 cs.CV 83%

Pretraining Deformable Image Registration Networks with Random Images

Junyu Chen, Shuwen Wei, Yihao Liu, Aaron Carass, Yong Du

机构 * Johns Hopkins University（约翰霍普金斯大学）； Vanderbilt University（范德比尔特大学）

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)

Comments Accepted by MIDL 2025. Code available at https://github.com/junyuchen245/Pretraining_Image_Registration_DNNs

2608.20365 2026-08-24 cs.CL cs.AI 新提交 82%

Trilingual Topic Modeling of Sri Lankan Parliamentary Debates

斯里兰卡议会辩论的三语主题建模

Himath Dhanapala, Haren Daishika, Himandhi Kuruppu, Sithija Seneviratne, Ashini Kavindya, Patalee Narasinghe, Sandeepa Weerasekara, Nisansa de Silva, Sandareka Wickramanayake

专题命中预训练与数据：LLM(summary_cn,abstract)；分类 cs.CL、cs.AI

AI总结针对斯里兰卡议会三语辩论语料库因多语言特性难以处理的问题，提出基于LLM的文本提取结合多语言嵌入与密度聚类的端到端框架，辅以BiTopic方法，成功识别三种语言的30个宏观主题，其时间轨迹与重大国家事件吻合，且性能优于传统LDA。

1. 预训练与数据 12403 篇

When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models

Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation

Multilingual BERT Post-Pretraining Alignment

MVP-BERT: Redesigning Vocabularies for Chinese BERT and Multi-Vocab Pretraining

When Do You Need Billions of Words of Pretraining Data?

Unleashing the Power of Neural Discourse Parsers -- A Context and Structure Aware Approach Using Large Scale Pretraining

Pretrained Language Model Embryology: The Birth of ALBERT

Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation

Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models

Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Models

Syntactic Structure Distillation Pretraining For Bidirectional Encoders

QURIOUS: Question Generation Pretraining for Text Generation

Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity

Zero-shot Text Classification With Generative Language Models

Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model

Language Models as Knowledge Bases?

Selfie: Self-supervised Pretraining for Image Embedding

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Using Similarity Measures to Select Pretraining Data for NER

Pretrained language model transfer on neural named entity recognition in Indonesian conversational texts

Language Model Pre-training for Hierarchical Document Representations

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

Large Scale Language Modeling in Automatic Speech Recognition

Information Extraction Using the Structured Language Model

Structured Language Modeling for Speech Recognition

Small Experiments, Cheaper Decisions: A Case Study in Staged Promotion for Micro-Pretraining

Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

Pretraining Deformable Image Registration Networks with Random Images

Trilingual Topic Modeling of Sri Lankan Parliamentary Debates