语言大模型 / LLM - arXivDaily 专题

1911.07613 2019-11-19 cs.CL cs.LG 81%

A Subword Level Language Model for Bangla Language

Aisha Khatun, Anisur Rahman, Hemayet Ahmed Chowdhury, Md. Saiful Islam, Ayesha Tasnim

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 12 pages, Conference Paper

1902.09243 2019-10-16 cs.CL cs.AI 81%

Pretraining-Based Natural Language Generation for Text Summarization

Haoyu Zhang, Jianjun Xu, Ji Wang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments 7 pages

Journal ref CoNLL'2019

1909.02244 2019-09-06 cs.CL cs.CV cs.LG 81%

Robust Navigation with Language Pretraining and Stochastic Sampling

Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, Yejin Choi

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 4 figures, EMNLP 2019

1908.09738 2019-08-27 eess.AS cs.CL cs.LG cs.SD 81%

Connecting and Comparing Language Model Interpolation Techniques

Ernest Pusateri, Christophe Van Gysel, Rami Botros, Sameer Badaskar, Mirko Hannemann, Youssef Oualil, Ilya Oparin

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

1907.07033 2019-07-17 cs.CL cs.LG 81%

Neural Language Model Based Training Data Augmentation for Weakly Supervised Early Rumor Detection

Sooji Han, Jie Gao, Fabio Ciravegna

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 8 pages

Journal ref The 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining

1907.03064 2019-07-09 cs.CL cs.LG eess.AS 81%

Improved low-resource Somali speech recognition by semi-supervised acoustic and language model training

Astik Biswas, Raghav Menon, Ewald van der Westhuizen, Thomas Niesler

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 5 pages, 6 Tables, 3 figures, 22 references (Accepted at Interspeech 2019)

1906.04284 2019-06-20 cs.CL cs.LG stat.ML 81%

Analyzing the Structure of Attention in a Transformer Language Model

Jesse Vig, Yonatan Belinkov

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

Comments To appear in ACL BlackboxNLP workshop

1906.05678 2019-06-14 eess.AS cs.CL cs.LG cs.SD stat.ML 81%

Telephonetic: Making Neural Language Models Robust to ASR and Semantic Noise

Chris Larson, Tarek Lahlou, Diana Mingels, Zachary Kulis, Erik Mueller

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

1906.02242 2019-06-07 cs.CL cs.LG 81%

Variational Pretraining for Semi-supervised Text Classification

Suchin Gururangan, Tam Dang, Dallas Card, Noah A. Smith

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments ACL 2019

1906.01496 2019-06-05 cs.CL cs.LG stat.ML 81%

Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains

Navid Rekabsaz, Nikolaos Pappas, James Henderson, Banriskhem K. Khonglah, Srikanth Madikeri

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

1906.00414 2019-06-05 cs.CL cs.AI 81%

Pretraining Methods for Dialog Context Representation Learning

Shikib Mehri, Evgeniia Razumovskaia, Tiancheng Zhao, Maxine Eskenazi

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2019

1904.09705 2019-04-23 cs.CL cs.AI 81%

Exploring Unsupervised Pretraining and Sentence Structure Modelling for Winograd Schema Challenge

Yu-Ping Ruan, Xiaodan Zhu, Zhen-Hua Ling, Zhan Shi, Quan Liu, Si Wei

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments 7 pages

1812.05199 2018-12-17 cs.CL cs.LG 81%

Recurrent Neural Networks with Pre-trained Language Model Embedding for Slot Filling Task

Liang Qiu, Yuanyi Ding, Lei He

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

1807.10857 2018-11-08 eess.AS cs.AI cs.CL cs.SD 81%

A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition

Shubham Toshniwal, Anjuli Kannan, Chung-Cheng Chiu, Yonghui Wu, Tara N Sainath, Karen Livescu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted in SLT 2018

1810.03459 2018-10-09 cs.CL cs.LG cs.SD eess.AS 81%

Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling

Jaejin Cho, Murali Karthick Baskar, Ruizhi Li, Matthew Wiesner, Sri Harish Mallidi, Nelson Yalta, Martin Karafiat, Shinji Watanabe, Takaaki Hori

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

1803.03665 2018-03-13 cs.CL cs.LG 81%

Syntax-Aware Language Modeling with Recurrent Neural Networks

Duncan Blythe, Alan Akbik, Roland Vollgraf

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

1801.10459 2018-02-12 cs.AI cs.LG stat.ML 81%

Pretraining Deep Actor-Critic Reinforcement Learning Algorithms With Expert Demonstrations

Xiaoqin Zhang, Huimin Ma

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Added acknowledgements, modified references. 7 pages, 4 figures

1709.00541 2017-09-05 cs.CL cs.LG 81%

Patterns versus Characters in Subword-aware Neural Language Modeling

Rustem Takhanov, Zhenisbek Assylbekov

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 10 pages

1708.07252 2017-08-25 cs.CL cs.AI 81%

A Study on Neural Network Language Modeling

Dengliang Shi

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 20 pages, 6 figures

1708.06989 2017-08-24 cs.CL cs.AI 81%

A Neural Network Approach for Mixing Language Models

Youssef Oualil, Dietrich Klakow

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Published at IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2017. arXiv admin note: text overlap with arXiv:1703.08068

Journal ref IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, 2017, pp. 5710-5714

1609.03777 2017-02-03 cs.LG cs.CL cs.NE 81%

Character-Level Language Modeling with Hierarchical Recurrent Neural Networks

Kyuyeon Hwang, Wonyong Sung

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Submitted to NIPS 2016 on May 20, 2016 (v1), accepted to ICASSP 2017 (v2)

1506.01192 2016-11-23 cs.CL cs.LG 81%

Personalizing Universal Recurrent Neural Network Language Model with User Characteristic Features by Social Network Crowdsouring

Bo-Hsiang Tseng, Hung-Yi Lee, Lin-Shan Lee

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

Comments IEEE Automatic Speech Recognition and Understanding Workshop (ASRU 2015), 13-17 Dec 2015, Scottsdale, Arizona, USA

1206.6426 2016-06-07 cs.CL cs.LG 81%

A Fast and Simple Algorithm for Training Neural Probabilistic Language Models

Andriy Mnih, Yee Whye Teh

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Appears in Proceedings of the 29th International Conference on Machine Learning (ICML 2012)

Journal ref In Proceedings of the 29th International Conference on Machine Learning, pages 1751-1758, 2012

1603.01913 2016-04-06 cs.CL cs.LG cs.NE stat.ML 81%

A Latent Variable Recurrent Neural Network for Discourse Relation Language Models

Yangfeng Ji, Gholamreza Haffari, Jacob Eisenstein

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.LG

Comments NAACL 2016 camera ready, 11 pages

2606.26467 2026-06-26 cs.LG 新提交 80%

A Causal Foundation Model for Structure and Outcome Prediction

用于结构与结果预测的因果基础模型

Max Zhu, Martino Mansoldo, Ching-Hao Wang, Stefan Groha

机构 * University of Cambridge, United Kingdom（英国剑桥大学）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

AI总结提出TabPFN-CFM，一种能处理多种因果问题的因果基础模型，从观测数据预测因果结构和结果，支持Pearl因果层次所有三层查询，在合成数据上训练并泛化到真实数据，优于结构和结果预测基线。

Comments 20 pages, 7 figures, 17 tables, 43rd ICML Workshop on Foundation Models for Structured Data

2603.13647 2026-03-17 cs.LG cs.NI 80%

PLUME: Building a Network-Native Foundation Model for Wireless Traces via Protocol-Aware Tokenization

PLUME：通过协议感知的分词构建无线痕迹的网络原生基础模型

Swadhin Pradhan, Shazal Irshad, Jerome Henry

机构 * Cisco Systems（思科系统）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

AI总结 PLUME通过协议感知分词技术，为无线痕迹构建网络原生基础模型，实现更高效的序列生成与异常检测。

Comments 14-pages, 802.11 foundation model, matches frontier LLMs with 600x fewer params via protocol-aware tokenization, 5 figures, 12 tables, AUROC>=0.99 for zero-shot anomaly detection

2505.19386 2025-11-27 cs.CV cs.AI 80%

Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signals

力提示：视频生成模型可以学习并泛化基于物理的控制信号

Nate Gillman, Charles Herrmann, Michael Freeman, Daksh Aggarwal, Evan Luo, Deqing Sun, Chen Sun

机构 * Brown University（布朗大学）； Google DeepMind（谷歌DeepMind）

专题命中预训练与数据：prompting(title,abstract)；分类 cs.AI

AI总结本文提出力提示方法，通过物理力信号生成逼真视频，利用视觉和运动先验实现物理控制信号的泛化，提升世界模型的物理真实性。

Comments Camera ready version (NeurIPS 2025). Code and interactive demos at https://force-prompting.github.io/

2411.15127 2025-11-19 cs.LG 80%

PRIMUS: Pretraining IMU Encoders with Multimodal Self-Supervision

Arnav M. Das, Chi Ian Tang, Fahim Kawsar, Mohammad Malekzadeh

机构 * Nokia Bell Labs Cambridge, UK（诺基亚贝尔实验室（剑桥，英国））； University of Washington, USA（华盛顿大学（美国））； University of Glasgow, UK（格拉斯哥大学（英国））

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Presented at ICASSP 2025. Also presented under the title "PRIMUS: Pretraining IMU Encoders with Multimodal and Self-Supervised Learning" at NeurIPS 2024 TSALM Workshop (Time Series in the Age of Large Models)

2511.11940 2025-11-18 cs.LG eess.SP 80%

Learning the relative composition of EEG signals using pairwise relative shift pretraining

Christopher Sandino, Sayeri Lala, Geeling Chau, Melika Ayoughi, Behrooz Mahasseni, Ellen Zippi, Ali Moin, Erdrin Azemi, Hanlin Goh

机构 * Apple（苹果公司）； Stanford University（斯坦福大学）； California Institute of Technology（加州理工学院）； University of Amsterdam（阿姆斯特丹大学）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG；foundation model(comments)

Comments Foundation Models for the Brain and Body NeurIPS 2025 Workshop

2403.08592 2025-09-18 cs.LG q-bio.QM 80%

Data-Efficient Sleep Staging with Synthetic Time Series Pretraining

Niklas Grieger, Siamak Mehrkanoon, Stephan Bialonski

机构 * Department of Medical Engineering and Technomathematics, FH Aachen University of Applied Sciences（弗劳恩霍夫亚琛应用科学大学医学工程与技术数学系）； Department of Information and Computing Sciences, Utrecht University（乌得勒支大学信息与计算科学系）； Institute for Data-Driven Technologies, FH Aachen University of Applied Sciences（弗劳恩霍夫亚琛应用科学大学数据驱动技术研究所）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 15 pages, 4 figures, 1 table

Journal ref Grieger, N., Mehrkanoon, S., Bialonski, S. (2025). Data-Efficient Sleep Staging with Synthetic Time Series Pretraining. Algorithms, 18(9), 580

1. 预训练与数据 12429 篇

A Subword Level Language Model for Bangla Language

Pretraining-Based Natural Language Generation for Text Summarization

Robust Navigation with Language Pretraining and Stochastic Sampling

Connecting and Comparing Language Model Interpolation Techniques

Neural Language Model Based Training Data Augmentation for Weakly Supervised Early Rumor Detection

Improved low-resource Somali speech recognition by semi-supervised acoustic and language model training

Analyzing the Structure of Attention in a Transformer Language Model

Telephonetic: Making Neural Language Models Robust to ASR and Semantic Noise

Variational Pretraining for Semi-supervised Text Classification

Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains

Pretraining Methods for Dialog Context Representation Learning

Exploring Unsupervised Pretraining and Sentence Structure Modelling for Winograd Schema Challenge

Recurrent Neural Networks with Pre-trained Language Model Embedding for Slot Filling Task

A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition

Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling

Syntax-Aware Language Modeling with Recurrent Neural Networks

Pretraining Deep Actor-Critic Reinforcement Learning Algorithms With Expert Demonstrations

Patterns versus Characters in Subword-aware Neural Language Modeling

A Study on Neural Network Language Modeling

A Neural Network Approach for Mixing Language Models

Character-Level Language Modeling with Hierarchical Recurrent Neural Networks

Personalizing Universal Recurrent Neural Network Language Model with User Characteristic Features by Social Network Crowdsouring

A Fast and Simple Algorithm for Training Neural Probabilistic Language Models

A Latent Variable Recurrent Neural Network for Discourse Relation Language Models

A Causal Foundation Model for Structure and Outcome Prediction

PLUME: Building a Network-Native Foundation Model for Wireless Traces via Protocol-Aware Tokenization

Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signals

PRIMUS: Pretraining IMU Encoders with Multimodal Self-Supervision

Learning the relative composition of EEG signals using pairwise relative shift pretraining

Data-Efficient Sleep Staging with Synthetic Time Series Pretraining