语言大模型 / LLM - arXivDaily 专题

2004.13952 2021-03-12 cs.CL 79%

Data Augmentation for Spoken Language Understanding via Pretrained Language Models

Baolin Peng, Chenguang Zhu, Michael Zeng, Jianfeng Gao

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 5 pages, 1 figure

2103.05131 2021-03-10 cs.CL 79%

Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data

Sanjeev Kumar Karn, Francine Chen, Yan-Ying Chen, Ulli Waltinger, Hinrich Schuetze

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Adapt-NLP: The Second Workshop on Domain Adaptation for NLP

2102.07370 2021-02-16 cs.CL 79%

Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification

Bidisha Sharma, Maulik Madhavi, Haizhou Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2102.04754 2021-02-10 cs.CL 79%

Bayesian Transformer Language Models for Speech Recognition

Boyang Xue, Jianwei Yu, Junhao Xu, Shansong Liu, Shoukang Hu, Zi Ye, Mengzhe Geng, Xunying Liu, Helen Meng

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2101.11562 2021-01-28 cs.CV cs.CL 79%

Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Network

Yehao Li, Yingwei Pan, Ting Yao, Jingwen Chen, Tao Mei

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments AAAI 2021; Code is publicly available at: https://github.com/YehLi/TDEN

2101.10717 2021-01-27 cs.CL 79%

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification

Yi Zhu, Ehsan Shareghi, Yingzhen Li, Roi Reichart, Anna Korhonen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments EACL 2021

2012.12543 2020-12-25 cs.CL cs.SD eess.AS 79%

Code Switching Language Model Using Monolingual Training Data

Asad Ullah, Tauseef Ahmed

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments submitted to ICASSP2021

2012.11995 2020-12-23 cs.CL 79%

Pre-Training a Language Model Without Human Language

Cheng-Han Chiang, Hung-yi Lee

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 9 pages, work in progress

2012.07331 2020-12-15 eess.AS cs.CL cs.SD 79%

Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval

Yuma Koizumi, Yasunori Ohishi, Daisuke Niizumi, Daiki Takeuchi, Masahiro Yasuda

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Submitted to ICASSP 2021

2012.03084 2020-12-08 q-bio.BM cs.CL 79%

Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream Tasks

Modestas Filipavicius, Matteo Manica, Joris Cadow, Maria Rodriguez Martinez

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 20 pages, 12 figures, accepted to Machine Learning for Structural Biology (MLSB) workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS)

2012.00413 2020-12-02 cs.CL 79%

CPM: A Large-scale Generative Chinese Pre-trained Language Model

Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, Zhiyuan Liu, Minlie Huang, Wentao Han, Jie Tang, Juanzi Li, Xiaoyan Zhu, Maosong Sun

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2011.08626 2020-11-20 cs.CL 79%

Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining

Zijun Sun, Chun Fan, Xiaofei Sun, Yuxian Meng, Fei Wu, Jiwei Li

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2011.04839 2020-11-11 cs.SD cs.CL 79%

Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis

Erica Cooper, Xin Wang, Yi Zhao, Yusuke Yasuda, Junichi Yamagishi

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Technical report

2007.13002 2020-10-30 eess.AS cs.CL cs.SD 79%

Unsupervised Subword Modeling Using Autoregressive Pretraining and Cross-Lingual Phone-Aware Modeling

Siyuan Feng, Odette Scharenborg

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 5 pages, 3 figures. Accepted for publication in INTERSPEECH 2020, Shanghai, China

2010.13870 2020-10-28 cs.CL 79%

Word Frequency Does Not Predict Grammatical Knowledge in Language Models

Charles Yu, Ryan Sie, Nico Tedeschi, Leon Bergen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2020

2010.11639 2020-10-23 cs.CL 79%

Towards Fully Bilingual Deep Language Modeling

Li-Hsin Chang, Sampo Pyysalo, Jenna Kanerva, Filip Ginter

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2010.11553 2020-10-23 cs.CL 79%

Incorporating Stylistic Lexical Preferences in Generative Language Models

Hrituraj Singh, Gaurav Verma, Balaji Vasan Srinivasan

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To Appear in Findings of EMNLP 2020

2010.06804 2020-10-15 cs.CL 79%

Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion

Ankur Goswami, Akshata Bhat, Hadar Ohana, Theodoros Rekatsinas

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 14 pages, 5 figures, Accepted to Findings of EMNLP 2020

2006.10413 2020-10-13 cs.CL 79%

Are Pretrained Language Models Symbolic Reasoners Over Knowledge?

Nora Kassner, Benno Krojer, Hinrich Schütze

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to CoNLL 2020

2010.01150 2020-10-06 cs.CL 79%

Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media

Xiang Dai, Sarvnaz Karimi, Ben Hachey, Cecile Paris

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Findings of EMNLP 2020

2010.01054 2020-10-05 cs.CL 79%

Unsupervised Text Style Transfer with Padded Masked Language Models

Eric Malmi, Aliaksei Severyn, Sascha Rothe

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2020

2009.10031 2020-09-22 cs.LG cs.CR stat.ML 79%

Training Production Language Models without Memorizing User Data

Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews, Galen Andrew, H. Brendan McMahan, Françoise Beaufays

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

2009.09223 2020-09-22 cs.CL 79%

BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition

Usman Naseem, Matloob Khushi, Vinay Reddy, Sakthivel Rajendran, Imran Razzak, Jinman Kim

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 7 pages

2004.07159 2020-09-22 cs.CL 79%

PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation

Bin Bi, Chenliang Li, Chen Wu, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, Luo Si

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at EMNLP 2020

2004.03012 2020-09-17 cs.CL 79%

"You are grounded!": Latent Name Artifacts in Pre-trained Language Models

Vered Shwartz, Rachel Rudinger, Oyvind Tafjord

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2020

2004.00584 2020-09-04 cs.DB cs.CL 79%

Deep Entity Matching with Pre-Trained Language Models

Yuliang Li, Jinfeng Li, Yoshihiko Suhara, AnHai Doan, Wang-Chiew Tan

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in VLDB 2021

2005.04828 2020-08-19 cs.LG stat.ML 79%

Pretraining Federated Text Models for Next Word Prediction

Joel Stremmel, Arjun Singh

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2008.03088 2020-08-10 eess.AS cs.CL cs.SD 79%

Pretraining Techniques for Sequence-to-Sequence Voice Conversion

Wen-Chin Huang, Tomoki Hayashi, Yi-Chiao Wu, Hirokazu Kameoka, Tomoki Toda

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Preprint. Under review

2008.00401 2020-08-04 cs.CL 79%

Multilingual Translation with Extensible Multilingual Pretraining and Finetuning

Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, Angela Fan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 10 pages (main) + 5 pages (appendices). 9 tables and 2 figures

2005.14165 2020-07-24 cs.CL 79%

Language Models are Few-Shot Learners

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 40+32 pages

1. 预训练与数据 12403 篇

Data Augmentation for Spoken Language Understanding via Pretrained Language Models

Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data

Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification

Bayesian Transformer Language Models for Speech Recognition

Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Network

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification

Code Switching Language Model Using Monolingual Training Data

Pre-Training a Language Model Without Human Language

Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval

Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream Tasks

CPM: A Large-scale Generative Chinese Pre-trained Language Model

Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining

Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis

Unsupervised Subword Modeling Using Autoregressive Pretraining and Cross-Lingual Phone-Aware Modeling

Word Frequency Does Not Predict Grammatical Knowledge in Language Models

Towards Fully Bilingual Deep Language Modeling

Incorporating Stylistic Lexical Preferences in Generative Language Models

Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion

Are Pretrained Language Models Symbolic Reasoners Over Knowledge?

Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media

Unsupervised Text Style Transfer with Padded Masked Language Models

Training Production Language Models without Memorizing User Data

BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition

PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation

"You are grounded!": Latent Name Artifacts in Pre-trained Language Models

Deep Entity Matching with Pre-Trained Language Models

Pretraining Federated Text Models for Next Word Prediction

Pretraining Techniques for Sequence-to-Sequence Voice Conversion

Multilingual Translation with Extensible Multilingual Pretraining and Finetuning

Language Models are Few-Shot Learners