语言大模型 / LLM - arXivDaily 专题

2102.01243 2021-11-18 cs.SD cs.LG eess.AS 79%

PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation

Yuan Gong, Yu-An Chung, James Glass

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Published in IEEE/ACM Transactions on Audio Speech and Language Processing. Code at https://github.com/YuanGongND/psla

Journal ref in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3292-3306, 2021

2110.08887 2021-10-19 cs.CL 79%

Schrödinger's Tree -- On Syntax and Neural Language Models

Artur Kulmizev, Joakim Nivre

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments preprint, submitted to Frontiers in Artificial Intelligence: Perspectives for Natural Language Processing between AI, Linguistics and Cognitive Science

1911.01528 2021-10-19 cs.CL 79%

BAS: An Answer Selection Method Using BERT Language Model

Jamshid Mozafari, Afsaneh Fatemi, Mohammad Ali Nematbakhsh

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 28 pages, 19 figures, 8 tables, Journal of Computing and Security

Journal ref Journal of Computing and Security 8 (2021) 1-18

2110.05896 2021-10-15 cs.CL 79%

LaoPLM: Pre-trained Language Models for Lao

Nankai Lin, Yingwen Fu, Chuwei Chen, Ziyu Yang, Shengyi Jiang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2108.07789 2021-10-04 cs.CL cs.SD eess.AS 79%

Adapting GPT, GPT-2 and BERT Language Models for Speech Recognition

Xianrui Zheng, Chao Zhang, Philip C. Woodland

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in ASRU 2021

2105.09996 2021-10-04 cs.CV cs.CL 79%

VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding

Hu Xu, Gargi Ghosh, Po-Yao Huang, Prahal Arora, Masoumeh Aminzadeh, Christoph Feichtenhofer, Florian Metze, Luke Zettlemoyer

专题命中预训练与数据：language model(title)；pretraining(abstract)；分类 cs.CL

Comments 9 pages, ACL Findings 2021

2109.15300 2021-10-01 cs.CL 79%

Semi-Supervised Text Classification via Self-Pretraining

Payam Karisani, Negin Karisani

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments WSDM 2021

2109.12383 2021-09-28 cs.CL 79%

Language Model Priming for Cross-Lingual Event Extraction

Steven Fincke, Shantanu Agarwal, Scott Miller, Elizabeth Boschee

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2109.12028 2021-09-27 cs.CL 79%

Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering

Fahim Faisal, Antonios Anastasopoulos

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments Accepted at MRQA Workshop 2021

2109.11778 2021-09-27 cs.CV cs.CL 79%

Dense Contrastive Visual-Linguistic Pretraining

Lei Shi, Kai Shuang, Shijie Geng, Peng Gao, Zuohui Fu, Gerard de Melo, Yunpeng Chen, Sen Su

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by ACM Multimedia 2021. arXiv admin note: text overlap with arXiv:2007.13135

2109.08354 2021-09-20 cs.CL 79%

Task-adaptive Pre-training of Language Models with Word Embedding Regularization

Kosuke Nishida, Kyosuke Nishida, Sen Yoshida

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL Findings 2021

2109.08249 2021-09-20 cs.CL 79%

Regularized Training of Nearest Neighbor Language Models

Jean-Francois Ton, Walter Talbott, Shuangfei Zhai, Josh Susskind

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2109.07765 2021-09-17 cs.CL 79%

Spanish Biomedical Crawled Corpus: A Large, Diverse Dataset for Spanish Biomedical Language Models

Casimiro Pio Carrino, Jordi Armengol-Estapé, Ona de Gibert Bonet, Asier Gutiérrez-Fandiño, Aitor Gonzalez-Agirre, Martin Krallinger, Marta Villegas

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2108.06209 2021-09-15 cs.LG cs.SD eess.AS 79%

W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training

Yu-An Chung, Yu Zhang, Wei Han, Chung-Cheng Chiu, James Qin, Ruoming Pang, Yonghui Wu

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

2102.05766 2021-09-15 cs.CL 79%

Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation

Renjie Zheng, Junkun Chen, Mingbo Ma, Liang Huang

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

2109.04080 2021-09-14 cs.CL 79%

Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining

Yicheng Zou, Bolin Zhu, Xingwu Hu, Tao Gui, Qi Zhang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2021, 12 pages

2109.04715 2021-09-13 cs.CL 79%

AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages

Machel Reid, Junjie Hu, Graham Neubig, Yutaka Matsuo

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments EMNLP 2021

2109.04666 2021-09-13 cs.CL 79%

Euphemistic Phrase Detection by Masked Language Model

Wanzheng Zhu, Suma Bhat

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2104.07829 2021-09-07 cs.CL 79%

A Masked Segmental Language Model for Unsupervised Natural Language Segmentation

C. M. Downey, Fei Xia, Gina-Anne Levow, Shane Steinert-Threlkeld

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2103.06678 2021-09-07 cs.CL 79%

The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models

Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, Nizar Habash

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to WANLP 2021

2011.13066 2021-09-03 cs.CV cs.AI 79%

USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning

Yixiong Chen, Chunhui Zhang, Li Liu, Cheng Feng, Changfeng Dong, Yongfang Luo, Xiang Wan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2108.13956 2021-09-01 cs.LG 79%

APS: Active Pretraining with Successor Features

Hao Liu, Pieter Abbeel

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Appeared in ICML 2021

2108.12216 2021-08-30 cs.CL 79%

Exploring the Capacity of a Large-scale Masked Language Model to Recognize Grammatical Errors

Ryo Nagata, Manabu Kimura, Kazuaki Hanawa

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2108.10986 2021-08-26 cs.CL 79%

Using BERT Encoding and Sentence-Level Language Model for Sentence Ordering

Melika Golestani, Seyedeh Zahra Razavi, Zeinab Borhanifard, Farnaz Tahmasebian, Hesham Faili

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 12 pages, 2 figures, The 24th International Conference of Text, Speech and Dialogue (TSD2021)

2108.09814 2021-08-24 cs.CL 79%

UzBERT: pretraining a BERT model for Uzbek

B. Mansurov, A. Mansurov

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments 9 pages, 1 table

2108.05540 2021-08-13 cs.IR cs.CL 79%

Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval

Luyu Gao, Jamie Callan

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2104.10972 2021-08-06 cs.CV cs.LG 79%

ImageNet-21K Pretraining for the Masses

Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, Lihi Zelnik-Manor

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to NeurIPS 2021 (Datasets and Benchmarks)

2107.12369 2021-07-27 cs.CV cs.LG 79%

Improve Unsupervised Pretraining for Few-label Transfer

Suichan Li, Dongdong Chen, Yinpeng Chen, Lu Yuan, Lei Zhang, Qi Chu, Bin Liu, Nenghai Yu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ICCV 2021. arXiv admin note: substantial text overlap with arXiv:2012.05899

2106.02965 2021-07-26 cs.LG cs.FL 79%

Extracting Weighted Automata for Approximate Minimization in Language Modelling

Clara Lacroce, Prakash Panangaden, Guillaume Rabusseau

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments Full version of ICGI 2020/21 paper, authors are listed in alphabetical order

2107.09433 2021-07-22 cs.CL 79%

Seed Words Based Data Selection for Language Model Adaptation

Roberto Gretter, Marco Matassoni, Daniele Falavigna

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 11 pages

Journal ref Proceedings of MT Summit 2021 - August 16-20, 2021

1. 预训练与数据 12403 篇

PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation

Schrödinger's Tree -- On Syntax and Neural Language Models

BAS: An Answer Selection Method Using BERT Language Model

LaoPLM: Pre-trained Language Models for Lao

Adapting GPT, GPT-2 and BERT Language Models for Speech Recognition

VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding

Semi-Supervised Text Classification via Self-Pretraining

Language Model Priming for Cross-Lingual Event Extraction

Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering

Dense Contrastive Visual-Linguistic Pretraining

Task-adaptive Pre-training of Language Models with Word Embedding Regularization

Regularized Training of Nearest Neighbor Language Models

Spanish Biomedical Crawled Corpus: A Large, Diverse Dataset for Spanish Biomedical Language Models

W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training

Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation

Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining

AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages

Euphemistic Phrase Detection by Masked Language Model

A Masked Segmental Language Model for Unsupervised Natural Language Segmentation

The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models

USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning

APS: Active Pretraining with Successor Features

Exploring the Capacity of a Large-scale Masked Language Model to Recognize Grammatical Errors

Using BERT Encoding and Sentence-Level Language Model for Sentence Ordering

UzBERT: pretraining a BERT model for Uzbek

Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval

ImageNet-21K Pretraining for the Masses

Improve Unsupervised Pretraining for Few-label Transfer

Extracting Weighted Automata for Approximate Minimization in Language Modelling

Seed Words Based Data Selection for Language Model Adaptation