语言大模型 / LLM - arXivDaily 专题

2209.07068 2022-09-16 cs.CL 79%

uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers

Piji Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments COLING2022,11 pages

2208.11663 2022-08-25 cs.CL 79%

PEER: A Collaborative Language Model

Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, Sebastian Riedel

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2208.07864 2022-08-18 cs.CL 79%

BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification

Vinura Dhananjaya, Piyumal Demotte, Surangika Ranathunga, Sanath Jayasena

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2203.05140 2022-08-15 cs.CL 79%

Speciesist Language and Nonhuman Animal Bias in English Masked Language Models

Masashi Takeshita, Rafal Rzepka, Kenji Araki

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments This paper is an accepted manuscript for publication in Information Processing & Management

2203.11709 2022-08-10 cs.CV cs.AI 79%

CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation

Feng Wang, Huiyu Wang, Chen Wei, Alan Yuille, Wei Shen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2208.01857 2022-08-04 cs.LG math.OC stat.ML 79%

The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift

Jingfeng Wu, Difan Zou, Vladimir Braverman, Quanquan Gu, Sham M. Kakade

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 32 pages, 1 figure, 1 table

2207.09643 2022-07-21 cs.CL 79%

Integrating Linguistic Theory and Neural Language Models

Bai Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments PhD dissertation

2204.02393 2022-07-19 cs.CV cs.LG cs.RO 79%

Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining

Qihang Zhang, Zhenghao Peng, Bolei Zhou

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ECCV accepted paper

2207.07611 2022-07-18 cs.LG cs.CV cs.SD eess.AS 79%

Position Prediction as an Effective Pretraining Strategy

Shuangfei Zhai, Navdeep Jaitly, Jason Ramapuram, Dan Busbridge, Tatiana Likhomanenko, Joseph Yitan Cheng, Walter Talbott, Chen Huang, Hanlin Goh, Joshua Susskind

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to ICML 2022

2207.07116 2022-07-15 cs.CV cs.LG 79%

Bootstrapped Masked Autoencoders for Vision BERT Pretraining

Xiaoyi Dong, Jianmin Bao, Ting Zhang, Dongdong Chen, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, Nenghai Yu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ECCV 2022, code is available at https://github.com/LightDXY/BootMAE

2207.06911 2022-07-15 eess.SP cs.LG 79%

Improving self-supervised pretraining models for epileptic seizure detection from EEG data

Sudip Das, Pankaj Pandey, Krishna Prasad Miyapuram

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2201.07219 2022-07-15 eess.IV cs.CV cs.LG 79%

Contrastive Pretraining for Echocardiography Segmentation with Limited Data

Mohamed Saeed, Rand Muhtaseb, Mohammad Yaqub

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2207.03208 2022-07-13 cs.LG 79%

Revisiting Pretraining Objectives for Tabular Deep Learning

Ivan Rubachev, Artem Alekberov, Yury Gorishniy, Artem Babenko

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Code: https://github.com/puhsu/tabular-dl-pretrain-objectives

2207.03637 2022-07-11 cs.CL 79%

OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

Zhengbao Jiang, Yi Mao, Pengcheng He, Graham Neubig, Weizhu Chen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments NAACL 2022

2207.01718 2022-07-06 cs.CL eess.AS 79%

BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model

Brooke Stephenson, Laurent Besacier, Laurent Girin, Thomas Hueber

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 5 pages

2204.05188 2022-07-04 cs.CL cs.SD eess.AS 79%

Tokenwise Contrastive Pretraining for Finer Speech-to-BERT Alignment in End-to-End Speech-to-Intent Systems

Vishal Sunder, Eric Fosler-Lussier, Samuel Thomas, Hong-Kwang J. Kuo, Brian Kingsbury

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 5 pages, 2 figures

2109.14989 2022-06-30 cs.CL 79%

Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations

Arabella Sinclair, Jaap Jumelet, Willem Zuidema, Raquel Fernández

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Published in TACL, MIT Press

2203.09690 2022-06-22 eess.AS cs.CL cs.SD 79%

A$^3$T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing

He Bai, Renjie Zheng, Junkun Chen, Xintong Li, Mingbo Ma, Liang Huang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by ICML 2022, 12 pages, 10 figures

2110.14378 2022-06-09 cs.AI 79%

Towards artificial general intelligence via a multimodal foundation model

Nanyi Fei, Zhiwu Lu, Yizhao Gao, Guoxing Yang, Yuqi Huo, Jingyuan Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, Hao Sun, Ji-Rong Wen

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

Comments Published by Nature Communications, see https://www.nature.com/articles/s41467-022-30761-2

2110.13492 2022-06-08 cs.LG cs.SD eess.AS 79%

TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining

Viet-Anh Nguyen, Anh H. T. Nguyen, Andy W. H. Khong

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Published as a conference paper at ICASSP 2022, 5 pages, 4 figures, 3 tables

2205.13607 2022-06-06 cs.LG cs.HC 79%

Self-supervised Pretraining and Transfer Learning Enable Flu and COVID-19 Predictions in Small Mobile Sensing Datasets

Mike A. Merrill, Tim Althoff

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2205.10233 2022-06-06 cs.CL 79%

RigoBERTa: A State-of-the-Art Language Model For Spanish

Alejandro Vaca Serrano, Guillem Garcia Subies, Helena Montoro Zamorano, Nuria Aldama Garcia, Doaa Samy, David Betancur Sanchez, Antonio Moreno Sandoval, Marta Guerrero Nieto, Alvaro Barbero Jimenez

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2012.05776 2022-06-02 cs.CL 79%

Multi-Sense Language Modelling

Andrea Lekkas, Peter Schneider-Kamp, Isabelle Augenstein

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2205.07083 2022-05-17 eess.AS cs.CL 79%

Pretraining Approaches for Spoken Language Recognition: TalTech Submission to the OLR 2021 Challenge

Tanel Alumäe, Kunnar Kukk

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted to Speaker Odyssey 2022

2205.03403 2022-05-09 cs.CL 79%

A Data Cartography based MixUp for Pre-trained Language Models

Seo Yeon Park, Cornelia Caragea

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at NAACL 2022 main conference. arXiv admin note: text overlap with arXiv:2203.07559

2110.08743 2022-05-05 cs.CL 79%

GNN-LM: Language Modeling based on Global Contexts via GNN

Yuxian Meng, Shi Zong, Xiaoya Li, Xiaofei Sun, Tianwei Zhang, Fei Wu, Jiwei Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear at ICLR 2022

2202.02093 2022-05-05 cs.CL 79%

Temporal Attention for Language Models

Guy D. Rosin, Kira Radinsky

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Findings of NAACL 2022. 9 pages

2205.00320 2022-05-03 cs.CL 79%

Detoxifying Language Models with a Toxic Corpus

Yoon A Park, Frank Rudzicz

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2204.10939 2022-04-29 cs.CL cs.CV 79%

Unified Pretraining Framework for Document Understanding

Jiuxiang Gu, Jason Kuen, Vlad I. Morariu, Handong Zhao, Nikolaos Barmpalios, Rajiv Jain, Ani Nenkova, Tong Sun

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 12 pages, 4 figures, NeurIPS 2021 (Updated Camera Ready)

2204.11989 2022-04-27 cs.IR cs.CL 79%

C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrieval

Eugene Yang, Suraj Nair, Ramraj Chandradevan, Rebecca Iglesias-Flores, Douglas W. Oard

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments 6 pages, 2 figures, accepted as a SIGIR 2022 Short Paper

1. 预训练与数据 12403 篇

uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers

PEER: A Collaborative Language Model

BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification

Speciesist Language and Nonhuman Animal Bias in English Masked Language Models

CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation

The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift

Integrating Linguistic Theory and Neural Language Models

Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining

Position Prediction as an Effective Pretraining Strategy

Bootstrapped Masked Autoencoders for Vision BERT Pretraining

Improving self-supervised pretraining models for epileptic seizure detection from EEG data

Contrastive Pretraining for Echocardiography Segmentation with Limited Data

Revisiting Pretraining Objectives for Tabular Deep Learning

OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model

Tokenwise Contrastive Pretraining for Finer Speech-to-BERT Alignment in End-to-End Speech-to-Intent Systems

Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations

A$^3$T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing

Towards artificial general intelligence via a multimodal foundation model

TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining

Self-supervised Pretraining and Transfer Learning Enable Flu and COVID-19 Predictions in Small Mobile Sensing Datasets

RigoBERTa: A State-of-the-Art Language Model For Spanish

Multi-Sense Language Modelling

Pretraining Approaches for Spoken Language Recognition: TalTech Submission to the OLR 2021 Challenge

A Data Cartography based MixUp for Pre-trained Language Models

GNN-LM: Language Modeling based on Global Contexts via GNN

Temporal Attention for Language Models

Detoxifying Language Models with a Toxic Corpus

Unified Pretraining Framework for Document Understanding

C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrieval