语言大模型 / LLM - arXivDaily 专题

2107.08582 2021-07-20 cs.CL 79%

Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision

Ning Bian, Xianpei Han, Bo Chen, Hongyu Lin, Ben He, Le Sun

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2104.08671 2021-07-07 cs.CL 79%

When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset

Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, Daniel E. Ho

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments ICAIL 2021. Code & data available at https://github.com/reglab/casehold

2106.13627 2021-06-28 cs.CL 79%

Language Models are Good Translators

Shuo Wang, Zhaopeng Tu, Zhixing Tan, Wenxuan Wang, Maosong Sun, Yang Liu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 12 pages. Work in progress. An earlier verison of this manuscript is under review

2106.10124 2021-06-21 cs.LG 79%

Graph Context Encoder: Graph Feature Inpainting for Graph Generation and Self-supervised Pretraining

Oriel Frigo, Rémy Brossard, David Dehaene

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 13 pages, 4 figures

2106.09065 2021-06-18 cs.CV cs.LG 79%

SPeCiaL: Self-Supervised Pretraining for Continual Learning

Lucas Caccia, Joelle Pineau

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2012.15525 2021-06-17 cs.CL 79%

BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining

Weizhen Qi, Yeyun Gong, Jian Jiao, Yu Yan, Weizhu Chen, Dayiheng Liu, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, Nan Duan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by ICML 2021

2105.07144 2021-06-11 cs.CL 79%

A Cognitive Regularizer for Language Modeling

Jason Wei, Clara Meister, Ryan Cotterell

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2021 Camera-ready (fixed ordering of affiliation emojis)

2106.04799 2021-06-10 cs.LG 79%

Pretraining Representations for Data-Efficient Reinforcement Learning

Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, Devon Hjelm, Philip Bachman, Aaron Courville

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2106.04121 2021-06-09 cs.CV cs.LG 79%

Multi-dataset Pretraining: A Unified Model for Semantic Segmentation

Bowen Shi, Xiaopeng Zhang, Haohang Xu, Wenrui Dai, Junni Zou, Hongkai Xiong, Qi Tian

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2106.04098 2021-06-09 cs.CL 79%

Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

Hongliang Dai, Yangqiu Song, Haixun Wang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2021

2104.04552 2021-06-08 cs.CL cs.SD eess.AS 79%

Lookup-Table Recurrent Language Models for Long Tail Speech Recognition

W. Ronny Huang, Tara N. Sainath, Cal Peyser, Shankar Kumar, David Rybach, Trevor Strohman

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Presented as conference paper at Interspeech 2021

2106.01091 2021-06-03 cs.CL 79%

belabBERT: a Dutch RoBERTa-based language model applied to psychiatric classification

Joppe Wouts, Janna de Boer, Alban Voppel, Sanne Brederoo, Sander van Splunter, Iris Sommer

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments arXiv admin note: substantial text overlap with arXiv:2008.01543

2106.00737 2021-06-03 cs.CL 79%

Implicit Representations of Meaning in Neural Language Models

Belinda Z. Li, Maxwell Nye, Jacob Andreas

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 15 pages, 6 figures; accepted to ACL 2021

2101.00438 2021-06-03 cs.CL 79%

Few-Shot Question Answering by Pretraining Span Selection

Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, Omer Levy

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2021

2011.15124 2021-06-01 cs.CL cs.CV 79%

Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs

Emanuele Bugliarello, Ryan Cotterell, Naoaki Okazaki, Desmond Elliott

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments To appear in TACL 2021

2105.12810 2021-05-28 cs.CV cs.LG 79%

ViPTT-Net: Video pretraining of spatio-temporal model for tuberculosis type classification from chest CT scans

Hasib Zunair, Aimon Rahman, Nabeel Mohammed

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Under review at CLEF 2021. 10 pages

2011.04393 2021-05-26 cs.CL 79%

Positional Artefacts Propagate Through Masked Language Model Embeddings

Ziyang Luo, Artur Kulmizev, Xiaoxi Mao

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted as long paper at ACL 2021

2003.07441 2021-05-19 cs.CV cs.LG 79%

Pretraining Image Encoders without Reconstruction via Feature Prediction Loss

Gustav Grund Pihlgren, Fredrik Sandin, Marcus Liwicki

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2105.07149 2021-05-18 cs.CL 79%

DirectQE: Direct Pretraining for Machine Translation Quality Estimation

Qu Cui, Shujian Huang, Jiahuan Li, Xiang Geng, Zaixiang Zheng, Guoping Huang, Jiajun Chen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2105.03519 2021-05-11 cs.CL 79%

Understanding by Understanding Not: Modeling Negation in Language Models

Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni, Aaron Courville

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2003.13198 2021-04-23 cs.CL cs.CV 79%

InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining

Junyang Lin, An Yang, Yichang Zhang, Jie Liu, Jingren Zhou, Hongxia Yang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 11 pages

2104.08405 2021-04-20 cs.CL cs.CV cs.IR 79%

LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding

Te-Lin Wu, Cheng Li, Mingyang Zhang, Tao Chen, Spurthi Amba Hombaiah, Michael Bendersky

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2104.07496 2021-04-16 cs.CL 79%

Unmasking the Mask -- Evaluating Social Biases in Masked Language Models

Masahiro Kaneko, Danushka Bollegala

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2104.04950 2021-04-13 cs.CL cs.SD eess.AS 79%

Innovative Bert-based Reranking Language Models for Speech Recognition

Shih-Hsuan Chiu, Berlin Chen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 6 pages, 3 figures, Published in IEEE SLT 2021

2102.06551 2021-04-13 cs.CL 79%

A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages

Jivnesh Sandhan, Amrith Krishna, Ashim Gupta, Laxmidhar Behera, Pawan Goyal

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 6 pages, The work is accepted at EACL-SRW, 2021, Kyiv, Ukraine Typos corrected in Section 3.2

2010.10077 2021-04-13 cs.CL 79%

Neural Language Modeling for Contextualized Temporal Graph Generation

Aman Madaan, Yiming Yang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments NAACL 2021

2012.06262 2021-03-29 cs.CL 79%

Morphology Matters: A Multilingual Language Modeling Analysis

Hyunji Hayley Park, Katherine J. Zhang, Coleman Haley, Kenneth Steimel, Han Liu, Lane Schwartz

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in TACL, a pre-MIT Press publication version; 15 pages, 3 figures; for the datasets, see https://github.com/hayleypark/MorphologyMatters

Journal ref Transactions of the Association for Computational Linguistics 9 (2021) 261-276

2103.08251 2021-03-16 cs.CV cs.AI 79%

Boosting ship detection in SAR images with complementary pretraining techniques

Wei Bao, Meiyu Huang, Yaqin Zhang, Yao Xu, Xuejiao Liu, Xueshuang Xiang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2103.08157 2021-03-16 cs.LG 79%

Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning

Kwanghee Choi, Minyoung Choe, Hyelee Lee

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to NeurIPS 2020 Workshop: Self-Supervised Learning - Theory and Practice

2010.12688 2021-03-16 cs.CL 79%

Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training

Oshin Agarwal, Heming Ge, Siamak Shakeri, Rami Al-Rfou

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at NAACL 2021

1. 预训练与数据 12403 篇

Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision

When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset

Language Models are Good Translators

Graph Context Encoder: Graph Feature Inpainting for Graph Generation and Self-supervised Pretraining

SPeCiaL: Self-Supervised Pretraining for Continual Learning

BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining

A Cognitive Regularizer for Language Modeling

Pretraining Representations for Data-Efficient Reinforcement Learning

Multi-dataset Pretraining: A Unified Model for Semantic Segmentation

Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

Lookup-Table Recurrent Language Models for Long Tail Speech Recognition

belabBERT: a Dutch RoBERTa-based language model applied to psychiatric classification

Implicit Representations of Meaning in Neural Language Models

Few-Shot Question Answering by Pretraining Span Selection

Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs

ViPTT-Net: Video pretraining of spatio-temporal model for tuberculosis type classification from chest CT scans

Positional Artefacts Propagate Through Masked Language Model Embeddings

Pretraining Image Encoders without Reconstruction via Feature Prediction Loss

DirectQE: Direct Pretraining for Machine Translation Quality Estimation

Understanding by Understanding Not: Modeling Negation in Language Models

InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining

LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding

Unmasking the Mask -- Evaluating Social Biases in Masked Language Models

Innovative Bert-based Reranking Language Models for Speech Recognition

A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages

Neural Language Modeling for Contextualized Temporal Graph Generation

Morphology Matters: A Multilingual Language Modeling Analysis

Boosting ship detection in SAR images with complementary pretraining techniques

Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning

Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training