语言大模型 / LLM - arXivDaily 专题

2211.10330 2022-11-21 cs.CL 79%

GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation

Biyang Guo, Yeyun Gong, Yelong Shen, Songqiao Han, Hailiang Huang, Nan Duan, Weizhu Chen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 21 pages

2211.04598 2022-11-10 cs.LG physics.atm-clus 79%

Reducing Down(stream)time: Pretraining Molecular GNNs using Heterogeneous AI Accelerators

Jenna A. Bilbrey, Kristina M. Herman, Henry Sprueill, Soritis S. Xantheas, Payel Das, Manuel Lopez Roldan, Mike Kraus, Hatem Helal, Sutanay Choudhury

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Machine Learning and the Physical Sciences Workshop at the 36th conference on Neural Information Processing Systems (NeurIPS)

2211.04281 2022-11-09 cs.CL 79%

SocioProbe: What, When, and Where Language Models Learn about Sociodemographics

Anne Lauscher, Federico Bianchi, Samuel Bowman, Dirk Hovy

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted for publication at EMNLP 2022

2207.06010 2022-11-03 cs.LG q-bio.BM 79%

Does GNN Pretraining Help Molecular Representation?

Ruoxi Sun, Hanjun Dai, Adams Wei Yu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2210.17236 2022-11-01 cs.PL cs.CL cs.SE 79%

When Language Model Meets Private Library

Daoguang Zan, Bei Chen, Zeqi Lin, Bei Guan, Yongji Wang, Jian-Guang Lou

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2022 Findings

2210.16029 2022-10-31 cs.CL cs.SD eess.AS 79%

Assessing Phrase Break of ESL speech with Pre-trained Language Models

Zhiyi Wang, Shaoguang Mao, Wenshan Wu, Yan Xia

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Under Review, ICASSP 2023

2210.14330 2022-10-27 q-bio.QM cs.AI q-bio.MN 79%

A single-cell gene expression language model

William Connell, Umair Khan, Michael J. Keiser

专题命中预训练与数据：language model(title)；pretraining(abstract)；分类 cs.AI

Comments 10 pages, 5 figures, Accepted at Learning Meaningful Representations of Life Workshop, 36th Conference on Neural Information Processing Systems (NeurIPS 2022)

2205.11482 2022-10-26 cs.CL cs.IR 79%

Towards Tracing Factual Knowledge in Language Models Back to the Training Data

Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, Kelvin Guu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Findings of EMNLP, 2022

2210.12694 2022-10-25 cs.CL 79%

Do Language Models Understand Measurements?

Sungjin Park, Seungwoo Ryu, Edward Choi

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Findings of EMNLP 2022

2210.11165 2022-10-21 cs.CL 79%

Pre-training Language Models with Deterministic Factual Knowledge

Shaobo Li, Xiaoguang Li, Lifeng Shang, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at EMNLP 2022

2210.06525 2022-10-14 cs.CL 79%

Subword Segmental Language Modelling for Nguni Languages

Francois Meyer, Jan Buys

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2206.01670 2022-10-14 cs.CV cs.AI 79%

Egocentric Video-Language Pretraining

Kevin Qinghong Lin, Alex Jinpeng Wang, Mattia Soldan, Michael Wray, Rui Yan, Eric Zhongcong Xu, Difei Gao, Rongcheng Tu, Wenzhe Zhao, Weijie Kong, Chengfei Cai, Hongfa Wang, Dima Damen, Bernard Ghanem, Wei Liu, Mike Zheng Shou

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by NeurIPS 2022. Double champions at Ego4D and EPIC-Kitchens, CVPR 2022 challenges. 23 pages, 13 figures, 12 tables. Code: https://github.com/showlab/EgoVLP

2210.05497 2022-10-12 cs.CL 79%

Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models

Qihuang Zhong, Liang Ding, Li Shen, Peng Mi, Juhua Liu, Bo Du, Dacheng Tao

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2022 (Findings)

2205.12662 2022-10-11 cs.CL 79%

DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning

Zhi Chen, Jijia Bao, Lu Chen, Yuncong Liu, Da Ma, Bei Chen, Mengyue Wu, Su Zhu, Xin Dong, Fujiang Ge, Qingliang Miao, Jian-Guang Lou, Kai Yu

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.CL

Comments Work in Progress

2210.02509 2022-10-07 cs.CL 79%

Revisiting Syllables in Language Modelling and their Application on Low-Resource Machine Translation

Arturo Oncevay, Kervy Dante Rivas Rojas, Liz Karen Chavez Sanchez, Roberto Zariquiey

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments COLING 2022, short-paper

2209.15101 2022-10-03 cs.LG physics.chem-ph q-bio.BM 79%

Improving Molecular Pretraining with Complementary Featurizations

Yanqiao Zhu, Dingshuo Chen, Yuanqi Du, Yingze Wang, Qiang Liu, Shu Wu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 24 pages, work in progress

2209.11102 2022-09-23 cs.CL 79%

Scope of Pre-trained Language Models for Detecting Conflicting Health Information

Joseph Gatto, Madhusudan Basak, Sarah M. Preum

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2209.07068 2022-09-16 cs.CL 79%

uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers

Piji Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments COLING2022,11 pages

2208.11663 2022-08-25 cs.CL 79%

PEER: A Collaborative Language Model

Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, Sebastian Riedel

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2208.07864 2022-08-18 cs.CL 79%

BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification

Vinura Dhananjaya, Piyumal Demotte, Surangika Ranathunga, Sanath Jayasena

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2203.05140 2022-08-15 cs.CL 79%

Speciesist Language and Nonhuman Animal Bias in English Masked Language Models

Masashi Takeshita, Rafal Rzepka, Kenji Araki

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments This paper is an accepted manuscript for publication in Information Processing & Management

2203.11709 2022-08-10 cs.CV cs.AI 79%

CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation

Feng Wang, Huiyu Wang, Chen Wei, Alan Yuille, Wei Shen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2208.01857 2022-08-04 cs.LG math.OC stat.ML 79%

The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift

Jingfeng Wu, Difan Zou, Vladimir Braverman, Quanquan Gu, Sham M. Kakade

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 32 pages, 1 figure, 1 table

2207.09643 2022-07-21 cs.CL 79%

Integrating Linguistic Theory and Neural Language Models

Bai Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments PhD dissertation

2204.02393 2022-07-19 cs.CV cs.LG cs.RO 79%

Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining

Qihang Zhang, Zhenghao Peng, Bolei Zhou

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ECCV accepted paper

2207.07611 2022-07-18 cs.LG cs.CV cs.SD eess.AS 79%

Position Prediction as an Effective Pretraining Strategy

Shuangfei Zhai, Navdeep Jaitly, Jason Ramapuram, Dan Busbridge, Tatiana Likhomanenko, Joseph Yitan Cheng, Walter Talbott, Chen Huang, Hanlin Goh, Joshua Susskind

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to ICML 2022

2207.07116 2022-07-15 cs.CV cs.LG 79%

Bootstrapped Masked Autoencoders for Vision BERT Pretraining

Xiaoyi Dong, Jianmin Bao, Ting Zhang, Dongdong Chen, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, Nenghai Yu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ECCV 2022, code is available at https://github.com/LightDXY/BootMAE

2207.06911 2022-07-15 eess.SP cs.LG 79%

Improving self-supervised pretraining models for epileptic seizure detection from EEG data

Sudip Das, Pankaj Pandey, Krishna Prasad Miyapuram

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2201.07219 2022-07-15 eess.IV cs.CV cs.LG 79%

Contrastive Pretraining for Echocardiography Segmentation with Limited Data

Mohamed Saeed, Rand Muhtaseb, Mohammad Yaqub

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2207.03208 2022-07-13 cs.LG 79%

Revisiting Pretraining Objectives for Tabular Deep Learning

Ivan Rubachev, Artem Alekberov, Yury Gorishniy, Artem Babenko

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Code: https://github.com/puhsu/tabular-dl-pretrain-objectives

1. 预训练与数据 12429 篇

GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation

Reducing Down(stream)time: Pretraining Molecular GNNs using Heterogeneous AI Accelerators

SocioProbe: What, When, and Where Language Models Learn about Sociodemographics

Does GNN Pretraining Help Molecular Representation?

When Language Model Meets Private Library

Assessing Phrase Break of ESL speech with Pre-trained Language Models

A single-cell gene expression language model

Towards Tracing Factual Knowledge in Language Models Back to the Training Data

Do Language Models Understand Measurements?

Pre-training Language Models with Deterministic Factual Knowledge

Subword Segmental Language Modelling for Nguni Languages

Egocentric Video-Language Pretraining

Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models

DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning

Revisiting Syllables in Language Modelling and their Application on Low-Resource Machine Translation

Improving Molecular Pretraining with Complementary Featurizations

Scope of Pre-trained Language Models for Detecting Conflicting Health Information

uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers

PEER: A Collaborative Language Model

BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification

Speciesist Language and Nonhuman Animal Bias in English Masked Language Models

CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation

The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift

Integrating Linguistic Theory and Neural Language Models

Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining

Position Prediction as an Effective Pretraining Strategy

Bootstrapped Masked Autoencoders for Vision BERT Pretraining

Improving self-supervised pretraining models for epileptic seizure detection from EEG data

Contrastive Pretraining for Echocardiography Segmentation with Limited Data

Revisiting Pretraining Objectives for Tabular Deep Learning