语言大模型 / LLM - arXivDaily 专题

2206.11953 2022-06-27 cs.CL cs.AI 73%

Do Trajectories Encode Verb Meaning?

Dylan Ebert, Chen Sun, Ellie Pavlick

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments NAACL 2022

2206.05802 2022-06-15 cs.CL cs.LG 73%

Self-critiquing models for assisting human evaluators

William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, Jan Leike

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2206.04591 2022-06-10 cs.CL cs.CR cs.LG 73%

Privacy Leakage in Text Classification: A Data Extraction Approach

Adel Elmahdy, Huseyin A. Inan, Robert Sim

专题命中预训练与数据：language model(abstract)；post-training(abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 4 tables. Accepted at NAACL 2022 Workshop on Privacy in NLP (PrivateNLP)

2205.10228 2022-05-23 cs.CL cs.CR cs.LG 73%

You Don't Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers' Private Personas

Haoran Li, Yangqiu Song, Lixin Fan

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Conference paper accepted by NAACL 2022

2205.05543 2022-05-12 cs.CV cs.AI cs.LG 73%

An Empirical Study Of Self-supervised Learning Approaches For Object Detection With Transformers

Gokul Karthik Kumar, Sahal Shaji Mullappilly, Abhishek Singh Gehlot

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Final Project for the course "Visual Object Detection And Recognition" (CV703) at MBZUAI

2205.04086 2022-05-10 cs.CL cs.AI 73%

A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank

Dan Malkin, Tomasz Limisiewicz, Gabriel Stanovsky

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments Accepted to NAACL 2022

2205.03873 2022-05-10 cs.CV cs.AI cs.LG 73%

Multimodal Semi-Supervised Learning for Text Recognition

Aviad Aberdam, Roy Ganz, Shai Mazor, Ron Litman

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Code will be published upon publication

2109.04223 2022-05-06 cs.CL cs.AI 73%

KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs

Yinquan Lu, Haonan Lu, Guirong Fu, Qun Liu

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2022 on DLG4NLP

Journal ref ICLR 2022 Workshop on Deep Learning on Graphs for Natural Language Processing,2022

2203.17189 2022-04-01 cs.LG cs.CL 73%

Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$

Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, Andrea Gesmundo

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2203.08242 2022-03-17 cs.CL cs.LG 73%

Data Contamination: From Memorization to Exploitation

Inbal Magar, Roy Schwartz

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Accepted to ACL 2022

2203.07785 2022-03-16 cs.CL cs.AI 73%

The Ghost in the Machine has an American accent: value conflict in GPT-3

Rebecca L Johnson, Giada Pistilli, Natalia Menédez-González, Leslye Denisse Dias Duran, Enrico Panai, Julija Kalpokiene, Donald Jay Bertulfo

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments There are a total of 15 pages of the PDF including 8 pages of the main manuscript, 3 pages of references, and 4 pages of appendices. The paper is currently under review by a conference

2107.07653 2022-03-15 cs.CL cs.AI 73%

TAPEX: Table Pre-training via Learning a Neural SQL Executor

Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, Jian-Guang Lou

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2022 camera ready version

2203.03131 2022-03-08 cs.CL cs.AI 73%

Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models

Shengnan An, Yifei Li, Zeqi Lin, Qian Liu, Bei Chen, Qiang Fu, Weizhu Chen, Nanning Zheng, Jian-Guang Lou

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

2109.07437 2022-02-08 cs.LG cs.CL 73%

Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative

Lucio M. Dery, Paul Michel, Ameet Talwalkar, Graham Neubig

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 18 pages, 4 figures

2102.00238 2022-02-04 cs.CL cs.LG 73%

ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models

Rutuja Taware, Shraddha Varat, Gaurav Salunke, Chaitanya Gawande, Geetanjali Kale, Rahul Khengare, Raviraj Joshi

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

2002.08131 2022-02-03 cs.CL cs.LG 73%

A Systematic Comparison of Architectures for Document-Level Sentiment Classification

Jeremy Barnes, Vinit Ravishankar, Lilja Øvrelid, Erik Velldal

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 5 pages, 2 figures

2110.02950 2021-12-21 cs.CL cs.CY cs.LG 73%

Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer

Wenda Xu, Michael Saxon, Misha Sra, William Yang Wang

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 12 pages, 8 tables, 3 figures. AAAI 2022 Conference Paper

2112.08616 2021-12-17 cs.CL cs.LG 73%

Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context

Daniel Spokoyny, Ivan Lee, Zhao Jin, Taylor Berg-Kirkpatrick

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Preprint

2112.05587 2021-12-16 cs.CV cs.CL cs.LG 73%

Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation

Tianyi Liu, Zuxuan Wu, Wenhan Xiong, Jingjing Chen, Yu-Gang Jiang

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

2108.02446 2021-11-25 cs.CL cs.AI 73%

Finetuning Pretrained Transformers into Variational Autoencoders

Seongmin Park, Jihwa Lee

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments Proceedings of the Second Workshop on Insights from Negative Results in NLP

2111.08137 2021-11-17 cs.CL cs.LG cs.SD eess.AS 73%

Joint Unsupervised and Supervised Training for Multilingual ASR

Junwen Bai, Bo Li, Yu Zhang, Ankur Bapna, Nikhil Siddhartha, Khe Chai Sim, Tara N. Sainath

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

2104.08758 2021-10-01 cs.CL cs.AI 73%

Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus

Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, Matt Gardner

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2021 accepted paper camera ready version

2108.01624 2021-08-04 cs.LG cs.CL cs.CR 73%

Large-Scale Differentially Private BERT

Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, Pasin Manurangsi

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 12 pages, 6 figures

2107.12591 2021-07-28 cs.LG cs.AI 73%

Combining Probabilistic Logic and Deep Learning for Self-Supervised Learning

Hoifung Poon, Hai Wang, Hunter Lang

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Book chapter. arXiv admin note: substantial text overlap with arXiv:2012.12474, arXiv:1808.08485, arXiv:2008.12878

2107.09931 2021-07-22 cs.CL cs.LG 73%

The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding

Archiki Prasad, Mohammad Ali Rehan, Shreya Pathak, Preethi Jyothi

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

2006.03659 2021-05-28 cs.CL cs.LG 73%

DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations

John Giorgi, Osvald Nitski, Bo Wang, Gary Bader

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments ACL2021 Camera Ready V2

2004.07683 2021-03-30 cs.CL cs.LG 73%

Do sequence-to-sequence VAEs learn global features of sentences?

Tom Bosc, Pascal Vincent

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Camera-ready version, EMNLP2020

2008.09075 2021-03-26 cs.CL cs.AI 73%

Controlling Dialogue Generation with Semantic Exemplars

Prakhar Gupta, Jeffrey P. Bigham, Yulia Tsvetkov, Amy Pavel

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at NAACL 2021

2101.10649 2021-01-27 cs.CL cs.AI 73%

Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks

Hyunjin Choi, Judong Kim, Seongho Joe, Seungjai Min, Youngjune Gwon

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments 6 pages, 4 figures, to be published in 25th International Conference on Pattern Recognition, ICPR 2020

2012.00363 2020-12-02 cs.CL cs.LG 73%

Modifying Memories in Transformer Models

Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, Sanjiv Kumar

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

1. 预训练与数据 12460 篇

Do Trajectories Encode Verb Meaning?

Self-critiquing models for assisting human evaluators

Privacy Leakage in Text Classification: A Data Extraction Approach

You Don't Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers' Private Personas

An Empirical Study Of Self-supervised Learning Approaches For Object Detection With Transformers

A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank

Multimodal Semi-Supervised Learning for Text Recognition

KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs

Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$

Data Contamination: From Memorization to Exploitation

The Ghost in the Machine has an American accent: value conflict in GPT-3

TAPEX: Table Pre-training via Learning a Neural SQL Executor

Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models

Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative

ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models

A Systematic Comparison of Architectures for Document-Level Sentiment Classification

Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer

Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context

Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation

Finetuning Pretrained Transformers into Variational Autoencoders

Joint Unsupervised and Supervised Training for Multilingual ASR

Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus

Large-Scale Differentially Private BERT

Combining Probabilistic Logic and Deep Learning for Self-Supervised Learning

The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding

DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations

Do sequence-to-sequence VAEs learn global features of sentences?

Controlling Dialogue Generation with Semantic Exemplars

Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks

Modifying Memories in Transformer Models