语言大模型 / LLM - arXivDaily 专题

2302.00083 2023-08-02 cs.CL cs.IR 79%

In-Context Retrieval-Augmented Language Models

Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted for publication in Transactions of the Association for Computational Linguistics (TACL). pre-MIT Press publication version

2307.13560 2023-08-01 cs.CL 79%

XDLM: Cross-lingual Diffusion Language Model for Machine Translation

Linyao Chen, Aosong Feng, Boming Yang, Zihui Li

专题命中预训练与数据：language model(title)；pretraining(abstract)；分类 cs.CL

2106.11483 2023-07-27 cs.CL 79%

A Comprehensive Comparison of Pre-training Language Models

Tong Guo

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2209.10901 2023-07-20 cs.LG 79%

Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning

Manuel Goulão, Arlindo L. Oliveira

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2307.07487 2023-07-17 cs.CV cs.LG 79%

DreamTeacher: Pretraining Image Backbones with Deep Generative Models

Daiqing Li, Huan Ling, Amlan Kar, David Acuna, Seung Wook Kim, Karsten Kreis, Antonio Torralba, Sanja Fidler

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Project page: https://research.nvidia.com/labs/toronto-ai/DreamTeacher/

2306.17649 2023-07-11 cs.CL 79%

Biomedical Language Models are Robust to Sub-optimal Tokenization

Bernal Jiménez Gutiérrez, Huan Sun, Yu Su

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments BioNLP @ ACL 2023

2307.03327 2023-07-10 cs.LG eess.SP 79%

Encoder-Decoder Networks for Self-Supervised Pretraining and Downstream Signal Bandwidth Regression on Digital Antenna Arrays

Rajib Bhattacharjea, Nathan West

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2307.01609 2023-07-06 cs.CL 79%

A Language Model for Grammatical Error Correction in L2 Russian

Nikita Remnev, Sergei Obiedkov, Ekaterina Rakhilina, Ivan Smirnov, Anastasia Vyrenkova

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2305.09790 2023-06-28 q-bio.BM cs.LG 79%

Molecule-Morphology Contrastive Pretraining for Transferable Molecular Representation

Cuong Q. Nguyen, Dante Pertusi, Kim M. Branson

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ICML 2023 Workshop on Computational Biology

2302.05895 2023-06-27 cs.CL 79%

Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues

Chuyuan Li, Patrick Huber, Wen Xiao, Maxime Amblard, Chloé Braud, Giuseppe Carenini

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Journal ref Findings of the Association for Computational Linguistics: EACL 2023 (2023) 2562--2579

2303.15693 2023-06-14 cs.CV cs.LG eess.IV 79%

Large-scale pretraining on pathological images for fine-tuning of small pathological benchmarks

Masataka Kawai, Noriaki Ota, Shinsuke Yamaoka

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 20 pages, 6 figures

2306.06892 2023-06-13 cs.CL 79%

On the N-gram Approximation of Pre-trained Language Models

Aravind Krishnan, Jesujoba Alabi, Dietrich Klakow

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at Interspeech 2023

2306.06598 2023-06-13 cs.CL 79%

RoBERTweet: A BERT Language Model for Romanian Tweets

Iulian-Marius Tăiatu, Andrei-Marius Avram, Dumitru-Clementin Cercel, Florin Pop

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at NLDB2023

2212.10449 2023-06-12 cs.CL 79%

Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization

Artidoro Pagnoni, Alexander R. Fabbri, Wojciech Kryściński, Chien-Sheng Wu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments To appear at ACL 2023

2110.15444 2023-06-12 cs.CR cs.LG 79%

10 Security and Privacy Problems in Large Foundation Models

Jinyuan Jia, Hongbin Liu, Neil Zhenqiang Gong

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments A book chapter

2306.04233 2023-06-08 cs.CL cs.SD eess.AS 79%

Transfer Learning from Pre-trained Language Models Improves End-to-End Speech Summarization

Kohei Matsuura, Takanori Ashihara, Takafumi Moriya, Tomohiro Tanaka, Takatomo Kano, Atsunori Ogawa, Marc Delcroix

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted by Interspeech 2023

2206.14268 2023-06-05 cs.CL 79%

BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models

Shibo Hao, Bowen Tan, Kaiwen Tang, Bin Ni, Xiyan Shao, Hengzhe Zhang, Eric P. Xing, Zhiting Hu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2023 (Findings); Code available at https://github.com/tanyuqian/knowledge-harvest-from-lms

2306.00931 2023-06-02 cs.CV cs.CL 79%

"Let's not Quote out of Context": Unified Vision-Language Pretraining for Context Assisted Image Captioning

Abisek Rajakumar Kalarani, Pushpak Bhattacharyya, Niyati Chhaya, Sumit Shekhar

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2305.19905 2023-06-01 cs.CL 79%

How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases

Aaron Mueller, Tal Linzen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2023

2305.17266 2023-06-01 cs.CL 79%

Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale

Vijeta Deshpande, Dan Pechi, Shree Thatte, Vladislav Lialin, Anna Rumshisky

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2023 Findings

2208.01006 2023-06-01 cs.CL 79%

Multi-Document Summarization with Centroid-Based Pretraining

Ratish Puduppully, Parag Jain, Nancy F. Chen, Mark Steedman

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments ACL 2023 camera-ready

2305.17540 2023-05-31 cs.CV cs.CL 79%

Learning from Children: Improving Image-Caption Pretraining via Curriculum

Hammad A. Ayyubi, Rahul Lokesh, Alireza Zareian, Bo Wu, Shih-Fu Chang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments ACL Findings 2023

2305.18294 2023-05-30 cs.CL 79%

Transformer Language Models Handle Word Frequency in Prediction Head

Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 11 pages, 12 figures, accepted to ACL 2023 Findings (short paper)

2305.17179 2023-05-30 cs.CL 79%

Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages

Tomasz Limisiewicz, Jiří Balhar, David Mareček

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments in ACL Findings 2023

2305.17019 2023-05-29 cs.CL 79%

Commonsense Knowledge Graph Completion Via Contrastive Pretraining and Node Clustering

Siwei Wu, Xiangqing Shen, Rui Xia

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Findings of ACL 2023

2305.16683 2023-05-29 cs.LG 79%

Future-conditioned Unsupervised Pretraining for Decision Transformer

Zhihui Xie, Zichuan Lin, Deheng Ye, Qiang Fu, Wei Yang, Shuai Li

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 17 pages, 9 figures, ICML 2023

2305.16157 2023-05-26 cs.CL 79%

Training Data Extraction From Pre-trained Language Models: A Survey

Shotaro Ishihara

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments TrustNLP workshop at ACL 2023

2210.01703 2023-05-25 cs.SD cs.HC cs.LG eess.AS 79%

Improving Label-Deficient Keyword Spotting Through Self-Supervised Pretraining

Holger Severin Bovbjerg, Zheng-Hua Tan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2305.13707 2023-05-24 cs.CL 79%

Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Jungo Kasai, David R. Mortensen, Noah A. Smith, Yulia Tsvetkov

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2211.01335 2023-05-24 cs.CV cs.CL 79%

Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese

An Yang, Junshu Pan, Junyang Lin, Rui Men, Yichang Zhang, Jingren Zhou, Chang Zhou

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

1. 预训练与数据 12403 篇

In-Context Retrieval-Augmented Language Models

XDLM: Cross-lingual Diffusion Language Model for Machine Translation

A Comprehensive Comparison of Pre-training Language Models

Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning

DreamTeacher: Pretraining Image Backbones with Deep Generative Models

Biomedical Language Models are Robust to Sub-optimal Tokenization

Encoder-Decoder Networks for Self-Supervised Pretraining and Downstream Signal Bandwidth Regression on Digital Antenna Arrays

A Language Model for Grammatical Error Correction in L2 Russian

Molecule-Morphology Contrastive Pretraining for Transferable Molecular Representation

Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues

Large-scale pretraining on pathological images for fine-tuning of small pathological benchmarks

On the N-gram Approximation of Pre-trained Language Models

RoBERTweet: A BERT Language Model for Romanian Tweets

Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization

10 Security and Privacy Problems in Large Foundation Models

Transfer Learning from Pre-trained Language Models Improves End-to-End Speech Summarization

BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models

"Let's not Quote out of Context": Unified Vision-Language Pretraining for Context Assisted Image Captioning

How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases

Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale

Multi-Document Summarization with Centroid-Based Pretraining

Learning from Children: Improving Image-Caption Pretraining via Curriculum

Transformer Language Models Handle Word Frequency in Prediction Head

Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages

Commonsense Knowledge Graph Completion Via Contrastive Pretraining and Node Clustering

Future-conditioned Unsupervised Pretraining for Decision Transformer

Training Data Extraction From Pre-trained Language Models: A Survey

Improving Label-Deficient Keyword Spotting Through Self-Supervised Pretraining

Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese