语言大模型 / LLM

Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization

Artidoro Pagnoni, Alexander R. Fabbri, Wojciech Kryściński, Chien-Sheng Wu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments To appear at ACL 2023

10 Security and Privacy Problems in Large Foundation Models

Jinyuan Jia, Hongbin Liu, Neil Zhenqiang Gong

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

Comments A book chapter

Transfer Learning from Pre-trained Language Models Improves End-to-End Speech Summarization

Kohei Matsuura, Takanori Ashihara, Takafumi Moriya, Tomohiro Tanaka, Takatomo Kano, Atsunori Ogawa, Marc Delcroix

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted by Interspeech 2023

BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models

Shibo Hao, Bowen Tan, Kaiwen Tang, Bin Ni, Xiyan Shao, Hengzhe Zhang, Eric P. Xing, Zhiting Hu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments ACL 2023 (Findings); Code available at https://github.com/tanyuqian/knowledge-harvest-from-lms

"Let's not Quote out of Context": Unified Vision-Language Pretraining for Context Assisted Image Captioning

Abisek Rajakumar Kalarani, Pushpak Bhattacharyya, Niyati Chhaya, Sumit Shekhar

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases

Aaron Mueller, Tal Linzen

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2023

Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale

Vijeta Deshpande, Dan Pechi, Shree Thatte, Vladislav Lialin, Anna Rumshisky

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2023 Findings

Multi-Document Summarization with Centroid-Based Pretraining

Ratish Puduppully, Parag Jain, Nancy F. Chen, Mark Steedman

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments ACL 2023 camera-ready

Learning from Children: Improving Image-Caption Pretraining via Curriculum

Hammad A. Ayyubi, Rahul Lokesh, Alireza Zareian, Bo Wu, Shih-Fu Chang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments ACL Findings 2023

Transformer Language Models Handle Word Frequency in Prediction Head

Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments 11 pages, 12 figures, accepted to ACL 2023 Findings (short paper)

Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages

Tomasz Limisiewicz, Jiří Balhar, David Mareček

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments in ACL Findings 2023

Commonsense Knowledge Graph Completion Via Contrastive Pretraining and Node Clustering

Siwei Wu, Xiangqing Shen, Rui Xia

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Findings of ACL 2023

Future-conditioned Unsupervised Pretraining for Decision Transformer

Zhihui Xie, Zichuan Lin, Deheng Ye, Qiang Fu, Wei Yang, Shuai Li

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 17 pages, 9 figures, ICML 2023

Training Data Extraction From Pre-trained Language Models: A Survey

Shotaro Ishihara

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments TrustNLP workshop at ACL 2023

Improving Label-Deficient Keyword Spotting Through Self-Supervised Pretraining

Holger Severin Bovbjerg, Zheng-Hua Tan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments To be published at ICASSP2023 Workshop on Self-supervision in Audio, Speech and Beyond, 10th of June 2023, Rhodes, Greece. Copyright (c) 2023 IEEE. 5 pages, 3 figures, 3 tables

Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Jungo Kasai, David R. Mortensen, Noah A. Smith, Yulia Tsvetkov

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese

An Yang, Junshu Pan, Junyang Lin, Rui Men, Yichang Zhang, Jingren Zhou, Chang Zhou

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Language-Universal Phonetic Representation in Multilingual Speech Pretraining for Low-Resource Speech Recognition

Siyuan Feng, Ming Tu, Rui Xia, Chuanzeng Huang, Yuxuan Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Accepted for publication in INTERSPEECH 2023

Self-Supervised Pretraining on Paired Sequences of fMRI Data for Transfer Learning to Brain Decoding Tasks

Sean Paulsen, Michael Casey

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments Preprint - Accepted to International Conference on Pattern Recognition, Machine Learning and Consciousness 2023

Comprehensive Solution Program Centric Pretraining for Table-and-Text Hybrid Numerical Reasoning

Qianying Liu, Dongsheng Yang, Wenjie Zhong, Fei Cheng, Sadao Kurohashi

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments 11 pages

IUST_NLP at SemEval-2023 Task 10: Explainable Detecting Sexism with Transformers and Task-adaptive Pretraining

Hadiseh Mahmoudi

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments 6 pages, 7 figures, Accepted at The 17th International Workshop on Semantic Evaluation, ACL 2023

mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video

Haiyang Xu, Qinghao Ye, Ming Yan, Yaya Shi, Jiabo Ye, Yuanhong Xu, Chenliang Li, Bin Bi, Qi Qian, Wei Wang, Guohai Xu, Ji Zhang, Songfang Huang, Fei Huang, Jingren Zhou

专题命中 预训练与数据 ：foundation model(title)；pretraining(abstract)；分类 cs.CL

Journal ref ICML2023

XTab: Cross-table Pretraining for Tabular Transformers

Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, Mahsa Shoaran

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Analyzing Leakage of Personally Identifiable Information in Language Models

Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, Santiago Zanella-Béguelin

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.LG

Comments IEEE Symposium on Security and Privacy (S&P) 2023

Movie Box Office Prediction With Self-Supervised and Visually Grounded Pretraining

Qin Chao, Eunsoo Kim, Boyang Li

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments accepted by IEEE International Conference on Multimedia and Expo (2023)

Towards Better Instruction Following Language Models for Chinese: Investigating the Impact of Training Data and Evaluation

Yunjie Ji, Yan Gong, Yong Deng, Yiping Peng, Qiang Niu, Baochang Ma, Xiangang Li

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation

Florian Bordes, Samuel Lavoie, Randall Balestriero, Nicolas Ballas, Pascal Vincent

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Exploring the Utility of Self-Supervised Pretraining Strategies for the Detection of Absent Lung Sliding in M-Mode Lung Ultrasound

Blake VanBerlo, Brian Li, Alexander Wong, Jesse Hoey, Robert Arntfield

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 10 pages, 6 figures, to be published in the 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations

Björn Plüster, Jakob Ambsdorf, Lukas Braach, Jae Hee Lee, Stefan Wermter

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Minor changes

Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm

Yichen Xie, Han Lu, Junchi Yan, Xiaokang Yang, Masayoshi Tomizuka, Wei Zhan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments Accepted by CVPR 2023