语言大模型 / LLM

Language-Universal Phonetic Representation in Multilingual Speech Pretraining for Low-Resource Speech Recognition

Siyuan Feng, Ming Tu, Rui Xia, Chuanzeng Huang, Yuxuan Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Accepted for publication in INTERSPEECH 2023

Self-Supervised Pretraining on Paired Sequences of fMRI Data for Transfer Learning to Brain Decoding Tasks

Sean Paulsen, Michael Casey

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments Preprint - Accepted to International Conference on Pattern Recognition, Machine Learning and Consciousness 2023

Comprehensive Solution Program Centric Pretraining for Table-and-Text Hybrid Numerical Reasoning

Qianying Liu, Dongsheng Yang, Wenjie Zhong, Fei Cheng, Sadao Kurohashi

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments 11 pages

IUST_NLP at SemEval-2023 Task 10: Explainable Detecting Sexism with Transformers and Task-adaptive Pretraining

Hadiseh Mahmoudi

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments 6 pages, 7 figures, Accepted at The 17th International Workshop on Semantic Evaluation, ACL 2023

mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video

Haiyang Xu, Qinghao Ye, Ming Yan, Yaya Shi, Jiabo Ye, Yuanhong Xu, Chenliang Li, Bin Bi, Qi Qian, Wei Wang, Guohai Xu, Ji Zhang, Songfang Huang, Fei Huang, Jingren Zhou

专题命中 预训练与数据 ：foundation model(title)；pretraining(abstract)；分类 cs.CL

Journal ref ICML2023

XTab: Cross-table Pretraining for Tabular Transformers

Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, Mahsa Shoaran

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Analyzing Leakage of Personally Identifiable Information in Language Models

Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, Santiago Zanella-Béguelin

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.LG

Comments IEEE Symposium on Security and Privacy (S&P) 2023

Movie Box Office Prediction With Self-Supervised and Visually Grounded Pretraining

Qin Chao, Eunsoo Kim, Boyang Li

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments accepted by IEEE International Conference on Multimedia and Expo (2023)

Towards Better Instruction Following Language Models for Chinese: Investigating the Impact of Training Data and Evaluation

Yunjie Ji, Yan Gong, Yong Deng, Yiping Peng, Qiang Niu, Baochang Ma, Xiangang Li

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation

Florian Bordes, Samuel Lavoie, Randall Balestriero, Nicolas Ballas, Pascal Vincent

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Exploring the Utility of Self-Supervised Pretraining Strategies for the Detection of Absent Lung Sliding in M-Mode Lung Ultrasound

Blake VanBerlo, Brian Li, Alexander Wong, Jesse Hoey, Robert Arntfield

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 10 pages, 6 figures, to be published in the 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations

Björn Plüster, Jakob Ambsdorf, Lukas Braach, Jae Hee Lee, Stefan Wermter

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Minor changes

Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm

Yichen Xie, Han Lu, Junchi Yan, Xiaokang Yang, Masayoshi Tomizuka, Wei Zhan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments Accepted by CVPR 2023

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions

Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, Yoav Goldberg

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments We received a criticism regarding the validity of the causal formulation in this paper. We will address them in an upcoming version

CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching

Xiaoshi Wu, Feng Zhu, Rui Zhao, Hongsheng Li

专题命中 预训练与数据 ：prompting(title,abstract)；分类 cs.AI

Comments 11 pages, 4 figures. Accepted by CVPR 2023

Suffix Retrieval-Augmented Language Modeling

Zecheng Wang, Yik-Cheung Tam

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments 5 pages, 1 figure. Submitted to ICASSP 2023

Spelling convention sensitivity in neural language models

Elizabeth Nielsen, Christo Kirov, Brian Roark

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Journal ref EACL Findings 2023

IPA-CLIP: Integrating Phonetic Priors into Vision and Language Pretraining

Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments 11 pages, 8 figures, 5 Tables

Transparency Helps Reveal When Language Models Learn Meaning

Zhaofeng Wu, William Merrill, Hao Peng, Iz Beltagy, Noah A. Smith

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted for publication in Transactions of the Association for Computational Linguistics (TACL), 2023. Author's final version (pre-MIT Press publication)

On the Provable Advantage of Unsupervised Pretraining

Jiawei Ge, Shange Tang, Jianqing Fan, Chi Jin

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching

Shengchao Liu, Hongyu Guo, Jian Tang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context

Alexandre Pasquiou, Yair Lakretz, Bertrand Thirion, Christophe Pallier

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments 19 pages, 8 figures, 10 pages of Appendix, 5 appendix figures

Linear pretraining in recurrent mixture density networks

Hubert Normandin-Taillon, Frédéric Godin, Chun Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Why Is Public Pretraining Necessary for Private Model Training?

Arun Ganesh, Mahdi Haghifam, Milad Nasr, Sewoong Oh, Thomas Steinke, Om Thakkar, Abhradeep Thakurta, Lun Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies

Emma Sarfati, Alexandre Bone, Marc-Michel Rohe, Pietro Gori, Isabelle Bloch

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Comments Accepted at IEEE ISBI 2023

ConEntail: An Entailment-based Framework for Universal Zero and Few Shot Classification with Supervised Contrastive Pretraining

Ranran Haoran Zhang, Aysa Xuemo Fan, Rui Zhang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by EACL 2023

Towards Understanding Why Mask-Reconstruction Pretraining Helps in Downstream Tasks

Jiachun Pan, Pan Zhou, Shuicheng Yan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Construction of English Resume Corpus and Test with Pre-trained Language Models

Chengguang Gan, Tatsunori Mori

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments The Association for Natural Language Processing in Japan. Submit in NLP2023

SPADE: Self-supervised Pretraining for Acoustic DisEntanglement

John Harvill, Jarred Barber, Arun Nair, Ramin Pishehvar

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Embedding Recycling for Language Models

Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, Doug Downey

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments EACL Findings 2023