语言大模型 / LLM

MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilization

Yu Zhang, Qi Zhang, Zixuan Gong, Yiwei Shi, Yepeng Liu, Duoqian Miao, Yang Liu, Ke Liu, Kun Yi, Wei Fan, Liang Hu, Changwei Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Comments ICML 2024

Position: Graph Foundation Models are Already Here

Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma, Tong Zhao, Neil Shah, Mikhail Galkin, Jiliang Tang

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

Comments 23 pages, 2 figures

Evaluating Vision-Language Models on Bistable Images

Artemis Panagopoulou, Coby Melkin, Chris Callison-Burch

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI

FaultFormer: Pretraining Transformers for Adaptable Bearing Fault Classification

Anthony Zhou, Amir Barati Farimani

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 10 pages, 6 figures

Journal ref in IEEE Access, vol. 12, pp. 70719-70728, 2024

A Study on Self-Supervised Pretraining for Vision Problems in Gastrointestinal Endoscopy

Edward Sanderson, Bogdan J. Matuszewski

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Analysis of Atom-level pretraining with Quantum Mechanics (QM) data for Graph Neural Networks Molecular property models

Jose Arjona-Medina, Ramil Nugmanov

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 6 pages + 11 Supplement Materials

ReactXT: Understanding Molecular "Reaction-ship" via Reaction-Contextualized Molecule-Text Pretraining

Zhiyuan Liu, Yaorui Shi, An Zhang, Sihang Li, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, Tat-Seng Chua

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments ACL 2024 Findings, 9 pages

PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology

George Shaikovski, Adam Casson, Kristen Severson, Eric Zimmermann, Yi Kan Wang, Jeremy D. Kunz, Juan A. Retamero, Gerard Oakley, David Klimstra, Christopher Kanan, Matthew Hanna, Michal Zelechowski, Julian Viret, Neil Tenenholtz, James Hall, Nicolo Fusi, Razik Yousfi, Peter Hamilton, William A. Moye, Eugene Vorontsov, Siqi Liu, Thomas J. Fuchs

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models

Yihong Liu, Chunlan Ma, Haotian Ye, Hinrich Schütze

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments ACL 2024

Incubating Text Classifiers Following User Instruction with Nothing but LLM

Letian Peng, Jingbo Shang

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL

CLIP with Quality Captions: A Strong Pretraining for Vision Tasks

Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Oncel Tuzel

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Investigating the 'Autoencoder Behavior' in Speech Self-Supervised Models: a focus on HuBERT's Pretraining

Valentin Vielzeuf

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts

Rabindra Lamsal, Maria Rodriguez Read, Shanika Karunasekera

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Journal ref Knowledge-Based Systems, 111916 (2024)

SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals

Cheng Ding, Zhicheng Guo, Zhaoliang Chen, Randall J Lee, Cynthia Rudin, Xiao Hu

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

PRODIS -- a speech database and a phoneme-based language model for the study of predictability effects in Polish

Zofia Malisz, Jan Foremski, Małgorzata Kul

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments To appear in the proceedings of LREC2024: Language Resources and Evaluation Conference 2024, Turin, Italy

The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text

Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, Chloé Clavel

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to NAACL 2024 Findings

On Retrieval Augmentation and the Limitations of Language Model Training

Ting-Rui Chiang, Xinyan Velocity Yu, Joshua Robinson, Ollie Liu, Isabelle Lee, Dani Yogatama

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to NAACL 2024

SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining

Chull Hwan Song, Taebaek Hwang, Jooyoung Yoon, Shunghyun Choi, Yeong Hyeon Gu

专题命中 预训练与数据 ：pretraining(title)；language model(abstract)；分类 cs.AI

Comments CVPR2024 Accepted

DailyMAE: Towards Pretraining Masked Autoencoders in One Day

Jiantao Wu, Shentong Mo, Sara Atito, Zhenhua Feng, Josef Kittler, Muhammad Awais

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Ethos: Rectifying Language Models in Orthogonal Parameter Space

Lei Gao, Yue Niu, Tingting Tang, Salman Avestimehr, Murali Annavaram

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Negative Label Guided OOD Detection with Pretrained Vision-Language Models

Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, Bo Han

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.LG

Comments ICLR 2024 Spotlight

Noise-Robust Keyword Spotting through Self-supervised Pretraining

Jacob Mørk, Holger Severin Bovbjerg, Gergely Kiss, Zheng-Hua Tan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?

Shaoxiong Ji, Timothee Mickus, Vincent Segonne, Jörg Tiedemann

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments LREC-COLING 2024

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?

Jingfeng Wu, Difan Zou, Zixiang Chen, Vladimir Braverman, Quanquan Gu, Peter L. Bartlett

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments ICLR 2024 Camera Ready

Do Language Models Care About Text Quality? Evaluating Web-Crawled Corpora Across 11 Languages

Rik van Noord, Taja Kuzman, Peter Rupnik, Nikola Ljubešić, Miquel Esplà-Gomis, Gema Ramírez-Sánchez, Antonio Toral

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to LREC-COLING 2024 (long)

Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers

Linyuan Gong, Chenyan Xiong, Xiaodong Liu, Payal Bajaj, Yiqing Xie, Alvin Cheung, Jianfeng Gao, Xia Song

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Published as a conference paper at ACL 2023. 9 pages

ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment

Bowen Gao, Yinjun Jia, Yuanle Mo, Yuyan Ni, Weiying Ma, Zhiming Ma, Yanyan Lan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Bridging the Gap between Chemical Reaction Pretraining and Conditional Molecule Generation with a Unified Model

Bo Qiang, Yiran Zhou, Yuheng Ding, Ningfeng Liu, Song Song, Liangren Zhang, Bo Huang, Zhenming Liu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining

Jiarun Liu, Hao Yang, Hong-Yu Zhou, Yan Xi, Lequan Yu, Yizhou Yu, Yong Liang, Guangming Shi, Shaoting Zhang, Hairong Zheng, Shanshan Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments Code and models of Swin-UMamba are publicly available at: https://github.com/JiarunLiu/Swin-UMamba

ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models

Jonas Belouadi, Steffen Eger

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted at ACL 2023 (main track)