语言大模型 / LLM - arXivDaily 专题

2404.17667 2024-04-30 eess.SP cs.LG 79%

SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals

Cheng Ding, Zhicheng Guo, Zhaoliang Chen, Randall J Lee, Cynthia Rudin, Xiao Hu

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2404.10112 2024-04-19 cs.CL cs.SD eess.AS 79%

PRODIS -- a speech database and a phoneme-based language model for the study of predictability effects in Polish

Zofia Malisz, Jan Foremski, Małgorzata Kul

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in the proceedings of LREC2024: Language Resources and Evaluation Conference 2024, Turin, Italy

2311.09807 2024-04-17 cs.CL 79%

The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text

Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, Chloé Clavel

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to NAACL 2024 Findings

2311.09615 2024-04-03 cs.CL 79%

On Retrieval Augmentation and the Limitations of Language Model Training

Ting-Rui Chiang, Xinyan Velocity Yu, Joshua Robinson, Ollie Liu, Isabelle Lee, Dani Yogatama

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to NAACL 2024

2404.01156 2024-04-02 cs.CV cs.AI 79%

SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining

Chull Hwan Song, Taebaek Hwang, Jooyoung Yoon, Shunghyun Choi, Yeong Hyeon Gu

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.AI

Comments CVPR2024 Accepted

2404.00509 2024-04-02 cs.LG cs.CV 79%

DailyMAE: Towards Pretraining Masked Autoencoders in One Day

Jiantao Wu, Shentong Mo, Sara Atito, Zhenhua Feng, Josef Kittler, Muhammad Awais

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2403.08994 2024-04-02 cs.CL 79%

Ethos: Rectifying Language Models in Orthogonal Parameter Space

Lei Gao, Yue Niu, Tingting Tang, Salman Avestimehr, Murali Annavaram

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2403.20078 2024-04-01 cs.CV cs.LG 79%

Negative Label Guided OOD Detection with Pretrained Vision-Language Models

Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, Bo Han

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments ICLR 2024 Spotlight

2403.18560 2024-03-28 eess.AS cs.LG cs.SD 79%

Noise-Robust Keyword Spotting through Self-supervised Pretraining

Jacob Mørk, Holger Severin Bovbjerg, Gergely Kiss, Zheng-Hua Tan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2403.16777 2024-03-26 cs.CL 79%

Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?

Shaoxiong Ji, Timothee Mickus, Vincent Segonne, Jörg Tiedemann

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments LREC-COLING 2024

2310.08391 2024-03-18 stat.ML cs.LG 79%

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?

Jingfeng Wu, Difan Zou, Zixiang Chen, Vladimir Braverman, Quanquan Gu, Peter L. Bartlett

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ICLR 2024 Camera Ready

2403.08693 2024-03-14 cs.CL 79%

Do Language Models Care About Text Quality? Evaluating Web-Crawled Corpora Across 11 Languages

Rik van Noord, Taja Kuzman, Peter Rupnik, Nikola Ljubešić, Miquel Esplà-Gomis, Gema Ramírez-Sánchez, Antonio Toral

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to LREC-COLING 2024 (long)

2305.12567 2024-03-11 cs.CL 79%

Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers

Linyuan Gong, Chenyan Xiong, Xiaodong Liu, Payal Bajaj, Yiqing Xie, Alvin Cheung, Jianfeng Gao, Xia Song

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Published as a conference paper at ACL 2023. 9 pages

2310.07229 2024-03-08 cs.LG 79%

ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment

Bowen Gao, Yinjun Jia, Yuanle Mo, Yuyan Ni, Weiying Ma, Zhiming Ma, Yanyan Lan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2303.06965 2024-03-08 cs.LG q-bio.BM 79%

Bridging the Gap between Chemical Reaction Pretraining and Conditional Molecule Generation with a Unified Model

Bo Qiang, Yiran Zhou, Yuheng Ding, Ningfeng Liu, Song Song, Liangren Zhang, Bo Huang, Zhenming Liu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2402.03302 2024-03-07 eess.IV cs.CV cs.LG 79%

Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining

Jiarun Liu, Hao Yang, Hong-Yu Zhou, Yan Xi, Lequan Yu, Yizhou Yu, Yong Liang, Guangming Shi, Shaoting Zhang, Hairong Zheng, Shanshan Wang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Code and models of Swin-UMamba are publicly available at: https://github.com/JiarunLiu/Swin-UMamba

2212.10474 2024-03-05 cs.CL 79%

ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models

Jonas Belouadi, Steffen Eger

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at ACL 2023 (main track)

2311.14534 2024-02-29 cs.LG 79%

Finding Foundation Models for Time Series Classification with a PreText Task

Ali Ismail-Fawaz, Maxime Devanne, Stefano Berretti, Jonathan Weber, Germain Forestier

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2402.15404 2024-02-26 cs.LG 79%

United We Pretrain, Divided We Fail! Representation Learning for Time Series by Pretraining on 75 Datasets at Once

Maurice Kraus, Felix Divo, David Steinmann, Devendra Singh Dhami, Kristian Kersting

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2402.12500 2024-02-21 cs.CV cs.LG eess.IV 79%

Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification

Sebastian Doerrich, Tobias Archut, Francesco Di Salvo, Christian Ledig

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Accepted at 21st IEEE International Symposium on Biomedical Imaging (IEEE ISBI 2024)

2402.10430 2024-02-19 cs.CL 79%

Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models

Dheeraj Mekala, Alex Nguyen, Jingbo Shang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2203.08774 2024-02-09 cs.CL 79%

CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signals

Scott Novotney, Sreeparna Mukherjee, Zeeshan Ahmed, Andreas Stolcke

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in Findings of ACL 2022

Journal ref Findings of ACL 2022, pp. 3368-3379

2307.04308 2024-02-02 cs.LG 79%

Towards Cross-Table Masked Pretraining for Web Data Mining

Chao Ye, Guoshan Lu, Haobo Wang, Liyao Li, Sai Wu, Gang Chen, Junbo Zhao

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to WWW 2024

2401.13536 2024-01-26 hep-ex cs.LG hep-ph physics.data-an 79%

Finetuning Foundation Models for Joint Analysis Optimization

Matthias Vigl, Nicole Hartman, Lukas Heinrich

专题命中预训练与数据：foundation model(title)；pretraining(abstract)；分类 cs.LG

Comments 13 pages, 12 figures

2312.16613 2024-01-24 cs.SD cs.LG eess.AS 79%

Self-supervised Pretraining for Robust Personalized Voice Activity Detection in Adverse Conditions

Holger Severin Bovbjerg, Jesper Jensen, Jan Østergaard, Zheng-Hua Tan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2310.01852 2024-01-23 cs.CV cs.AI 79%

LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, HongFa Wang, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zongwei Li, Wancai Zhang, Zhifeng Li, Wei Liu, Li Yuan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by ICLR 2024

2401.07990 2024-01-17 eess.IV cs.CV cs.LG 79%

How does self-supervised pretraining improve robustness against noisy labels across various medical image classification datasets?

Bidur Khanal, Binod Bhattarai, Bishesh Khanal, Cristian Linte

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2306.01684 2024-01-12 cs.LG cs.CR 79%

Harnessing large-language models to generate private synthetic text

Alexey Kurakin, Natalia Ponomareva, Umar Syed, Liam MacDermed, Andreas Terzis

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments 31 pages; 7 figures; compared to previous version added result of LoRa-finetuning

2309.09431 2024-01-05 cs.CV cs.AI 79%

FactoFormer: Factorized Hyperspectral Transformers with Self-Supervised Pretraining

Shaheer Mohamed, Maryam Haghighat, Tharindu Fernando, Sridha Sridharan, Clinton Fookes, Peyman Moghadam

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted to IEEE Transactions on Geoscience and Remote Sensing in December 2023

2312.15273 2023-12-27 cs.CV cs.AI 79%

Benefit from public unlabeled data: A Frangi filtering-based pretraining network for 3D cerebrovascular segmentation

Gen Shi, Hao Lu, Hui Hui, Jie Tian

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Under Review

1. 预训练与数据 12429 篇

SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals

PRODIS -- a speech database and a phoneme-based language model for the study of predictability effects in Polish

The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text

On Retrieval Augmentation and the Limitations of Language Model Training

SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining

DailyMAE: Towards Pretraining Masked Autoencoders in One Day

Ethos: Rectifying Language Models in Orthogonal Parameter Space

Negative Label Guided OOD Detection with Pretrained Vision-Language Models

Noise-Robust Keyword Spotting through Self-supervised Pretraining

Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?

Do Language Models Care About Text Quality? Evaluating Web-Crawled Corpora Across 11 Languages

Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers

ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment

Bridging the Gap between Chemical Reaction Pretraining and Conditional Molecule Generation with a Unified Model

Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining

ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models

Finding Foundation Models for Time Series Classification with a PreText Task

United We Pretrain, Divided We Fail! Representation Learning for Time Series by Pretraining on 75 Datasets at Once

Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification

Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models

CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signals

Towards Cross-Table Masked Pretraining for Web Data Mining

Finetuning Foundation Models for Joint Analysis Optimization

Self-supervised Pretraining for Robust Personalized Voice Activity Detection in Adverse Conditions

LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

How does self-supervised pretraining improve robustness against noisy labels across various medical image classification datasets?

Harnessing large-language models to generate private synthetic text

FactoFormer: Factorized Hyperspectral Transformers with Self-Supervised Pretraining

Benefit from public unlabeled data: A Frangi filtering-based pretraining network for 3D cerebrovascular segmentation