语言大模型 / LLM - arXivDaily 专题

2411.16618 2024-11-26 cs.CL 79%

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training

Kaustubh Ponkshe, Venkatapathy Subramanian, Natwar Modani, Ganesh Ramakrishnan

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2411.16339 2024-11-26 astro-ph.SR astro-ph.IM cs.LG physics.space-ph 79%

Solaris: A Foundation Model of the Sun

Harris Abdul Majid, Pietro Sittoni, Francesco Tudisco

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2411.16096 2024-11-26 cs.CV cs.AI cs.MM 79%

ENCLIP: Ensembling and Clustering-Based Contrastive Language-Image Pretraining for Fashion Multimodal Search with Limited Data and Low-Quality Images

Prithviraj Purushottam Naik, Rohit Agarwal

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2411.11458 2024-11-25 eess.IV cs.AI cs.CV 79%

HistoEncoder: a digital pathology foundation model for prostate cancer

Joona Pohjonen, Abderrahim-Oussama Batouche, Antti Rannikko, Kevin Sandeman, Andrew Erickson, Esa Pitkanen, Tuomas Mirtti

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2411.08868 2024-11-14 cs.CL 79%

CamemBERT 2.0: A Smarter French Language Model Aged to Perfection

Wissam Antoun, Francis Kulumba, Rian Touchent, Éric de la Clergerie, Benoît Sagot, Djamé Seddah

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2411.06764 2024-11-12 cs.CV cs.LG 79%

Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning

Hongsheng Zhang, Zhong Ji, Jingren Liu, Yanwei Pang, Jungong Han

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

2406.05839 2024-11-12 eess.AS cs.AI 79%

MaLa-ASR: Multimedia-Assisted LLM-Based ASR

Guanrou Yang, Ziyang Ma, Fan Yu, Zhifu Gao, Shiliang Zhang, Xie Chen

专题命中预训练与数据：LLM(title,abstract)；分类 cs.AI

2403.17860 2024-11-05 cs.CL 79%

Illuminating Blind Spots of Language Models with Targeted Agent-in-the-Loop Synthetic Data

Philip Lippmann, Matthijs T. J. Spaan, Jie Yang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Preprint. Under review

2410.18371 2024-11-05 cs.SD cs.AI eess.AS 79%

Gibberish is All You Need for Membership Inference Detection in Contrastive Language-Audio Pretraining

Ruoxi Cheng, Yizhong Ding, Shuirong Cao, Shitong Shao, Zhiqiang Wang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2403.12553 2024-11-04 cs.LG 79%

Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs

Md Ashiqur Rahman, Robert Joseph George, Mogab Elleithy, Daniel Leibovici, Zongyi Li, Boris Bonev, Colin White, Julius Berner, Raymond A. Yeh, Jean Kossaifi, Kamyar Azizzadenesheli, Anima Anandkumar

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2410.22906 2024-10-31 cs.CL 79%

From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes

Zébulon Goriely, Richard Diehl Martinez, Andrew Caines, Lisa Beinborn, Paula Buttery

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2402.02103 2024-10-30 cs.CV cs.LG 79%

Déjà Vu Memorization in Vision-Language Models

Bargav Jayaraman, Chuan Guo, Kamalika Chaudhuri

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

2410.11627 2024-10-23 cs.CL 79%

Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5

Thao Anh Dang, Limor Raviv, Lukas Galke

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 8 pages

2402.05755 2024-10-22 cs.CL cs.SD eess.AS 79%

Spirit LM: Interleaved Spoken and Written Language Model

Tu Anh Nguyen, Benjamin Muller, Bokai Yu, Marta R. Costa-jussa, Maha Elbayad, Sravya Popuri, Christophe Ropers, Paul-Ambroise Duquenne, Robin Algayres, Ruslan Mavlyutov, Itai Gat, Mary Williamson, Gabriel Synnaeve, Juan Pino, Benoit Sagot, Emmanuel Dupoux

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2410.13516 2024-10-18 cs.LG 79%

PORTAL: Scalable Tabular Foundation Models via Content-Specific Tokenization

Marco Spinaci, Marek Polewczyk, Johannes Hoffart, Markus C. Kohler, Sam Thelin, Tassilo Klein

专题命中预训练与数据：foundation model(title)；pretraining(abstract)；分类 cs.LG

Comments Accepted at Table Representation Learning Workshop at NeurIPS 2024

2410.13259 2024-10-18 cs.CL 79%

From Babbling to Fluency: Evaluating the Evolution of Language Models in Terms of Human Language Acquisition

Qiyuan Yang, Pengda Wang, Luke D. Plonsky, Frederick L. Oswald, Hanjie Chen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2306.01181 2024-10-18 cs.LG cs.CR 79%

TMI! Finetuned Models Leak Private Information from their Pretraining Data

John Abascal, Stanley Wu, Alina Oprea, Jonathan Ullman

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2402.18567 2024-10-17 cs.LG q-bio.BM 79%

Diffusion Language Models Are Versatile Protein Learners

Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, Quanquan Gu

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments ICML 2024 camera-ready version

2410.10006 2024-10-15 cs.LG 79%

TapWeight: Reweighting Pretraining Objectives for Task-Adaptive Pretraining

Ruiyi Zhang, Sai Ashish Somayajula, Pengtao Xie

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2410.09978 2024-10-15 cs.CL cs.CY 79%

When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries

Supriti Vijay, Aman Priyanshu, Ashique R. KhudaBukhsh

专题命中预训练与数据：LLM(title,abstract)；分类 cs.CL

Comments 12 pages, 3 figures, 4 tables

2410.09199 2024-10-15 cs.LG 79%

An Efficient Contrastive Unimodal Pretraining Method for EHR Time Series Data

Ryan King, Shivesh Kodali, Conrad Krueger, Tianbao Yang, Bobak J. Mortazavi

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2403.09199 2024-10-14 cs.CV cs.AI 79%

Task-Specific Adaptation of Segmentation Foundation Model via Prompt Learning

Hyung-Il Kim, Kimin Yun, Jun-Seok Yun, Yuseok Bae

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

Comments Workshop on OOD Generalization in Computer Vision, ECCV 2024

2410.07124 2024-10-10 cs.CV cs.AI 79%

Cross-Task Pretraining for Cross-Organ Cross-Scanner Adenocarcinoma Segmentation

Adrian Galdran

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments MICCAI2024 COSAS Challenge - short abstract

2401.10440 2024-10-10 cs.CL 79%

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models

Terra Blevins, Tomasz Limisiewicz, Suchin Gururangan, Margaret Li, Hila Gonen, Noah A. Smith, Luke Zettlemoyer

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2024

2409.18082 2024-10-08 cs.RO cs.AI cs.CV 79%

SKT: Integrating State-Aware Keypoint Trajectories with Vision-Language Models for Robotic Garment Manipulation

Xin Li, Siyuan Huang, Qiaojun Yu, Zhengkai Jiang, Ce Hao, Yimeng Zhu, Hongsheng Li, Peng Gao, Cewu Lu

专题命中预训练与数据：language model(title,abstract)；分类 cs.AI

2210.11815 2024-10-08 cs.CV cs.LG 79%

Self-Supervised Pretraining on Satellite Imagery: a Case Study on Label-Efficient Vehicle Detection

Jules BOURCIER, Thomas Floquet, Gohar Dashyan, Tugdual Ceillier, Karteek Alahari, Jocelyn Chanussot

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Journal ref Conference on Artificial Intelligence for Defense (CAID) 2022, DGA Maîtrise de l'Information, Nov 2022, Rennes, France

2410.03194 2024-10-07 cs.CL 79%

Parallel Corpus Augmentation using Masked Language Models

Vibhuti Kumari, Narayana Murthy Kavi

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 21 Pages, 3 Figures. arXiv admin note: text overlap with arXiv:2011.01536 by other authors

2410.03001 2024-10-07 cs.CL 79%

Can Transformers Learn $n$-gram Language Models?

Anej Svete, Nadav Borenstein, Mike Zhou, Isabelle Augenstein, Ryan Cotterell

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2406.08473 2024-10-03 cs.LG 79%

Strategies for Pretraining Neural Operators

Anthony Zhou, Cooper Lorsung, AmirPouya Hemmasian, Amir Barati Farimani

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 29 pages, 5 figures

Journal ref Transactions on Machine Learning Research, 2024

2409.19735 2024-10-01 cs.CL 79%

Scrambled text: training Language Models to correct OCR errors using synthetic data

Jonathan Bourne

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 21 pages, 6300 words, 6 Figures, 5 tables

1. 预训练与数据 12429 篇

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training

Solaris: A Foundation Model of the Sun

ENCLIP: Ensembling and Clustering-Based Contrastive Language-Image Pretraining for Fashion Multimodal Search with Limited Data and Low-Quality Images

HistoEncoder: a digital pathology foundation model for prostate cancer

CamemBERT 2.0: A Smarter French Language Model Aged to Perfection

Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning

MaLa-ASR: Multimedia-Assisted LLM-Based ASR

Illuminating Blind Spots of Language Models with Targeted Agent-in-the-Loop Synthetic Data

Gibberish is All You Need for Membership Inference Detection in Contrastive Language-Audio Pretraining

Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs

From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes

Déjà Vu Memorization in Vision-Language Models

Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5

Spirit LM: Interleaved Spoken and Written Language Model

PORTAL: Scalable Tabular Foundation Models via Content-Specific Tokenization

From Babbling to Fluency: Evaluating the Evolution of Language Models in Terms of Human Language Acquisition

TMI! Finetuned Models Leak Private Information from their Pretraining Data

Diffusion Language Models Are Versatile Protein Learners

TapWeight: Reweighting Pretraining Objectives for Task-Adaptive Pretraining

When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries

An Efficient Contrastive Unimodal Pretraining Method for EHR Time Series Data

Task-Specific Adaptation of Segmentation Foundation Model via Prompt Learning

Cross-Task Pretraining for Cross-Organ Cross-Scanner Adenocarcinoma Segmentation

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models

SKT: Integrating State-Aware Keypoint Trajectories with Vision-Language Models for Robotic Garment Manipulation

Self-Supervised Pretraining on Satellite Imagery: a Case Study on Label-Efficient Vehicle Detection

Parallel Corpus Augmentation using Masked Language Models

Can Transformers Learn $n$-gram Language Models?

Strategies for Pretraining Neural Operators

Scrambled text: training Language Models to correct OCR errors using synthetic data