语言大模型 / LLM - arXivDaily 专题

2505.13089 2025-05-28 cs.CL 79%

Systematic Generalization in Language Models Scales with Information Entropy

Sondre Wold, Lucas Georges Gabriel Charpentier, Étienne Simon

机构 * Language Technology Group University of Oslo（奥斯陆大学语言技术组）

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2025: Findings

2505.20052 2025-05-27 cs.LG q-bio.QM 79%

Ankh3: Multi-Task Pretraining with Sequence Denoising and Completion Enhances Protein Representations

Hazem Alsamkary, Mohamed Elshaffei, Mohamed Elkerdawy, Ahmed Elnaggar

机构 * Proteinea Inc.（Proteinea公司）

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.LG

Comments 8 pages, 0 figures

2310.13312 2025-05-27 cs.CL 79%

Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models

Jaeyoung Choe, Keonwoong Noh, Nayeon Kim, Seyun Ahn, Woohwan Jung

机构 * Department of Applied Artificial Intelligence, Hanyang University（应用人工智能系，翰阳大学）； Division of Computer Science, Hanyang University（计算机科学系，翰阳大学）； Ramply Inc.（Ramply公司）

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2023 (Findings)

2505.17631 2025-05-26 cs.IR cs.AI 79%

BehaveGPT: A Foundation Model for Large-scale User Behavior Modeling

Jiahui Gong, Jingtao Ding, Fanjin Meng, Chen Yang, Hong Chen, Zuojian Wang, Haisheng Lu, Yong Li

专题命中预训练与数据：foundation model(title)；pretraining(abstract)；分类 cs.AI

Comments 22 pages, 8 figures, 5 tables

2410.00871 2025-05-26 cs.CV cs.AI 79%

MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining

Yunze Liu, Li Yi

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Journal ref Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2025

2505.15559 2025-05-22 cs.SD cs.AI eess.AS 79%

Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes

Zixun Guo, Simon Dixon

机构 * Centre for Digital Music（数字音乐中心）； Queen Mary University of London（伦敦大学玛丽女王学院）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2505.12711 2025-05-21 cs.CV cs.AI 79%

Any-to-Any Learning in Computational Pathology via Triplet Multimodal Pretraining

Qichen Sun, Zhengrui Guo, Rui Peng, Hao Chen, Jinzhuo Wang

机构 * Peking University（北京大学）； The Hong Kong University of Science and Technology（香港科技大学）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2505.13466 2025-05-21 cs.AI 79%

AgentSGEN: Multi-Agent LLM in the Loop for Semantic Collaboration and GENeration of Synthetic Data

Vu Dinh Xuan, Hao Vo, David Murphy, Hoang D. Nguyen

机构 * University of Information Technology, VNU–HCM（越南胡志明市信息技术大学）； University College Cork（科尔克大学）

专题命中预训练与数据：LLM(title,abstract)；分类 cs.AI

2502.05505 2025-05-21 cs.LG cs.CR cs.CV stat.ML 79%

Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model

Zinan Lin, Tadas Baltrusaitis, Wenyu Wang, Sergey Yekhanin

机构 * Microsoft Research Redmond（微软研究院红mond分部）； Microsoft Cambridge（微软剑桥分部）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Published in: (1) ICLR 2025 Workshop on Data Problems, (2) ICLR 2025 Workshop on Synthetic Data

2502.08949 2025-05-20 cs.LG 79%

DICE: Device-level Integrated Circuits Encoder with Graph Contrastive Pretraining

Sungyoung Lee, Ziyi Wang, Seunggeun Kim, Taekyun Lee, Yao Lai, David Z. Pan

机构 * Electrical and Computer Engineering, The University of Texas at Austin（得克萨斯大学奥斯汀分校电子与计算机工程系）； Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）； Computer Science, The University of Hong Kong（香港大学计算机科学系）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2505.10848 2025-05-20 cs.LG 79%

Foundation model for mass spectrometry proteomics

Justin Sanders, Melih Yilmaz, Jacob H. Russell, Wout Bittremieux, William E. Fondrie, Nicholas M. Riley, Sewoong Oh, William Stafford Noble

机构 * Paul G. Allen School of Computer Science and Engineering, University of Washington（保罗·G·阿伦计算机科学与工程学院，华盛顿大学）； Department of Chemistry, University of Washington（华盛顿大学化学系）； Adrem Data Lab, University of Antwerp（阿德雷姆数据实验室，安特卫普大学）； Talus Bioscience（塔卢斯生物科学）； Department of Genome Sciences, University of Washington（华盛顿大学基因组科学系）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2305.15560 2025-05-20 cs.CV cs.CR cs.LG 79%

Differentially Private Synthetic Data via Foundation Model APIs 1: Images

Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, Sergey Yekhanin

机构 * Microsoft Research（微软研究院）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Published in ICLR 2024

2505.03844 2025-05-13 eess.IV cs.AI cs.CV 79%

From Spaceborne to Airborne: SAR Image Synthesis Using Foundation Models for Multi-Scale Adaptation

Solene Debuysere, Nicolas Trouve, Nathan Letheule, Olivier Leveque, Elise Colin

机构 * ONERA - DEMR ； Université Paris Saclay（巴黎萨克雷大学）； ONERA - DTIS

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2405.16731 2025-05-12 cs.LG cs.NE 79%

Pretraining with Random Noise for Fast and Robust Learning without Weight Transport

Jeonghwan Cheon, Sang Wan Lee, Se-Bum Paik

机构 * Department of Brain and Cognitive Sciences（脑科学与认知科学系）； Graduate School of Data Science（数据科学研究生院）； Kim Jaechul Graduate School of AI（金在哲人工智能研究生院）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Journal ref Advances in Neural Information Processing Systems 37, 13748-13768, 2024

2505.03426 2025-05-07 cs.CV cs.AI 79%

Phenotype-Guided Generative Model for High-Fidelity Cardiac MRI Synthesis: Advancing Pretraining and Clinical Applications

Ziyu Li, Yujian Hu, Zhengyao Ding, Yiheng Mao, Haitao Li, Fan Yi, Hongkun Zhang, Zhengxing Huang

机构 * Zhejiang University（浙江大学）； The First Affiliated Hospital of Zhejiang University School of Medicine（浙江大学医学院附属第一医院）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2408.06150 2025-05-06 cs.CL physics.chem-ph q-bio.BM 79%

LipidBERT: A Lipid Language Model Pre-trained on METiS de novo Lipid Library

Tianhao Yu, Cai Yao, Zhuorui Sun, Feng Shi, Lin Zhang, Kangjie Lyu, Xuan Bai, Andong Liu, Xicheng Zhang, Jiali Zou, Wenshou Wang, Chris Lai, Kai Wang

机构 * METiS Pharmaceuticals（METiS制药公司）

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2504.17315 2025-04-25 cs.CV cs.AI 79%

DIMT25@ICDAR2025: HW-TSC's End-to-End Document Image Machine Translation System Leveraging Large Vision-Language Model

Zhanglin Wu, Tengfei Song, Ning Xie, Weidong Zhang, Pengfei Li, Shuang Wu, Chong Li, Junhao Zhu, Hao Yang

机构 * Huawei Translation Service Center（华为翻译服务中心）

专题命中预训练与数据：language model(title,abstract)；分类 cs.AI

Comments 7 pages, 1 figures, 2 tables

2412.02856 2025-04-23 cs.CV cs.LG 79%

Is Large-Scale Pretraining the Secret to Good Domain Generalization?

Piotr Teterwak, Kuniaki Saito, Theodoros Tsiligkaridis, Bryan A. Plummer, Kate Saenko

机构 * Boston University（波士顿大学）； MIT Lincoln Laboratory（麻省理工学院林肯实验室）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted at ICLR 2025

2504.15432 2025-04-23 cs.CL 79%

Feeding LLM Annotations to BERT Classifiers at Your Own Risk

Yucheng Lu, Kazimier Smith

机构 * New York University（纽约大学）

专题命中预训练与数据：LLM(title,abstract)；分类 cs.CL

2401.10375 2025-04-22 cs.CR cs.DC cs.LG 79%

Foundation Models in Federated Learning: Assessing Backdoor Vulnerabilities

Xi Li, Chen Wu, Jiaqi Wang

机构 * University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）； Meta ； The Pennsylvania State University（宾夕法尼亚州立大学）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Xi Li and Chen Wu are equal contribution. The corresponding author is Jiaqi Wang. This paper has been accepted by IJCNN 2025

2408.11054 2025-04-18 cs.CV cs.AI 79%

Near, far: Patch-ordering enhances vision foundation models' scene understanding

Valentinos Pariza, Mohammadreza Salehi, Gertjan Burghouts, Francesco Locatello, Yuki M. Asano

专题命中预训练与数据：foundation model(title)；pretraining(abstract)；分类 cs.AI

Comments Accepted at ICLR25. The webpage is accessible at: https://vpariza.github.io/NeCo/

2504.11054 2025-04-16 cs.LG 79%

Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models

Andrea Tirinzoni, Ahmed Touati, Jesse Farebrother, Mateusz Guzek, Anssi Kanervisto, Yingchen Xu, Alessandro Lazaric, Matteo Pirotta

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Published at ICLR 2025

2504.02116 2025-04-04 cs.CL 79%

Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive ziji

Xiulin Yang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Journal ref COLING 2025

2503.24326 2025-04-02 cs.CV cs.LG 79%

Self-Supervised Pretraining for Aerial Road Extraction

Rupert Polley, Sai Vignesh Abishek Deenadayalan, J. Marius Zöllner

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted at 36th IEEE Intelligent Vehicles Symposium (IV) 2025 Joint Workshop on Safety, Metrics and Benchmarks for Autonomous Driving

2503.21788 2025-04-02 q-bio.BM cs.LG 79%

PharMolixFM: All-Atom Foundation Models for Molecular Modeling and Generation

Yizhen Luo, Jiashuo Wang, Siqi Fan, Zaiqing Nie

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2405.04912 2025-04-02 q-bio.BM cs.LG physics.chem-ph 79%

GP-MoLFormer: A Foundation Model For Molecular Generation

Jerret Ross, Brian Belgodere, Samuel C. Hoffman, Vijil Chenthamarakshan, Jiri Navratil, Youssef Mroueh, Payel Das

专题命中预训练与数据：foundation model(title)；language model(abstract)；分类 cs.LG

2410.13567 2025-04-01 cs.CV cs.AI 79%

CCUP: A Controllable Synthetic Data Generation Pipeline for Pretraining Cloth-Changing Person Re-Identification Models

Yujian Zhao, Chengru Wu, Yinong Xu, Xuanzheng Du, Ruiyu Li, Guanglin Niu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by ICME 2025

2402.01929 2025-04-01 cs.LG stat.ML 79%

Sample, estimate, aggregate: A recipe for causal discovery foundation models

Menghua Wu, Yujia Bao, Regina Barzilay, Tommi Jaakkola

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Our code is available at https://github.com/rmwu/sea

Journal ref Transactions on Machine Learning Research (03/2025)

2404.07943 2025-03-27 cs.CE cs.LG 79%

A Pretraining-Finetuning Computational Framework for Material Homogenization

Yizheng Wang, Xiang Li, Ziming Yan, Shuaifeng Ma, Jinshuai Bai, Bokai Liu, Timon Rabczuk, Yinghua Liu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2503.18117 2025-03-25 cs.CL 79%

Detection of Somali-written Fake News and Toxic Messages on the Social Media Using Transformer-based Language Models

Muhidin A. Mohamed, Shuab D. Ahmed, Yahye A. Isse, Hanad M. Mohamed, Fuad M. Hassan, Houssein A. Assowe

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

1. 预训练与数据 12429 篇

Systematic Generalization in Language Models Scales with Information Entropy

Ankh3: Multi-Task Pretraining with Sequence Denoising and Completion Enhances Protein Representations

Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models

BehaveGPT: A Foundation Model for Large-scale User Behavior Modeling

MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining

Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes

Any-to-Any Learning in Computational Pathology via Triplet Multimodal Pretraining

AgentSGEN: Multi-Agent LLM in the Loop for Semantic Collaboration and GENeration of Synthetic Data

Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model

DICE: Device-level Integrated Circuits Encoder with Graph Contrastive Pretraining

Foundation model for mass spectrometry proteomics

Differentially Private Synthetic Data via Foundation Model APIs 1: Images

From Spaceborne to Airborne: SAR Image Synthesis Using Foundation Models for Multi-Scale Adaptation

Pretraining with Random Noise for Fast and Robust Learning without Weight Transport

Phenotype-Guided Generative Model for High-Fidelity Cardiac MRI Synthesis: Advancing Pretraining and Clinical Applications

LipidBERT: A Lipid Language Model Pre-trained on METiS de novo Lipid Library

DIMT25@ICDAR2025: HW-TSC's End-to-End Document Image Machine Translation System Leveraging Large Vision-Language Model

Is Large-Scale Pretraining the Secret to Good Domain Generalization?

Feeding LLM Annotations to BERT Classifiers at Your Own Risk

Foundation Models in Federated Learning: Assessing Backdoor Vulnerabilities

Near, far: Patch-ordering enhances vision foundation models' scene understanding

Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models

Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive ziji

Self-Supervised Pretraining for Aerial Road Extraction

PharMolixFM: All-Atom Foundation Models for Molecular Modeling and Generation

GP-MoLFormer: A Foundation Model For Molecular Generation

CCUP: A Controllable Synthetic Data Generation Pipeline for Pretraining Cloth-Changing Person Re-Identification Models

Sample, estimate, aggregate: A recipe for causal discovery foundation models

A Pretraining-Finetuning Computational Framework for Material Homogenization

Detection of Somali-written Fake News and Toxic Messages on the Social Media Using Transformer-based Language Models