语言大模型 / LLM - arXivDaily 专题

2506.14474 2025-10-07 cs.CL cs.CR 87%

LexiMark: Robust Watermarking via Lexical Substitutions to Enhance Membership Verification of an LLM's Textual Training Data

Eyal German, Sagiv Antebi, Edan Habler, Asaf Shabtai, Yuval Elovici

机构 * Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev（软件与信息系统工程系，贝内尔根大学）

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

2509.25050 2025-09-30 cs.LG 87%

Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models

Shuchen Xue, Chongjian Ge, Shilong Zhang, Yichen Li, Zhi-Ming Ma

机构 * UCAS（中国科学院大学）； Adobe Research（Adobe研究）； HKU（香港大学）； MIT（麻省理工学院）

专题命中预训练与数据：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

2503.13794 2025-09-22 cs.CV cs.AI 87%

LED: LLM Enhanced Open-Vocabulary Object Detection without Human Curated Data Generation

Yang Zhou, Shiyu Zhao, Yuxiao Chen, Zhenting Wang, Can Jin, Dimitris N. Metaxas

机构 * Rutgers University（新泽西罗格斯大学）

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

2504.16980 2025-09-16 cs.LG 87%

Safety Pretraining: Toward the Next Generation of Safe AI

Pratyush Maini, Sachin Goyal, Dylan Sam, Alex Robey, Yash Savani, Yiding Jiang, Andy Zou, Matt Fredrikson, Zacharcy C. Lipton, J. Zico Kolter

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2509.01649 2025-09-03 cs.LG 87%

Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling

Sachin Goyal, David Lopez-Paz, Kartik Ahuja

机构 * FAIR at Meta（Meta旗下的FAIR）； Carnegie Mellon University（卡内基梅隆大学）

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2412.14373 2025-07-31 cs.CL eess.SP 87%

ECG-Byte: A Tokenizer for End-to-End Generative Electrocardiogram Language Modeling

William Han, Chaojing Duan, Michael A. Rosenberg, Emerson Liu, Ding Zhao

机构 * Carnegie Mellon University（卡内基梅隆大学）； Allegheny Health Network（阿勒格尼健康网络）； University of Colorado（科罗拉多大学）

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Comments 38 pages, 9 figures; Accepted to MLHC 2025

2506.10395 2025-07-15 cs.CV cs.AI 87%

Pisces: An Auto-regressive Foundation Model for Image Understanding and Generation

Zhiyang Xu, Jiuhai Chen, Zhaojiang Lin, Xichen Pan, Lifu Huang, Tianyi Zhou, Madian Khabsa, Qifan Wang, Di Jin, Michihiro Yasunaga, Lili Yu, Xi Victoria Lin, Shaoliang Nie

机构 * Meta ； University of Maryland（马里兰大学）； New York University（纽约大学）

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Unified image understanding and generation model

2502.05551 2025-06-03 cs.CL 87%

FRAME: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy

Xuemiao Zhang, Feiyu Duan, Liangyu Xu, Yongwei Zhou, Sirui Wang, Rongxiang Weng, Jingang Wang, Xunliang Cai

机构 * Peking University（北京大学）； Beihang University（北航）； Tsinghua University（清华大学）； Meituan（美团）

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2310.08232 2025-05-23 cs.CL 87%

Language Models are Universal Embedders

Xin Zhang, Zehan Li, Yanzhao Zhang, Dingkun Long, Pengjun Xie, Meishan Zhang, Min Zhang

机构 * Harbin Institute of Technology（哈尔滨工业大学）； The Hong Kong Polytechnic University（香港理工大学）

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Comments XLLM Workshop, ACL 2025

2502.03251 2025-04-09 cs.LG 87%

RiemannGFM: Learning a Graph Foundation Model from Riemannian Geometry

Li Sun, Zhenhao Huang, Suyang Zhou, Qiqi Wan, Hao Peng, Philip Yu

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Accepted by WWW 2025 (Oral)

2503.11411 2025-03-17 cs.LG 87%

Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models

Xu Liu, Taha Aksu, Juncheng Liu, Qingsong Wen, Yuxuan Liang, Caiming Xiong, Silvio Savarese, Doyen Sahoo, Junnan Li, Chenghao Liu

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

2410.00263 2025-03-14 cs.CV cs.AI 87%

Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation

Kun Yuan, Vinkle Srivastav, Nassir Navab, Nicolas Padoy

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted at the 38th Conference on Neural Information Processing Systems (NeurIPS 2024 Spolight)

2502.11275 2025-02-18 cs.CL 87%

Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM's Nest

Letian Peng, Zilong Wang, Feng Yao, Jingbo Shang

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

2501.07288 2025-02-04 cs.AI 87%

LLM-Net: Democratizing LLMs-as-a-Service through Blockchain-based Expert Networks

Zan-Kai Chong, Hiroyuki Ohsaki, Bryan Ng

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 8 pages

2501.07721 2025-01-15 cs.CL 87%

LLMic: Romanian Foundation Language Model

Vlad-Andrei Bădoiu, Mihai-Valentin Dumitru, Alexandru M. Gherghescu, Alexandru Agache, Costin Raiciu

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；foundation model(abstract)；pretraining(abstract)

2411.17672 2024-11-27 cs.LG 87%

Synthetic Data Generation with LLM for Improved Depression Prediction

Andrea Kang, Jun Yu Chen, Zoe Lee-Youngzie, Shuhao Fu

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 6 pages excluding references and appendix

2409.19854 2024-10-01 cs.CL econ.GN q-fin.CP q-fin.EC 87%

The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Merging

Masanori Hirano, Kentaro Imajo

专题命中预训练与数据：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments 9 pages

2409.14988 2024-09-24 cs.CL 87%

Beyond Fine-tuning: Unleashing the Potential of Continuous Pretraining for Clinical LLMs

Clément Christophe, Tathagata Raha, Svetlana Maslenkova, Muhammad Umar Salman, Praveen K Kanithi, Marco AF Pimentel, Shadab Khan

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2402.09834 2024-06-25 cs.LG 87%

All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining

Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, Jia Li

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted to KDD'24, August 25-29, 2024, Barcelona, Spain

2406.13873 2024-06-21 cs.AI 87%

A Pure Transformer Pretraining Framework on Text-attributed Graphs

Yu Song, Haitao Mao, Jiachen Xiao, Jingzhe Liu, Zhikai Chen, Wei Jin, Carl Yang, Jiliang Tang, Hui Liu

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2406.09282 2024-06-14 cs.CL cs.SD eess.AS 87%

On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Models

Jinchuan Tian, Yifan Peng, William Chen, Kwanghee Choi, Karen Livescu, Shinji Watanabe

专题命中预训练与数据：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2404.13847 2024-04-23 cs.CV cs.CL 87%

EventLens: Leveraging Event-Aware Pretraining and Cross-modal Linking Enhances Visual Commonsense Reasoning

Mingjie Ma, Zhihuan Yu, Yichao Ma, Guohui Li

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2404.00899 2024-04-02 cs.CL 87%

TM-TREK at SemEval-2024 Task 8: Towards LLM-Based Automatic Boundary Detection for Human-Machine Mixed Text

Xiaoyan Qu, Xiangfeng Meng

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 1st place at SemEval-2024 Task 8, Subtask C, to appear in SemEval-2024 proceedings

2404.00457 2024-04-02 cs.CL 87%

MetaIE: Distilling a Meta Model from LLM for All Kinds of Information Extraction Tasks

Letian Peng, Zilong Wang, Feng Yao, Zihan Wang, Jingbo Shang

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

2310.08754 2024-03-19 cs.LG 87%

Tokenizer Choice For LLM Training: Negligible or Crucial?

Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Schulze Buschhoff, Charvi Jain, Alexander Arno Weber, Lena Jurkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte Ostendorff, Samuel Weinbach, Rafet Sifa, Stefan Kesselheim, Nicolas Flores-Herr

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

2402.14270 2024-03-04 cs.LG 87%

Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization

Xuxi Chen, Zhendong Wang, Daouda Sow, Junjie Yang, Tianlong Chen, Yingbin Liang, Mingyuan Zhou, Zhangyang Wang

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments Preprint; updated reference and related works

2402.19204 2024-03-01 cs.CL 87%

PeLLE: Encoder-based language models for Brazilian Portuguese based on open data

Guilherme Lamartine de Mello, Marcelo Finger, and Felipe Serras, Miguel de Mello Carpi, Marcos Menon Jose, Pedro Henrique Domingues, Paulo Cavalim

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Comments 15 pages

2401.03321 2024-02-27 cs.CL 87%

PIXAR: Auto-Regressive Language Modeling in Pixel Space

Yintao Tai, Xiyang Liao, Alessandro Suglia, Antonio Vergari

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

2310.11716 2023-10-19 cs.CL 87%

Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning

Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Heng Huang, Jiuxiang Gu, Tianyi Zhou

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

2305.12182 2023-08-31 cs.CL 87%

Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages

Ayyoob Imani, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André F. T. Martins, François Yvon, Hinrich Schütze

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Comments ACL 2023

1. 预训练与数据 12429 篇

LexiMark: Robust Watermarking via Lexical Substitutions to Enhance Membership Verification of an LLM's Textual Training Data

Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models

LED: LLM Enhanced Open-Vocabulary Object Detection without Human Curated Data Generation

Safety Pretraining: Toward the Next Generation of Safe AI

Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling

ECG-Byte: A Tokenizer for End-to-End Generative Electrocardiogram Language Modeling

Pisces: An Auto-regressive Foundation Model for Image Understanding and Generation

FRAME: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy

Language Models are Universal Embedders

RiemannGFM: Learning a Graph Foundation Model from Riemannian Geometry

Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models

Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation

Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM's Nest

LLM-Net: Democratizing LLMs-as-a-Service through Blockchain-based Expert Networks

LLMic: Romanian Foundation Language Model

Synthetic Data Generation with LLM for Improved Depression Prediction

The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Merging

Beyond Fine-tuning: Unleashing the Potential of Continuous Pretraining for Clinical LLMs

All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining

A Pure Transformer Pretraining Framework on Text-attributed Graphs

On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Models

EventLens: Leveraging Event-Aware Pretraining and Cross-modal Linking Enhances Visual Commonsense Reasoning

TM-TREK at SemEval-2024 Task 8: Towards LLM-Based Automatic Boundary Detection for Human-Machine Mixed Text

MetaIE: Distilling a Meta Model from LLM for All Kinds of Information Extraction Tasks

Tokenizer Choice For LLM Training: Negligible or Crucial?

Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization

PeLLE: Encoder-based language models for Brazilian Portuguese based on open data

PIXAR: Auto-Regressive Language Modeling in Pixel Space

Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning

Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages