语言大模型 / LLM - arXivDaily 专题

2508.17256 2025-08-26 cs.LG cs.AI stat.ML 73%

Provable Generalization in Overparameterized Neural Nets

Aviral Dhingra

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 8 Pages

2508.15437 2025-08-26 cs.IR cs.AI cs.LG 73%

Test-time Corpus Feedback: From Retrieval to RAG

Mandeep Rathee, V Venktesh, Sean MacAvaney, Avishek Anand

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 1 figure

2409.08239 2025-08-21 cs.CL cs.AI 73%

Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources

Alisia Lupidi, Carlos Gemmell, Nicola Cancedda, Jane Dwivedi-Yu, Jason Weston, Jakob Foerster, Roberta Raileanu, Maria Lomeli

机构 * Meta Superintelligence labs（Meta超智能实验室）； University of Oxford（牛津大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2408.06569 2025-08-21 cs.CL cs.AI 73%

Social Debiasing for Fair Multi-modal LLMs

Harry Cheng, Yangyang Guo, Qingpei Guo, Ming Yang, Tian Gan, Weili Guan, Liqiang Nie

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Project page: https://github.com/xaCheng1996/Social_Debiasing_For_Fair_MLLMs

2504.16438 2025-08-20 cs.LG cs.AI cs.CR cs.DC 73%

POPri: Private Federated Learning using Preference-Optimized Synthetic Data

Charlie Hou, Mei-Yu Wang, Yige Zhu, Daniel Lazar, Giulia Fanti

机构 * Pittsburgh Supercomputing Center, Pittsburgh, USA（匹兹堡超级计算中心）； Department of ECE, Carnegie Mellon University, Pittsburgh, PA（电子工程系，卡内基梅隆大学）

专题命中预训练与数据：LLM(abstract)；preference optimization(abstract)；分类 cs.AI、cs.LG

Comments ICML 2025 camera-ready

2411.15734 2025-08-20 cs.CL cs.LG 73%

Development of Pre-Trained Transformer-based Models for the Nepali Language

Prajwal Thapa, Jinu Nyachhyon, Mridul Sharma, Bal Krishna Bal

机构 * Information and Language Processing Research Lab (ILPRL)（信息与语言处理研究实验室）； Kathmandu University（加德满都大学）

专题命中预训练与数据：language model(abstract)；instruction tuning(abstract)；分类 cs.CL、cs.LG

Journal ref https://aclanthology.org/2025.chipsal-1.2.pdf

2503.01307 2025-08-18 cs.CL cs.LG 73%

Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs

Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, Noah D. Goodman

机构 * Stanford University（斯坦福大学）； SynthLabs

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

2502.08512 2025-08-15 cs.CL cs.AI 73%

Measuring Diversity in Synthetic Datasets

Yuchang Zhu, Huizhe Zhang, Bingzhe Wu, Jintang Li, Zibin Zheng, Peilin Zhao, Liang Chen, Yatao Bian

机构 * School of Computer Science（计算机科学学院）； Engineering, Sun Yat-sen University（工程，中山大学）； School of Software Engineering（软件工程学院）； Sun Yat-sen University（中山大学）； Zhuhai Key Laboratory of Trusted Large Language Models（珠海可信大语言模型实验室）； Tencent AI Lab（腾讯AI实验室）； National University of Singapore（新加坡国立大学）； Shenzhen University（深圳大学）； Shanghai Jiao Tong University（上海交通大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICML 2025

2407.02596 2025-08-11 cs.CR cs.CL cs.LG 73%

Towards More Realistic Extraction Attacks: An Adversarial Perspective

Yash More, Prakhar Ganesh, Golnoosh Farnadi

机构 * McGill University（麦吉尔大学）； Mila（Mila研究所）

专题命中预训练与数据：language model(abstract)；prompting(abstract)；分类 cs.CL、cs.LG

Comments To appear in TACL

2505.07258 2025-08-11 cs.CL cs.AI 73%

No Query, No Access

Wenqiang Wang, Siyuan Liang, Yangshijie Zhang, Xiaojun Jia, Hao Lin, Xiaochun Cao

机构 * Sun Yat-sen University, Guangzhou, China（中山大学）； Lanzhou University, Gansu, China（兰州大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2508.04956 2025-08-08 cs.LG cs.AI 73%

MENDR: Manifold Explainable Neural Data Representations

Matthew Chen, Micky Nnamdi, Justin Shao, Andrew Hornback, Hongyun Huang, Ben Tamo, Yishan Zhong, Benoit Marteau, Wenqi Shi, May Dongmei Wang

专题命中预训练与数据：foundation model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2506.09251 2025-08-05 cs.CL cs.AI 73%

Extrapolation by Association: Length Generalization Transfer in Transformers

Ziyang Cai, Nayoung Lee, Avi Schwarzschild, Samet Oymak, Dimitris Papailiopoulos

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments 23 pages, 20 figures

2507.22250 2025-07-31 cs.LG cs.AI 73%

Using Scaling Laws for Data Source Utility Estimation in Domain-Specific Pre-Training

Oleksiy Ostapenko, Charles Guille-Escuret, Luke Kumar, Max Tian, Denis Kocetkov, Gopeshh Subbaraj, Raymond Li, Joel Lamy-Poirier, Sebastien Paquet, Torsten Scholak

机构 * ServiceNow Research（ServiceNow 研究所）； Mila — Quebec AI Institute（魁北克人工智能研究所）

专题命中预训练与数据：foundation model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2507.18583 2025-07-25 cs.IR cs.AI cs.CL 73%

DR.EHR: Dense Retrieval for Electronic Health Record with Knowledge Injection and Synthetic Data

Zhengyun Zhao, Huaiyuan Ying, Yue Zhong, Sheng Yu

机构 * Tsinghua University（清华大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Model and code released upon acceptance

2507.18055 2025-07-25 cs.CL cs.CR cs.LG 73%

Privacy-Preserving Synthetic Review Generation with Diverse Writing Styles Using LLMs

Tevin Atwal, Chan Nam Tieu, Yefeng Yuan, Zhan Shi, Yuhong Liu, Liang Cheng

机构 * Department of Computer Science and Engineering（计算机科学与工程系）； Santa Clara University（圣克拉拉大学）； eBay（eBay公司）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2507.16820 2025-07-24 cs.SI cs.AI cs.CL cs.DL 73%

Disaster Informatics after the COVID-19 Pandemic: Bibliometric and Topic Analysis based on Large-scale Academic Literature

Ngan Tran, Haihua Chen, Ana Cleveland, Yuhan Zhou

机构 * Department of Information Science, University of North Texas（信息科学系，北卡罗来纳州立大学）； Department of Data Science, University of North Texas（数据科学系，北卡罗来纳州立大学）

专题命中预训练与数据：LLM(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 36 pages, 14 figures, 5 tables

2503.02382 2025-07-24 cs.CL cs.AI 73%

An Efficient and Precise Training Data Construction Framework for Process-supervised Reward Model in Mathematical Reasoning

Wei Sun, Qianlong Du, Fuwei Cui, Jiajun Zhang

机构 * Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2411.05195 2025-07-23 cs.LG cs.CL cs.CV 73%

Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder

Siting Li, Pang Wei Koh, Simon Shaolei Du

机构 * University of Washington（华盛顿大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments ACL 2025; 19 pages, 3 figures

2507.13822 2025-07-21 cs.IR cs.AI cs.CL 73%

RAG-based Architectures for Drug Side Effect Retrieval in LLMs

Shad Nygren, Pinar Avci, Andre Daniels, Reza Rassol, Afshin Beheshti, Diego Galeano

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2507.10613 2025-07-16 cs.LG cs.AI 73%

Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs

Zhengyu Chen, Siqi Wang, Teng Xiao, Yudong Wang, Shiqi Chen, Xunliang Cai, Junxian He, Jingang Wang

机构 * Meituan Inc.（美团公司）； The University of Hong Kong（香港大学）； Pennsylvania State University（宾夕法尼亚州立大学）； Peking University（北京大学）； The Hong Kong University of Science and Technology（香港科学与技术大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2406.10268 2025-07-15 cs.AI cs.CL cs.HC 73%

Autograding Mathematical Induction Proofs with Natural Language Processing

Chenyan Zhao, Mariana Silva, Seth Poulsen

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2507.08665 2025-07-14 cs.CL cs.AI 73%

KELPS: A Framework for Verified Multi-Language Autoformalization via Semantic-Syntactic Alignment

Jiyao Zhang, Chengli Zhong, Hui Xu, Qige Li, Yi Zhou

机构 * School of Information Science and Technology, University of Science and Technology of China（信息科学与技术学院，中国科学技术大学）； USTC Knowledge Computing Lab（中国科学技术大学知识计算实验室）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by the ICML 2025 AI4MATH Workshop. 22 pages, 16 figures, 2 tables

2507.05121 2025-07-08 cs.IT cs.AI cs.CV cs.LG math.IT 73%

LVM4CSI: Enabling Direct Application of Pre-Trained Large Vision Models for Wireless Channel Tasks

Jiajia Guo, Peiwen Jiang, Chao-Kai Wen, Shi Jin, Jun Zhang

机构 * nsysu.edu.tw（国立交通大学）； seu.edu.cn（南京师范大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments This work has been submitted for possible publication

2402.00795 2025-07-02 cs.LG cs.AI 73%

LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law

Toni J. B. Liu, Nicolas Boullé, Raphaël Sarfati, Christopher J. Earls

机构 * Cornell University, USA（康奈尔大学）； Imperial College London, UK（伦敦帝国理工学院）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Journal ref EMNLP 2024

2502.18435 2025-07-01 cs.CL cs.IT cs.LG math.IT 73%

What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?

Yizhe Zhang, Richard Bai, Zijin Gu, Ruixiang Zhang, Jiatao Gu, Emmanuel Abbe, Samy Bengio, Navdeep Jaitly

机构 * Apple（苹果公司）

专题命中预训练与数据：LLM(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 10 pages for the main text

2506.17847 2025-06-24 cs.LG cs.AI 73%

A Comparative Study of Open-Source Libraries for Synthetic Tabular Data Generation: SDV vs. SynthCity

Cristian Del Gobbo

机构 * Cristian Del Gobbo（独立研究者）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 23 Pages, 5 figures, and 6 tables

2506.00418 2025-06-24 cs.CL cs.AI 73%

Dual Debiasing for Noisy In-Context Learning for Text Generation

Siqi Liang, Sumyeong Ahn, Paramveer S. Dhillon, Jiayu Zhou

机构 * University of Michigan Ann Arbor（密歇根大学安娜堡分校）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by 2025 ACL Findings

2506.17080 2025-06-23 cs.CL cs.AI 73%

Tower+: Bridging Generality and Translation Specialization in Multilingual LLMs

Ricardo Rei, Nuno M. Guerreiro, José Pombal, João Alves, Pedro Teixeirinha, Amin Farajian, André F. T. Martins

机构 * Unbabel ； Instituto de Telecomunicações（电信研究所）； Instituto Superior Técnico & Universidade de Lisboa (Lisbon ELLIS Unit)（里斯本大学技术学院及里斯本大学（里斯本ELLIS单位））； MICS, CentraleSupélec, Université Paris-Saclay（MICS，CentraleSupélec，巴黎萨克雷大学）

专题命中预训练与数据：pretraining(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2506.03147 2025-06-23 cs.CV cs.AI cs.CL 73%

UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation

Bin Lin, Zongjian Li, Xinhua Cheng, Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangbo Yu, Shaodong Wang, Yunyang Ge, Yatian Pang, Li Yuan

机构 * Peking University（北京大学）； Shenzhen Graduate School（深圳研究生院）； Peng Cheng Laboratory（鹏城实验室）； Rabbitpre AI

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2506.15271 2025-06-19 cs.LG cs.AI 73%

Unlocking Post-hoc Dataset Inference with Synthetic Data

Bihe Zhao, Pratyush Maini, Franziska Boenisch, Adam Dziedzic

机构 * cispa（CISPA 河岸信息安全中心）； cmu（卡内基梅隆大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ICML 2025

1. 预训练与数据 12460 篇

Provable Generalization in Overparameterized Neural Nets

Test-time Corpus Feedback: From Retrieval to RAG

Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources

Social Debiasing for Fair Multi-modal LLMs

POPri: Private Federated Learning using Preference-Optimized Synthetic Data

Development of Pre-Trained Transformer-based Models for the Nepali Language

Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs

Measuring Diversity in Synthetic Datasets

Towards More Realistic Extraction Attacks: An Adversarial Perspective

No Query, No Access

MENDR: Manifold Explainable Neural Data Representations

Extrapolation by Association: Length Generalization Transfer in Transformers

Using Scaling Laws for Data Source Utility Estimation in Domain-Specific Pre-Training

DR.EHR: Dense Retrieval for Electronic Health Record with Knowledge Injection and Synthetic Data

Privacy-Preserving Synthetic Review Generation with Diverse Writing Styles Using LLMs

Disaster Informatics after the COVID-19 Pandemic: Bibliometric and Topic Analysis based on Large-scale Academic Literature

An Efficient and Precise Training Data Construction Framework for Process-supervised Reward Model in Mathematical Reasoning

Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder

RAG-based Architectures for Drug Side Effect Retrieval in LLMs

Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs

Autograding Mathematical Induction Proofs with Natural Language Processing

KELPS: A Framework for Verified Multi-Language Autoformalization via Semantic-Syntactic Alignment

LVM4CSI: Enabling Direct Application of Pre-Trained Large Vision Models for Wireless Channel Tasks

LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law

What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?

A Comparative Study of Open-Source Libraries for Synthetic Tabular Data Generation: SDV vs. SynthCity

Dual Debiasing for Noisy In-Context Learning for Text Generation

Tower+: Bridging Generality and Translation Specialization in Multilingual LLMs

UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation

Unlocking Post-hoc Dataset Inference with Synthetic Data