语言大模型 / LLM

How to inject knowledge efficiently? Knowledge Infusion Scaling Law for Pre-training Large Language Models

Kangtao Lv, Haibin Chen, Yujin Yuan, Langming Liu, Shilei Liu, Yongwei Wang, Wenbo Su, Bo Zheng

机构 * Zhejiang University（浙江大学） ； Taobao & Tmall Group of Alibaba（阿里巴巴集团） ； Shanghai AI Laboratory（上海人工智能实验室） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

GALLa: Graph Aligned Large Language Models for Improved Source Code Understanding

Ziyin Zhang, Hang Yu, Shijie Li, Peng Di, Jianguo Li, Rui Wang

机构 * Ant Group（蚂蚁集团） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025

Once Upon a Time: Interactive Learning for Storytelling with Small Language Models

Jonas Mayer Martins, Ali Hamza Bashir, Muhammad Rehan Khalid, Lisa Beinborn

机构 * University of Göttingen, Institute of Computer Science（哥廷根大学计算机科学学院） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；small language model(title)；large language model(abstract)；pretraining(abstract)

Comments EMNLP 2025, BabyLM Challenge; 16 pages, 6 figures

Persona-Based Synthetic Data Generation Using Multi-Stage Conditioning with Large Language Models for Emotion Recognition

Keito Inoshita, Rushia Harada

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Models

Haokun Chen, Sebastian Szyller, Weilin Xu, Nageen Himayat

机构 * Ludwig Maximilian University of Munich（慕尼黑路德维希-马克西米利安大学） ； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心） ； Aalto University（艾尔沃斯大学） ； Intel Labs（英特尔实验室） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2025 Findings

Robust Detection of Watermarks for Large Language Models Under Human Edits

Xiang Li, Feng Ruan, Huiyuan Wang, Qi Long, Weijie J. Su

机构 * University of Pennsylvania（宾夕法尼亚大学） ； Northwestern University（西北大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments To appear in Journal of the Royal Statistical Society: Series B

ALOPE: Adaptive Layer Optimization for Translation Quality Estimation using Large Language Models

Archchana Sindhujan, Shenbin Qian, Chan Chi Chun Matthew, Constantin Orasan, Diptesh Kanojia

机构 * Institute for People-Centred AI and Centre for Translation Studies, School of Computer Science and Electronic Engineering, University of Surrey（以人为本的人工智能研究所和翻译研究中心，计算机科学与电子工程学院，萨里大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted to COLM 2025 Conference

Do Large Language Models Have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMs

Yanzhu Guo, Simone Conia, Zelin Zhou, Min Li, Saloni Potdar, Henry Xiao

机构 * Apple（苹果公司） ； Inria Paris（巴黎国家信息与自动化研究所） ； École Polytechnique（巴黎高等理工学院） ； Sapienza University of Rome（罗马萨皮恩扎大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025

Promptomatix: An Automatic Prompt Optimization Framework for Large Language Models

Rithesh Murthy, Ming Zhu, Liangwei Yang, Jielin Qiu, Juntao Tan, Shelby Heinecke, Caiming Xiong, Silvio Savarese, Huan Wang

机构 * Salesforce AI Research（Salesforce AI研究院） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Synthetic Data Generation for Phrase Break Prediction with Large Language Model

Hoyeon Lee, Sejung Son, Ye-Eun Kang, Jong-Hwan Kim

机构 * NAVER Cloud ； NHNSouth Korea ； Yale University（耶鲁大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted at Interspeech 2025

Predictable Scale: Part II, Farseer: A Refined Scaling Law in Large Language Models

Houyi Li, Wenzhen Zheng, Qiufeng Wang, Zhenyu Ding, Haoying Wang, Zili Wang, Shijie Xuyang, Ning Ding, Shuigeng Zhou, Xiangyu Zhang, Daxin Jiang

机构 * Fudan University（复旦大学） ； Xi’an Jiaotong University（西安交通大学） ； Megvii Technology（科大讯飞） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments 34

Model Inversion Attacks on Llama 3: Extracting PII from Large Language Models

Sathesh P. Sivashanmugam

机构 * Senior Software Engineer American Express（美国运通高级软件工程师） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling

Tianhao Chen, Xin Xu, Zijing Liu, Pengxiang Li, Xinyuan Song, Ajay Kumar Jaiswal, Fan Zhang, Jishan Hu, Yang Wang, Hao Chen, Shizhe Diao, Shiwei Liu, Yu Li, Lu Yin, Can Yang

机构 * The Hong Kong University of Science and Technology（香港科技大学） ； International Digital Economy Academy（国际数字经济学院） ； Dalian University of Technology（大连理工大学） ； Emory University（埃默里大学） ； University of Texas at Austin（德克萨斯大学奥斯汀分校） ； NVIDIA（英伟达） ； University of Oxford（牛津大学） ； University of Surrey（萨里大学） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generation

Deyu Zou, Yongqiang Chen, Mufei Li, Siqi Miao, Chenxi Liu, Bo Han, James Cheng, Pan Li

机构 * The Chinese University of Hong Kong（香港中文大学） ； Georgia Institute of Technology（佐治亚理工学院） ； Hong Kong Baptist University（香港 Baptist大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Learning Instruction-Following Policies through Open-Ended Instruction Relabeling with Large Language Models

Zhicheng Zhang, Ziyan Wang, Yali Du, Fei Fang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Under Review

Is There a Case for Conversation Optimized Tokenizers in Large Language Models?

Raquel Ferrando, Javier Conde, Gonzalo Martínez, Pedro Reviriego

机构 * ETSI de Telecomunicación Universidad Politécnica de Madrid（电信工程学院马德里理工大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

GenDFIR: Advancing Cyber Incident Timeline Analysis Through Retrieval Augmented Generation and Large Language Models

Fatma Yasmine Loumachi, Mohamed Chahine Ghanem, Mohamed Amine Ferrag

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments 24 pages V5.3

AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling

Yuheng Huang, Jiayang Song, Qiang Hu, Felix Juefei-Xu, Lei Ma

机构 * The University of Tokyo（东京大学） ； University of Alberta（阿尔伯塔大学） ； Tianjin University（天津大学） ； New York University（纽约大学） ； The University of Tokyo, Japan（东京大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments To appear in ACM Transactions on Software Engineering and Methodology (2025)

TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining

Jeffrey Li, Mohammadreza Armandpour, Iman Mirzadeh, Sachin Mehta, Vaishaal Shankar, Raviteja Vemulapalli, Samy Bengio, Oncel Tuzel, Mehrdad Farajtabar, Hadi Pouransari, Fartash Faghri

机构 * University of Washington（华盛顿大学） ； Apple（苹果公司） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

Comments Code available at: https://github.com/apple/ml-tic-lm

SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition

Shuangrui Ding, Zihan Liu, Xiaoyi Dong, Pan Zhang, Rui Qian, Junhao Huang, Conghui He, Dahua Lin, Jiaqi Wang

机构 * The Chinese University of Hong Kong（中国香港大学） ； Beihang University（北航） ； Shanghai AI Laboratory（上海人工智能实验室） ； CPII under InnoHK（创新香港下的CPII） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 main. project page: https://pjlab-songcomposer.github.io/ code: https://github.com/pjlab-songcomposer/songcomposer

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning

Wanyun Xie, Francesco Tonin, Volkan Cevher

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments ICML 2025

FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation

Shaolin Zhu, Tianyu Dong, Bo Li, Deyi Xiong

机构 * College of Intelligence and Computing, Tianjin University（智能与计算学院，天津大学） ； School of Software, Tsinghua University（软件学院，清华大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Neural Thermodynamic Laws for Large Language Model Training

Ziming Liu, Yizhou Liu, Jeff Gore, Max Tegmark

机构 * Massachusetts Institute of Technology（麻省理工学院） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 10 figures

Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning

Yoichi Ishibashi, Taro Yano, Masafumi Oyamada

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

Advanced Crash Causation Analysis for Freeway Safety: A Large Language Model Approach to Identifying Key Contributing Factors

Ahmed S. Abdelrahman, Mohamed Abdel-Aty, Samgyu Yang, Abdulrahman Faden

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Aligning Large Language Models with Healthcare Stakeholders: A Pathway to Trustworthy AI Integration

Kexin Ding, Mu Zhou, Akshay Chaudhari, Shaoting Zhang, Dimitris N. Metaxas

机构 * Rutgers University（罗格斯大学） ； Stanford University（斯坦福大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactions

Roshan Patel, Saeed Moayedpour, Louis De Lescure, Lorenzo Kogler-Anele, Alan Cherney, Sven Jager, Yasser Jangjou

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.AI、cs.LG

DataMan: Data Manager for Pre-training Large Language Models

Ru Peng, Kexin Yang, Yawen Zeng, Junyang Lin, Dayiheng Liu, Junbo Zhao

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments ICLR2025 paper

SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models

Anil Ramakrishna, Yixin Wan, Xiaomeng Jin, Kai-Wei Chang, Zhiqi Bu, Bhanukiran Vinzamuri, Volkan Cevher, Mingyi Hong, Rahul Gupta

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Are Large Language Models Memorizing Bug Benchmarks?

Daniel Ramos, Claudia Mamede, Kush Jain, Paulo Canelas, Catarina Gamboa, Claire Le Goues

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG